Rate Limiter

Related: vector-database | postgresql-extensibility

Limits how many requests a user/service can make in a time window.

Algorithms

1. Token Bucket

• Bucket holds capacity tokens, refill at rate per sec
• Each request consumes 1 token. If empty -> reject (429)
• Allows bursts up to capacity.

2. Leaky Bucket

• Requests enter queue, leak out at fixed rate
• Smooths traffic, no bursts
• Good for outbound rate limiting.

3. Fixed Window Counter

• counter++ per window (e.g., 1 min). Simple but border burst problem.
• Fix: Sliding window log / sliding window counter (Redis).

Redis Example (Token Bucket)

# pseudo
tokens = redis.get("bucket:user:1")
if tokens > 0:
    redis.decr("bucket:user:1")
    allow()
else:
    reject(429)

With refill via SET bucket 10 EX 60 or Lua script for atomic refill.

When to use

• API Gateway (Kong, Nginx), per-IP limits
• Linked with vector-database for LLM API rate limiting

Ref: Cloudflare Rate Limiting

Last updated: 2026-08-23