GitHub

rate_limiter

API reference for rate_limiter.

Rate limits per key (Effect's effect/persistence/RateLimiter)::

limiter = yield from RateLimiter()
result = yield from limiter.consume(key="user:1", limit=100, window="1 minute")
# fails with RateLimiterError(RateLimitExceeded) once over the limit

consume charges tokens to a key by a "fixed-window" or "token-bucket" algorithm and, over the limit, fails or gives the delay to wait (on_exceeded="delay"); sleep waits that delay, and make_with_rate_limiter gives a function that limits any effect. adaptive_consume / adaptive_feedback learn a server's limit from its 429 responses.

The counters live in a RateLimiterStore: layer_store_memory, or layer_store_redis (shared by every process using that Redis).

Values

AttributeTypeDescription
RATE_LIMITER_ERROR_REASONUnionSchema[RateLimitExceeded | RateLimitStoreError, Mapping[str, object], Never, Never]

The schema of every reason.

layerLayer[RateLimiter, Never, RateLimiterStore]

Provides RateLimiter on the RateLimiterStore.

makeEffect[RateLimiterService, Never, RateLimiterStore]

A RateLimiter on the RateLimiterStore.

make_with_rate_limiterEffect[WithRateLimiter, Never, RateLimiter]

The RateLimiter's with_rate_limiter function::

with_rate_limiter = yield from rate_limiter.make_with_rate_limiter yield from call_api.pipe(with_rate_limiter(key="api", limit=10, window="1 minute"))

layer_store_memoryLayer[RateLimiterStore]

Counters in memory, fresh each time the layer is built.

make_memory_storeEffect[MemoryRateLimiterStore, Never, Scope]

A store in memory whose expired keys a fiber in the scope sweeps away every minute; its size counts the keys holding state.

Types

TypeDefinitionDescription
RateLimiterErrorReasontype RateLimiterErrorReason = RateLimitExceeded | RateLimitStoreError
Algorithmtype Algorithm = Literal['fixed-window', 'token-bucket']

"fixed-window": limit tokens per window, each token reserving window / limit of it. "token-bucket": a bucket of limit tokens refilled one every window / limit.

OnExceededtype OnExceeded = Literal['delay', 'fail']

Over the limit, "fail" with RateLimitExceeded (charging nothing), or reserve the tokens and give the delay to wait first.

AdaptivePhasetype AdaptivePhase = Literal['inactive', 'cooldown', 'learning', 'learned']

Where an adaptive key is: no 429 seen ("inactive"), waiting out a Retry-After ("cooldown"), counting what the server accepts ("learning"), or pacing at the limit it learned ("learned").

Classes

Functions

sleep

sleep(limiter: RateLimiterService, /) -> Sleep
sleep(limiter: RateLimiterService, /, *, key: str, limit: float, window: DurationInput, algorithm: Algorithm = 'fixed-window', tokens: float = 1) -> Effect[ConsumeResult, RateLimiterError]

Consume in "delay" mode and sleep the delay, so the call goes on once it is under the limit; without options, the function doing that for limiter.

ParameterTypeDescription
limiter
key= None
limit= 0
window= '0 millis'
algorithm= 'fixed-window'
tokens= 1
@overload
def sleep(limiter: RateLimiterService, /) -> Sleep: ...
@overload
def sleep(
    limiter: RateLimiterService,
    /,
    *,
    key: str,
    limit: float,
    window: DurationInput,
    algorithm: Algorithm = "fixed-window",
    tokens: float = 1,
) -> Effect[ConsumeResult, RateLimiterError]: ...
def sleep(
    limiter: RateLimiterService,
    /,
    *,
    key: str | None = None,
    limit: float = 0,
    window: DurationInput = "0 millis",
    algorithm: Algorithm = "fixed-window",
    tokens: float = 1,
) -> Sleep | Effect[ConsumeResult, RateLimiterError]:
    """Consume in ``"delay"`` mode and sleep the delay, so the call goes on
    once it is under the limit; without options, the function doing that
    for ``limiter``."""
    if key is None:

        def partial(
            *,
            key: str,
            limit: float,
            window: DurationInput,
            algorithm: Algorithm = "fixed-window",
            tokens: float = 1,
        ) -> Effect[ConsumeResult, RateLimiterError]:
            return sleep(
                limiter, key=key, limit=limit, window=window, algorithm=algorithm, tokens=tokens
            )

        return partial

    def wait(result: ConsumeResult) -> Effect[ConsumeResult]:
        return succeed(result) if result.delay.is_zero() else clock_sleep(result.delay).as_(result)

    return limiter.consume(
        key=key,
        limit=limit,
        window=window,
        algorithm=algorithm,
        on_exceeded="delay",
        tokens=tokens,
    ).flat_map(wait)

layer_store_redis

layer_store_redis(*, prefix='ratelimiter:') -> Layer[RateLimiterStore, Never, Redis]

Counters in Redis, under prefix.

ParameterTypeDescription
prefixstr = 'ratelimiter:'

Returns Layer[RateLimiterStore, Never, Redis]

def layer_store_redis(*, prefix: str = "ratelimiter:") -> Layer[RateLimiterStore, Never, Redis]:
    """Counters in Redis, under ``prefix``."""
    return layer_effect(RateLimiterStore, make_store_redis(prefix=prefix))

layer_store_redis_config

layer_store_redis_config(options) -> Layer[RateLimiterStore, ConfigError, Redis]

layer_store_redis with its options read from the ConfigProvider: a Config of them, or a mapping from option names to their Config (Effect's Config.Wrap).

ParameterTypeDescription
optionsConfig[Mapping[str, Any]] | Mapping[str, Any]

Returns Layer[RateLimiterStore, ConfigError, Redis]

def layer_store_redis_config(
    options: Config[Mapping[str, Any]] | Mapping[str, Any],
) -> Layer[RateLimiterStore, ConfigError, Redis]:
    """``layer_store_redis`` with its options read from the
    ``ConfigProvider``: a ``Config`` of them, or a mapping from option names
    to their ``Config`` (Effect's ``Config.Wrap``)."""
    read: Effect[Mapping[str, Any], ConfigError] = unwrap(options)
    return layer_effect(RateLimiterStore, read.flat_map(lambda fields: make_store_redis(**fields)))

make_store_redis

make_store_redis(*, prefix='ratelimiter:') -> Effect[RateLimiterStoreService, Never, Redis]

A store in the Redis service: a rate-limit key's state under <prefix><key>, its token-bucket refill time under ...:refill and its adaptive state under ...:adaptive, with % and : in the key percent-encoded so different keys never share Redis state.

ParameterTypeDescription
prefixstr = 'ratelimiter:'

Returns Effect[RateLimiterStoreService, Never, Redis]

def make_store_redis(
    *, prefix: str = "ratelimiter:"
) -> Effect[RateLimiterStoreService, Never, Redis]:
    """A store in the ``Redis`` service: a rate-limit key's state under
    ``<prefix><key>``, its token-bucket refill time under ``...:refill`` and
    its adaptive state under ``...:adaptive``, with ``%`` and ``:`` in the
    key percent-encoded so different keys never share Redis state."""

    def build() -> Generator[Effect[Any, Never, Any], Any, RateLimiterStoreService]:
        redis = yield from Redis()
        return RedisRateLimiterStore(redis, prefix)

    return gen(build)