Rate Limiting

A rate limit caps how many times a task may run. The cap can be over a stretch of time — say twenty runs a minute per user — or over what is running right now — say one at a time per user. Anything past the cap is rejected instead of executed.

The framework provides both kinds. A task can declare either or both:

• Request rate — at most N per window.
• Concurrency — at most N running at the same time.

Limits are declared in a task's options() and checked before it runs. By default a request past the cap is rejected with a 429 and is not charged; enforce lets you take the verdict and decide for yourself instead.

Nothing about a limit is registered or stored — limit and windowSeconds are arguments you pass on every call.

API

// one entry per limit — the key is the discriminator, e.g. `user=123`
interface RateSpec        { key: string; limit: number; windowSeconds: number; label?: string }
interface ConcurrencySpec { key: string; limit: number; label?: string }

// what a call gives back
interface RateLimitDecision {
    overLimit: boolean;         // true if ANY spec in the call is over
    retryAfterSeconds: number;  // max reset among the over-limit keys; 0 when allowed
    degraded: boolean;          // the limiter could not answer, so nothing was checked
    results: RateLimitResult[]; // per-key detail; empty when degraded
}

flow.rateLimit().consume(specs: RateSpec[],        opts?: { enforce?: boolean }): Promise<RateLimitDecision>
flow.rateLimit().acquire(specs: ConcurrencySpec[], opts?: { enforce?: boolean }): Promise<RateLimitDecision | null>

Prerequisites

Rate limits are declared in options(), so the task needs enablePreflight:

tasks:
  - name: example-task
    publish: true
    enablePreflight: true

info

Because options() runs for every execution of the task, the limit applies to it as a root task and as a sub-task — flow.executeTask(), flow.streamTask() and deferred sub-tasks all count against the same limits.

Request rate

await flow.rateLimit().consume([
    { key: `user=${flow.meta().userId()}`, limit: 20, windowSeconds: 60 },
]);

Keys can be built from flow.meta(), which exposes the request's headers inside options() — userId(), countryCode(), touchpoint() and so on.

Counts one hit against the key. When over, it throws RateLimitExceededError — a 429 with reason: 'rate_limited'.

Several limits in one call are evaluated all-or-nothing: if any would exceed, nothing is counted.

await flow.rateLimit().consume([
    { key: `user=${uid}`, limit: 20,   windowSeconds: 60 },
    { key: 'global',      limit: 5000, windowSeconds: 60 },
]);

warning

consume() and acquire() must be awaited.

Concurrency

await flow.rateLimit().acquire([
    { key: `user=${flow.meta().userId()}`, limit: 1 },
]);

Reserves one in-flight slot per key. The framework releases it when the task reaches a terminal status — you never release manually and never state a duration. The slot survives flow.defer().

warning

A slot is held for at most one hour. The service reclaims it after that even if the task is still running, so a task that can run longer than an hour cannot be reliably limited this way.

enforce

enforce decides whether an over-limit result throws. It defaults to true.

It does not change what is counted: an allowed hit is always counted, a rejected one never is, and a rejected acquire() never takes a slot.

	over limit	under limit
enforce: true (default)	throws RateLimitExceededError	returns the decision
enforce: false	returns the decision	returns the decision

const decision = await flow.rateLimit().consume(specs, { enforce: false });
if (decision.overLimit) {
    return { monetization: { toolId: 'example_paid' } };
}

On the /options preview

The public /options endpoint is a quote: it must not spend anything, so nothing is counted and no slot is taken there.

• consume() evaluates the same limits read-only and returns the real verdict — the same overLimit the execution would get.
• acquire() returns null. A slot has no meaningful preview value, so there is no verdict to give.
• Neither throws, whatever enforce says.

So acquire() is typed RateLimitDecision | null and needs a null check if you branch on it:

const decision = await flow.rateLimit().acquire(specs, { enforce: false });
if (decision !== null) {
    // this run is not a quote
}

What reaches the client

By default a rejection carries only the retry hint:

{ "reason": "rate_limited", "details": { "retryAfterSeconds": 42 } }

Adding a label to a spec is what makes that limit identifiable to the caller. It appears in two places: the rejection's details, and the /options response.

await flow.rateLimit().consume([
    { key: `free:user=${uid}`, limit: 3, windowSeconds: THIRTY_DAYS, label: 'free_tries' },
    { key: `abuse:ip=${ip}`,   limit: 100, windowSeconds: 60 },
]);

{ "reason": "rate_limited", "details": { "retryAfterSeconds": 42, "label": "free_tries", "limit": 3 } }

{ "status": "success", "response": {
    "credits": 12,
    "rateLimits": [
      { "label": "free_tries", "overLimit": true, "limit": 3, "remaining": 0, "resetSeconds": 42 }
    ]
} }

Without a label, a limit contributes nothing to either — the caller learns only that something was hit. The key is never sent in any case.

When the limiter cannot answer

If the limiter is unreachable the call does not throw. It returns degraded: true, which means the limit was not checked — not that it was checked and passed:

const decision = await flow.rateLimit().consume(specs, { enforce: false });
if (decision.degraded) {
    throw new PicsArtError('service_unavailable', 'Cannot verify quota', 503);
}

The default is to continue: an outage degrades limiting, not execution.

Things to know

• Attempts are counted, not successes. A task that fails still spends its allowance.
• Fixed window. The window starts on the first hit against a key and the whole allowance refreshes at once — not a rolling "N in any 24h", so a burst can span the reset.
• limit changes apply immediately, windowSeconds does not. The window is set when the counter is created and ignored until it drains. Change the key if a new window must apply now.
• A Kafka redelivery counts twice.
• Your task command is prefixed automatically — write only the discriminator.
• The key decides who shares a counter — a user id, not a prompt. One that changes every request gets a fresh allowance every request, so nothing is ever limited.
• Keys allow only A-Z a-z 0-9 . _ = @ | / : -.