Skip to content

Rate limits

Reads and downloads have separate budgets, so metadata polling and body fetching do not compete.

Both APIs rate-limit. Over budget answers 429.

Separate budgets#

Limits are not one pool. Each class of request has its own, so a client that reads often and downloads rarely is not slowed down by its own reads.

Roost
general

Every JSON request: browsing, the catalog, program records, requirements, the version check.

download

The three download routes. Separate from the JSON budget on purpose, because a client fetching metadata constantly and bodies occasionally should not be throttled by the first.

Accounts
general

Discovery, keys and the permission catalog.

token

The token endpoint. Tight enough that a device-flow client which ignores interval will notice.

How they are counted#

Per address for an anonymous caller, and per client or account where there is one.

That matters for a client behind a busy shared address: it shares a budget with everything else behind it, which is one more reason to poll politely rather than aggressively.

Handling a 429#

  1. Stop

    Do not retry immediately. A tight retry loop is what turned a warning into a block.

  2. Respect Retry-After if it is there

    Wait at least that long.

  3. Back off exponentially otherwise

    Double the wait each time, with a ceiling, and add jitter so many clients do not retry in lockstep.

  4. Give up eventually

    A client refused five times in a row should surface it rather than keep going.

delay = 1s
for attempt in 1..5:
    answer = request()
    if answer.status != 429: return answer
    sleep(delay + random(0, delay/2))
    delay = min(delay * 2, 60s)
fail

Staying under them#

Use the bulk version check

One request for 32 addresses beats 32 requests.

Cache the catalog

Categories, languages and sort orders barely change. Only the counts move.

Compare checksums before downloading

They are on the record, on every requirement and in the download headers precisely so you do not have to fetch a body to find out you already have it.

Honour the device-flow interval

And increase it on slow_down rather than resetting it.

Keeping a calculator current

The bulk check that turns a whole calculator into one request.

Read on
CEagle documentation