Reference
Rate limits
How A8 limits requests and tokens per organization, the headers that report the limits, and how to raise them.
This page as Markdown: https://u22a8.ai/docs/rate-limits.md. The index of every docs page: https://u22a8.ai/llms.txt.
A rate limit caps how much an organization sends to A8 in a minute. Each organization has two: requests per minute (RPM) and input tokens per minute (TPM). A request over either returns 429.
§1How limits work
- Per organization. Every key and every interface of an organization draws on the same limits.
- Input tokens. TPM counts the tokens A8 reads, as
usagereports them. The output count is always0. - Every call counts. Judging, teaching and retracting each count as one request. A call over MCP counts the same as one over HTTP.
- A rolling minute. The limits refill continuously, so a burst at the start of a minute draws on the same budget as an even spread.
§2Tiers
| Tier | Qualification | RPM | TPM |
|---|---|---|---|
| Preview | Every organization in the A8 preview | TBD (requests per minute) | TBD (tokens per minute) |
§3Response headers
Every response carries the rate-limit headers the OpenAI API sends, under the same names, on every interface. Clients and gateways that read them from OpenAI read them from A8 unchanged.
| Header | Meaning |
|---|---|
x-ratelimit-limit-requests | The organization’s RPM |
x-ratelimit-limit-tokens | The organization’s TPM |
x-ratelimit-remaining-requests | Requests left in the current minute |
x-ratelimit-remaining-tokens | Input tokens left in the current minute |
x-ratelimit-reset-requests | Time until the request budget refills, as a duration such as 1s |
x-ratelimit-reset-tokens | Time until the token budget refills, as a duration such as 12s |
retry-after | On a 429: the seconds to wait before retrying |
x-ratelimit-* headers are on every response; retry-after is on a 429.The SDKs retry a 429 twice by default, waiting for retry-after each time. To read the headers yourself, ask the SDK for the raw response:
raw = client.chat.completions.with_raw_response.create(**request)
print(raw.headers["x-ratelimit-remaining-requests"]) # left this minute
resp = raw.parse() # the usual response object
const { data: resp, response } = await client.chat.completions
.create(request)
.withResponse();
console.log(response.headers.get("x-ratelimit-remaining-requests"));
§4Stay within the limits
- Spread a batch. A regression suite of many subjects runs within RPM when its requests are spread across the minute.
- Pace by the headers.
x-ratelimit-remaining-tokenssays how much of the minute’s budget is left. - Reuse verdicts. The same request against the same pinned version returns the same verdict. A suite that stores verdicts by request sends each request once.
§5Raise your limits
Write to [email protected] from an owner or admin of the organization. Name the organization, the RPM and TPM it needs, and the workload behind them.