AI Rate Limits Explained

AI rate limits control how much API traffic an account, project, organization, or model can process during a period of time.

Implementation choices

They are different from a model's context window.

A context limit controls how much information can be handled within an individual interaction.

Test against provider output

A rate limit controls how quickly requests or tokens can be processed across multiple interactions.

Providers commonly express rate limits using measurements such as:

Keep production limits in view

RPM: requests per minute

TPM: tokens per minute

Practical checks before you proceed

RPD: requests per day

Other provider-specific limits may also apply.

Practical checks before you proceed

For example, if your account has a limit of 500 requests per minute and your application suddenly sends 800, some requests may be throttled even though every individual prompt is small.

Likewise, a token-per-minute limit can be reached with fewer requests when each prompt is extremely large.

Practical checks before you proceed

OpenAI's documented GPT-5 limits vary by account usage tier. Its model documentation, for example, currently shows increasing RPM and TPM allowances across API tiers.

Production systems should handle rate-limit responses gracefully.

Practical checks before you proceed

Useful techniques include exponential backoff, retry queues, concurrency control, request batching where appropriate, and traffic distribution across time.

Do not blindly retry immediately. Hundreds of clients retrying simultaneously can make the problem worse.

Practical checks before you proceed

Monitor both request volume and token throughput so you can identify limits before they become user-facing failures.

For high-volume AI systems, rate limits are not merely API errors—they are an important capacity-planning constraint that should be considered alongside model cost, latency, and context size.