Summary: A 429 Too Many Requests error is the first thing that breaks when an LLM app meets real load, and OpenRouter published a tutorial on 11/11/2026 showing that the default retry behavior in the OpenAI and Anthropic SDKs often isn’t enough. In their test, a 60-prompt batch lost 23 requests to rate limits even after the SDK’s retries. The fix is to read the headers and metadata the error already carries, then choose between waiting out the window, falling back to another model, or letting another provider serve the same model.
What the error is actually telling you
A 429 means you sent more requests or tokens than the current window allows. The useful part is that the error says whose limit you hit. On OpenRouter, error.metadata.limit_source distinguishes a free-tier per-minute cap (openrouter_free_tier_per_minute) from an upstream provider running out of capacity (upstream_provider_shared_pool). The free-tier error also carries three headers: X-RateLimit-Limit (the cap), X-RateLimit-Remaining (what’s left), and X-RateLimit-Reset, a Unix timestamp in milliseconds marking when a retry can succeed. There’s also a remedy_hint written for humans reading logs — OpenRouter’s advice is to branch on limit_source in code, not on hint text.
OpenAI and Anthropic send the same information under different names. OpenAI’s x-ratelimit-reset-requests is a duration like 1s, and both providers can send Retry-After in seconds. One trap worth knowing: some 429s are about money, not traffic. OpenAI and Anthropic both return a 429 when you hit a monthly spend cap, and no amount of retrying fixes that until the cap resets. On OpenRouter, credit problems come back as 402 instead.
Why 38 requests got through a limit of 20
The OpenRouter team sent 40 concurrent requests to poolside/laguna-s-2.1:free, a model with a stated limit of 20 requests per minute. Thirty-eight succeeded; two came back as 429s from the upstream provider’s shared pool. Limiters don’t count the way you’d guess — requests arriving at the same moment can slip through before the count catches up, and most limiters don’t reset cleanly at the top of the minute. Anthropic uses a token bucket that refills continuously, and OpenAI can return a 429 with the code slow_down when traffic ramps too fast even while you’re under your per-minute limits.
Across 161 more test requests in various patterns, more than 20 requests succeeded in every minute tested. So treat the stated limit as the point where 429s can start, not an exact count. And the free-model cap is account-wide: one per-minute limiter covers all free models on an account, so switching to a different free model mid-window can still get you the same 429.
Default retries quietly drop requests
This is the number that should make you check your own error handling. OpenRouter ran 60 prompts, 10 at a time, through the OpenAI Python SDK (version 3.26.1) with default settings against a per-minute limit. The SDK sent 112 HTTP requests for those 60 prompts, retried each 429 twice, and gave up within about 2 seconds per request. Twenty-three requests still failed. Meanwhile, the window reset 4 to 8 seconds after the SDK gave up on each one — the requests died just before they would have succeeded.
The problem is mechanical: the SDKs retry twice with a short backoff starting at 0.5 seconds and never waiting more than 8, and they follow Retry-After when the server sends it. But OpenRouter’s free-tier 429 sends X-RateLimit-Reset instead, which the SDKs don’t read, so both retries landed inside the same full minute. Raising max_retries would eventually get through, but it means firing requests into a full window every few seconds — and per OpenAI’s documentation, failed requests still count toward your per-minute limit.
Three fixes, three different costs
The tutorial fixes the same batch three ways, each with a real trade-off.
Wait for the reset time. Read Retry-After first (it’s the server’s direct instruction), then X-RateLimit-Reset, then fall back to jittered exponential backoff. Add 1–5 seconds of random jitter after the reset so ten workers don’t refill the new window simultaneously. Result: all 60 requests completed in every run, in 89 to 92 seconds in Python and 106 to 218 seconds in TypeScript. That’s the cost — nothing fails, but the batch takes 89 seconds instead of 12. Also set max_retries=0 on your SDK client if you wrap it in your own loop, or each attempt turns into three requests.
Fall back to another model. Overflow goes to a different model instead of waiting. All 60 finished in 13 seconds, at a cost of $0.000066, with no waiting at all. The trade-off is quality: you’re accepting whatever the fallback model produces, which may not match your evaluation.
Let another provider serve the same model. Pinned to one provider with failover off, up to 32 of 150 requests failed per run. With failover on, all 150 succeeded every run. This works for the upstream-pool 429 specifically, because the limit belongs to the provider, not to you. One limitation: in OpenRouter’s runs, upstream 429s carried no reset header, because providers don’t say when capacity frees up — so there’s nothing to wait on; you have to route around it.
If you’re already thinking in terms of routing, this connects to what OpenRouter has been building more broadly — its rebuilt Auto Router, covered earlier on this blog, applies the same idea of letting the platform pick the path instead of hardcoding one model.
What to check next
A few caveats before you apply this. Every result above comes from tests against the OpenRouter API on specific free models, so the exact numbers don’t establish behavior for paid tiers or other providers’ hard limits. The tutorial also continues past where the excerpt I’m working from cuts off — mid-way through the TypeScript retry snippet — so the later sections, including the FAQ, aren’t reflected here.
The practical next step is cheap: reproduce the default-settings test against your own key. Fire a modest concurrent batch at your normal model and log two things — how many requests fail after your SDK’s built-in retries, and what limit_source and reset headers came back on each 429. If your failure count looks anything like 23 out of 60, the retry logic you ship should read Retry-After and X-RateLimit-Reset itself, and you should decide in advance whether overflow waits, falls back, or routes to another provider. And if you’re on free models, remember the daily floor: accounts that haven’t bought 10 credits get 50 free-model requests per day, which no retry strategy can work around.