API rate limiting
API rate limiting is the practice of capping how many requests a client can make to an API within a set time window, protecting the service from overload and enforcing fair use across all consumers.
Also called: rate limiting, API throttling, rate limit.
Key points
- API rate limiting caps how many requests a client can make in a time window, per key, user, or IP.
- APIs enforce it for stability, fair use across consumers, and to back tiered pricing.
- Common algorithms include fixed window, sliding window, token bucket, and leaky bucket; token bucket is the most common.
- Consumers handle limits by reading rate-limit headers, respecting 429 responses and Retry-After, and using exponential backoff.
What API rate limiting is and why APIs enforce it
An API rate limit is a ceiling on request volume - for example, 100 requests per minute, or 5,000 per month - applied per client, per key, or per IP address. When a client crosses the ceiling, further requests are rejected until the window resets. Almost every production API enforces some form of it.
APIs rate-limit for three reasons. The first is stability: without a cap, one runaway client - a buggy loop, a misconfigured job - can overwhelm the backend and degrade the service for everyone. The second is fairness: limits stop a single heavy consumer from starving others of capacity. The third is commercial: rate limits are how tiered pricing is enforced, with a free tier capped lower than paid plans.
How rate limiting works: algorithms, headers, and 429s
Several algorithms implement the cap. A fixed window counts requests in each clock interval. A sliding window smooths the boundary so a burst across two windows still counts. A token bucket hands out tokens at a steady rate and lets clients spend a saved-up burst. A leaky bucket drains requests at a constant rate regardless of how they arrive. Token bucket is the most common because it allows short bursts while holding the long-run average.
As a consumer you see rate limiting through two signals. The HTTP status 429 Too Many Requests means you have hit the cap. Response headers carry the detail: variants of X-RateLimit-Limit, X-RateLimit-Remaining, and X-RateLimit-Reset tell you the ceiling, how much is left, and when the window resets, while a Retry-After header on a 429 tells you exactly how long to wait.
Handling rate limits as an API consumer
Well-behaved clients treat rate limits as normal, not exceptional. Read the X-RateLimit-Remaining header and slow down before you hit zero rather than after. On a 429, respect the Retry-After value instead of retrying immediately, and use exponential backoff so repeated failures wait progressively longer. Cache responses that do not change often so you are not spending requests to re-fetch the same data.
This matters most when consuming job data at scale. Pulling postings across many sources, or crawling career pages, runs straight into rate limits - both the source's and your provider's. Batching with proper pagination, scheduling heavy jobs outside peak hours, and designing for backoff from the start keeps a large job-data pipeline inside its limits. JobsPipe applies its own limits per key, with a free tier of 5,000 requests per month.
FAQ
What does HTTP 429 mean?+
429 Too Many Requests is the standard HTTP status an API returns when a client has exceeded its rate limit. It signals that the request was rejected not because it was malformed, but because too many requests arrived too fast. A 429 response usually includes a Retry-After header telling the client how long to wait before trying again.
What is the difference between rate limiting and throttling?+
The terms overlap and are often used interchangeably. Rate limiting usually means rejecting requests once a hard cap is crossed, returning a 429. Throttling more often means deliberately slowing requests down - queuing or delaying them - to stay within a limit rather than rejecting them outright. Both control request volume; the difference is whether excess requests are dropped or delayed.
How should I handle rate limits when consuming a jobs API?+
Read the rate-limit headers and slow down before you hit the cap, not after. On a 429, wait for the Retry-After interval and apply exponential backoff. Cache data that does not change often, batch reads with pagination, and schedule heavy pulls outside peak hours. Designing for limits from the start is far easier than retrofitting it once a pipeline is already failing.
JobsPipe is the jobs-data API behind this glossary - 30+ sources, one schema, 5,000 requests/month free.
Get a free API keyRelated