How to scrape Workday job postings legally (and why you shouldn't)
Workday tenant career pages are public. Scraping them is technically legal in most jurisdictions if you respect robots.txt and rate limits. Here's the right way — and why a managed API is almost always cheaper than DIY.
Eng team
Engineering
“Scrape Workday” sits in an interesting legal grey zone. The tenant career pages at *.myworkdayjobs.com are publicly indexed, intended for public consumption, and explicitly listed in the parent company’s robots.txt as crawlable. So what’s the catch?
This post covers the legal posture, the technical reality, and why most teams that start with a DIY Workday scraper end up on a managed source within 6–12 months.
The legal picture (broadly)
In the US, the hiQ Labs v. LinkedIn case (2019, reaffirmed 2022) established that scraping publicly accessible data is not a violation of the Computer Fraud and Abuse Act. Workday job postings are publicly accessible. So far so good.
The asterisks:
- Workday’s TOS for the customer (the Workday tenant) covers the customer’s data, not yours-as-scraper.
- Some tenants set custom robots.txt rules that disallow crawling. Respect those.
- Rate limiting and respect of
Retry-Afterheaders matters legally and practically. - Republishing scraped postings without attribution may run into copyright on the description text.
None of this is legal advice. Talk to a lawyer if you’re shipping a commercial product.
The technical reality
Public Workday tenants expose a JSON feed at a predictable URL. You can POST a search filter and paginate through results. Sounds simple. The operational pain:
- Akamai bot management. A single static IP gets blocked within minutes once it crosses the rate threshold.
- Per-tenant URL variation. Different region prefixes, different tenant slugs.
- 10,000-result hard cap per search. To enumerate everything you slice by location/category.
- Tenant list maintenance. ~500 public tenants today, growing weekly as companies onboard Workday.
The build-vs-buy math
Running this in production typically takes one engineer ~3 months to ship and ~1 day/week ongoing. At loaded cost, that’s roughly $4,000–$6,000/month forever. JobsPipe’s Growth tier includes the entire Workday source for $199/month.
Most teams who go the DIY route do it because they want to learn the problem. Six months in, they migrate to a managed API and keep their original scraper as a fallback. That’s a sensible progression.
Skip the build — Workday + 30 other sources, one API.
Get a free API key