JobSpy review: the Python job scraper tested, and where it breaks

JobSpy is the most popular open-source job scraper in the Python ecosystem: one pip install python-jobspy, one function call, and you get postings from Indeed, LinkedIn, Glassdoor, ZipRecruiter, Google, and more back as a pandas DataFrame. We build a commercial jobs API, so we are exactly the vendor you’d expect to trash it. We’re not going to - JobSpy is genuinely good at what it is for. This review covers what it does well, where it breaks, and how to make the scraper-vs-API call honestly.

What JobSpy does well

The core loop is hard to beat for exploratory work:

from jobspy import scrape_jobs

jobs = scrape_jobs(
    site_name=["indeed", "linkedin", "zip_recruiter", "glassdoor"],
    search_term="data engineer",
    location="Austin, TX",
    results_wanted=100,
)
print(jobs.head())

• One interface over many boards. The same call shape works across every supported site, and results land in a DataFrame ready for analysis.
• Free and open source. For a student project, a one-off market analysis, or a weekend prototype, the cost argument ends here.
• Actively maintained. The project keeps pace with board layout changes better than most scraping libraries, and the proxy support is built in rather than bolted on.

Where it breaks

• Blocking, immediately at scale. LinkedIn rate-limits unauthenticated scraping within a few hundred results; Indeed and Glassdoor run serious bot management. Past toy volumes you are buying rotating residential proxies and babysitting 429s - the costs the “free” framing hides.
• Query-shaped, not feed-shaped. You get results per search term per location, capped per run. Comprehensive or continuous coverage means enumerating a large query matrix on a schedule you maintain.
• No dedup, no change tracking. The same role appears on three boards as three rows, and nothing tells you when a posting closed. Both become your code.
• Maintenance transfers to you. When a board re-skins, your pipeline is down until the library updates and you upgrade. Fine for research; a pager problem in production.

Scraper or API: the actual decision

The decision is not about tool quality - it is about whether the collection layer is your product. If you are running a one-off analysis, learning scraping, or prototyping, JobSpy is the right tool and you should use it. If jobs data feeds something with users, revenue, or an SLA, the DIY pipeline math turns against you fast - we’ve run the numbers for Indeed and Glassdoor, and proxies plus maintenance plus on-call time reliably clears $3,000 a month before opportunity cost.

There is also a structural difference in what you can collect. JobSpy reads consumer job boards. JobsPipe reads the upstream ATS layer - Workday, Greenhouse, Lever, Ashby, and 30+ more - where postings appear first, with stable IDs, structured salary, and webhook delivery on changes:

curl https://api.jobspipe.dev/v1/jobs/search \
  -H "Authorization: Bearer jp_live_your_key_here" \
  -H "Content-Type: application/json" \
  -d '{ "job_title_or": ["data engineer"], "posted_at_gte": "2026-07-12" }'

A reasonable path many teams take: prototype the idea on JobSpy, and switch the collection layer to an API when the prototype earns production status.