From the blog · Page 9
How we build, what we learn, what we ship.
Posts about indexing public job postings at scale, salary parsing, and the systems that keep one universal JSON schema feeling small even as the source list keeps growing.
Which Websites Allow Web Scraping?
Does Amazon allow web scraping? No, like most large sites. How to read any site's terms, robots.txt and API. Reddit, Wikipedia and YouTube included.
Can ChatGPT Do Web Scraping?
Can ChatGPT do web scraping? It fetches single pages through tools and writes scrapers, but does not crawl. Give it an API or MCP server for live data.
Job scraping explained: how it works, what you get by board, and when it is legal
What job scraping involves in production (rendering, sessions, structure drift, dedup, liveness), the share of postings with a stated salary, a labelled seniority, an employment type and a thin description on every collected board, a plain summary of the legal position with a link to the full treatment, and the honest math on scraping versus using an API.
What "job closed or expired" means on Indeed (and what to do about it)
On Indeed, "job closed or expired" means the listing is no longer taking applications - either the employer closed it, or the posting ran out its visibility window. It is one combined status, so it does not tell you which happened, and it does not mean the role was filled. Here is how to tell the difference, how long Indeed postings really stay up, and whether to reapply when one is reposted.
How to build an MCP server: a step-by-step guide with a real API
A runnable tutorial that builds a jobs MCP server on POST /v1/jobs/search with the TypeScript SDK pinned to the version the hosted JobsPipe server runs: tool definition with zod schemas and structured output, a Streamable HTTP transport on Hono, a stdio variant, how to test it with curl, the MCP inspector and Claude Code, and the four decisions that decide whether a model can use it.
The best niche job boards in 2026: 18 that work, and why
What niche job boards are, 18 examples by category (remote, startup, tech, AI, web3, design, climate, public sector, academia, nonprofit), how they get their listings, and a data section on what employer-direct ATS postings carry that LinkedIn and Indeed listings do not. Plus how to pick a niche you can actually keep full.
Job title hierarchy: levels and seniority, explained (2026)
The standard corporate ladder from intern to C-level, what each level actually means, how it differs across engineering, sales, finance, and healthcare, how it maps to normalized job-posting seniority, and the seniority mix on every collected job board.
How to find old job postings: 6 ways to dig up an expired Indeed or LinkedIn listing
Whether you're prepping for an interview, comparing comp bands historically, or hunting for a posting that was taken down - here's how to find an old job posting anywhere, including expired listings on Indeed and closed job postings on LinkedIn, plus how long postings actually stay up by board.
How to scrape Workday job postings legally (and why you shouldn't)
Workday tenant career pages are public. Scraping them is technically legal in most jurisdictions if you respect robots.txt and rate limits. Here is the right way, what Workday postings look like once collected (salary, remote, seniority and employment type shares), and why a managed API is almost always cheaper than DIY.