Can ChatGPT Do Web Scraping?

Can ChatGPT do web scraping? Partly. It can fetch a page through its browsing tool, read it and summarise it, and it can write a scraper for you to run. It cannot crawl a site at scale from inside a chat, and its code sandbox usually has no internet access. The reliable path to live data is an API or an MCP server the assistant can call.

How to do web scraping with ChatGPT, Claude or another assistant

1. Separate the three tools. Ask “can ChatGPT do web scraping” and the honest answer depends on which tool is on. Browsing fetches one page at a time and sites can block it. Code execution runs Python in a sandbox that, in most products, cannot reach the internet, so it parses data you paste or upload. Connectors and MCP servers call an API on your behalf, with the server doing the fetching.
2. Use it as the author, not the runner. For a one-off pull, ask for the script, run it on your own machine and paste any error back. The assistant is good at selectors, pagination and normalisation code; the fetching happens where you have a real network and a real IP. A worked script is in how to web scrape with Python.
3. For repeated pulls, connect a tool. ChatGPT and Claude both accept remote MCP servers as connectors, and coding agents such as Claude Code and Cursor read them from a config file. The assistant then calls named tools, and the server behind them handles authentication, rate limits and the actual data access. What MCP is and how the calls travel is in what is an MCP server.
4. Worked example: job postings. Add the JobsPipe server and ask a plain question such as remote data engineer roles posted this week. The assistant calls search_jobs, which maps the arguments onto POST /v1/jobs/search and returns structured postings it can filter and cite.

   {
     "mcpServers": {
       "jobspipe": { "url": "https://mcp.jobspipe.dev/mcp" }
     }
   }

5. Give coding agents the CLI. An agent with a shell can skip the protocol and call one command, which prints JSON it can read directly.

   npm i -g jobspipe-cli
   export JOBSPIPE_API_KEY=jp_live_...
   jobspipe jobs --title "data engineer" --country US --remote --limit 5

6. Know the limits. A connected assistant still cannot bypass a login wall or a bot challenge, and asking it to is asking it to break a site’s terms. What it can do well is query a source that was built to be queried, and reason over the result.

The pattern generalises. An assistant that scrapes is fragile and slow; an assistant with a tool that returns clean records is fast and checkable, because every answer comes with ids and dates you can inspect. What that looks like for a job seeker, and what ChatGPT can and cannot do without a tool attached, is in ChatGPT for job search.

Where JobsPipe fits

JobsPipe is a jobs data API that collects live postings from LinkedIn, Indeed, Y Combinator, Naukri, Workday, Greenhouse, Workable, SmartRecruiters, Ashby, Lever and Paylocity, returns them as one schema with closure tracking and a ghost score, and includes a free tier of 1,000 jobs a month at jobspipe.dev. Its hosted MCP server at https://mcp.jobspipe.dev exposes search_jobs, detect_company_tech_stack, list_pricing_plans and search_documentation, and agents read /agents.md to set themselves up. Client-by-client instructions are on the MCP server page.