
Google Search Scraper
Free and fully open-source Google Search scraper for Python. Get structured SERP data through a real Chrome browser and your NodeMaven residential proxy. Hybrid mode keeps each search fast, with a 100% success rate



How the Google search scraper works
Four steps run for every proxy IP. The run summary and per-step timings show each one
01
The scraper opens a sticky proxy session and a fresh browser profile. Chrome then visits a few ordinary pages and stays on each for 20 to 45 seconds. This collects the cookies Google trusts. A fresh warm-up uses about 60 MB, so the first query takes a few minutes. Later queries on the same proxy IP take about 1.4 seconds
02
Hybrid mode is on by default. The first query on a proxy IP opens Google’s results page in Chrome. The next results pages come over light HTTP requests with Chrome’s cookies, and the browser steps back in when Google checks again. Add --no-hybrid and every search runs in a full Chrome window
03
Every page is classified first. CAPTCHAs and refusals are reported as such. Poisoned results, where Google serves results for a different query, are detected and the proxy IP is replaced. Only real results reach your data
04
A proxy IP Google serves is reused for the next queries. A refused proxy IP is dropped, and the query is retried on a new one. If proxy IPs keep getting refused, the run stops instead of spending more traffic
How to start scraping Google search results
Three steps from install to your first JSON file. The exact commands are in the README on GitHub
Install
Connect NodeMaven
Run your first search
Google search scraper features
Everything runs on your machine, through NodeMaven proxies. Every page shows what it cost and whether Google served it
Transparent costs per Google search
Cost per search
Tested in hybrid mode, the default. Multiply the traffic by your NodeMaven price per GB to get the cost of 1,000 searches
Cookie copies
New proxy IPs start from the cookies of a proxy IP Google already serves. A copy costs a few MB instead of a full warm-up. It waits 40 to 60 seconds before its first search.
Long runs
Served proxy IPs stay in use for hundreds of queries. Regular browser restarts keep memory flat over long holds
Why scrape Google with a real browser
Google often expects a browser that runs JavaScript. A fresh browser on a fresh proxy IP can be refused quickly. That changes how a Google scraper has to work
A script sends raw requests and parses whatever HTML comes back. Google often expects JavaScript, so responses are unreliable. A refused page can be parsed as an empty result. You cannot tell "no results" from "refused"
The Google Search Scraper approach
Each proxy IP is warmed up in real Chrome on a sticky session. In hybrid mode, the default, Chrome passes Google’s checks and results pages then come over HTTP with its cookies. With --no-hybrid, every search runs in Chrome. Every page is classified before parsing. Refused pages come back labeled, and the proxy IP is replaced
Use cases for a self-hosted Google scraper
Live Google results for SEO teams, AI builders and developers who already use NodeMaven
SEO monitoring
Track positions for your own keywords and watch how results change. Run keyword and competitor research on live Google results
AI agents and LLM apps
Give Claude, Cursor and other agents live Google search through the MCP server. Each call returns structured results, not raw HTML
SERP API replacement
Already on NodeMaven? Run your own SERP API on localhost and point existing code at /search.json. You pay for proxy traffic, not per search
Try the scraper on a $3.50 trial with 750 MB of residential traffic
Frequently asked questions
It is NodeMaven’s free, open-source Google search scraper for Python. It collects Google SERP results through a real Chrome browser and NodeMaven residential proxies. In hybrid mode, the default, Chrome passes Google’s checks and results pages come over fast HTTP requests with its cookies. You can also run every search in the browser. Results come back as JSON
The code is fully free and open-source. The searches are not. Every search runs through NodeMaven residential proxies, and proxy traffic is paid. In hybrid mode, 1,000 searches use about 0.45 GB of residential traffic on long runs
A paid SERP API runs on its own servers and charges per search. This tool runs on your machine through NodeMaven residential proxies. You pay for traffic, and every run shows the cost per page. The HTTP API uses a familiar SERP API format
Install the scraper with the [nodemaven] extra, set NODEMAVEN_LOGIN and NODEMAVEN_PASSWORD, and add --nodemaven to your command. The session id is handled for you. Choose the proxy country with --country. Set the results language and country with --hl and --gl
A new proxy IP is warmed up first. The browser visits ordinary pages and stays on each for 20 to 45 seconds. Later searches on the same proxy IP take about 1.4 seconds. The scraper is not frozen
The page is labeled captcha in page_verdict. It is never returned as an empty result. The proxy IP is dropped and the query is retried on a new one. Google may also answer with normal-looking results for a different query. The scraper checks every page against the query, so those results never reach you. The check costs no extra requests or traffic
Yes. The mcp command starts an MCP server with one tool, google_search(query, pages, hl, gl). It returns organic results, related questions and searches, and the AI Overview and map places when Google shows them. Claude, Cursor and other MCP clients can call it
Organic results, ads, related questions, related searches, pagination and search metadata. AI Overview on English results pages, and places from the map on local queries. Knowledge panels, shopping and images are not parsed
Yes. A ready image is published to GitHub Container Registry with every release. Run doctor inside it first to check the setup.
It depends on your use and where you run it. You are responsible for using the tool within Google’s terms and local law








