Buy proxies. Win an iPhone 18 Pro and more
20 prizes
Start your trial

Google Search Scraper

Free and fully open-source Google Search scraper for Python. Get structured SERP data through a real Chrome browser and your NodeMaven residential proxy. Hybrid mode keeps each search fast, with a 100% success rate

Browser trust, HTTP speed
SERP API-style JSON
Completely free
Open source, MIT license

How the Google search scraper works

Four steps run for every proxy IP. The run summary and per-step timings show each one

New session and warm-up

01

The scraper opens a sticky proxy session and a fresh browser profile. Chrome then visits a few ordinary pages and stays on each for 20 to 45 seconds. This collects the cookies Google trusts. A fresh warm-up uses about 60 MB, so the first query takes a few minutes. Later queries on the same proxy IP take about 1.4 seconds

Search in hybrid or browser mode

02

Hybrid mode is on by default. The first query on a proxy IP opens Google’s results page in Chrome. The next results pages come over light HTTP requests with Chrome’s cookies, and the browser steps back in when Google checks again. Add --no-hybrid and every search runs in a full Chrome window

Check before parsing

03

Every page is classified first. CAPTCHAs and refusals are reported as such. Poisoned results, where Google serves results for a different query, are detected and the proxy IP is replaced. Only real results reach your data

Keep or replace

04

A proxy IP Google serves is reused for the next queries. A refused proxy IP is dropped, and the query is retried on a new one. If proxy IPs keep getting refused, the run stops instead of spending more traffic

How to start scraping Google search results

Three steps from install to your first JSON file. The exact commands are in the README on GitHub

Install

Install the Python package with NodeMaven support, then add the Chromium browser it runs on. It takes two short commands

Connect NodeMaven

Add your NodeMaven login and password to your environment. The scraper reads them and manages proxy sessions for you

Run your first search

Put your keywords in a text file and start a search. You get one JSON record for every Google results page

Google search scraper features

Everything runs on your machine, through NodeMaven proxies. Every page shows what it cost and whether Google served it

Poisoned results caught
Google sometimes serves bots results for a different query. The scraper catches them and replaces the proxy IP, at no extra traffic
Cost counter
Each run ends with pages served, traffic used and the hosts that used it. Add --price-per-gb and the scraper prints the cost per 1,000 pages
NodeMaven built in
NodeMaven residential proxies connect with one flag: --nodemaven. Sessions and credentials are handled for you. Set language and country with --hl and --gl
AI overview and places
Each record includes Google’s AI Overview when an English results page shows one. Local queries also return places from the map, with rating, address and opening hours
Hybrid or full browser
Chrome passes Google’s checks, then results come over HTTP with its cookies. If that fails, the browser takes over. --no-hybrid runs every search in Chrome
Free and open source
MIT-licensed code you can read, fork and run yourself. No subscription or per-search fees, just proxy traffic

Transparent costs per Google search

Cost per search

Tested in hybrid mode, the default. Multiply the traffic by your NodeMaven price per GB to get the cost of 1,000 searches

Cost per search
0.4 MB
per search without warm-up
1.4 s
per search on a warm proxy IP
~0.45 GB
per 1,000 searches on long runs
~60 MB
per fresh warm-up

Cookie copies

New proxy IPs start from the cookies of a proxy IP Google already serves. A copy costs a few MB instead of a full warm-up. It waits 40 to 60 seconds before its first search.

Cookie copies
10
copied proxy IPs by default
180 min
cookie lifetime
40-60 s
start delay
Few MB
per copy

Long runs

Served proxy IPs stay in use for hundreds of queries. Regular browser restarts keep memory flat over long holds

Long runs
300
queries per proxy IP
50
pages between restarts
1
retry on a browser error
N
parallel workers

Why scrape Google with a real browser

Google often expects a browser that runs JavaScript. A fresh browser on a fresh proxy IP can be refused quickly. That changes how a Google scraper has to work

The plain HTTP approach

A script sends raw requests and parses whatever HTML comes back. Google often expects JavaScript, so responses are unreliable. A refused page can be parsed as an empty result. You cannot tell "no results" from "refused"

The Google Search Scraper approach

Each proxy IP is warmed up in real Chrome on a sticky session. In hybrid mode, the default, Chrome passes Google’s checks and results pages then come over HTTP with its cookies. With --no-hybrid, every search runs in Chrome. Every page is classified before parsing. Refused pages come back labeled, and the proxy IP is replaced

Use cases for a self-hosted Google scraper

Live Google results for SEO teams, AI builders and developers who already use NodeMaven

SEO monitoring

SEO monitoring

Track positions for your own keywords and watch how results change. Run keyword and competitor research on live Google results

Recommended for:
AI agents and LLM apps

AI agents and LLM apps

Give Claude, Cursor and other agents live Google search through the MCP server. Each call returns structured results, not raw HTML

Recommended for:
SERP API replacement

SERP API replacement

Already on NodeMaven? Run your own SERP API on localhost and point existing code at /search.json. You pay for proxy traffic, not per search

Recommended for:
Not sure which mode fits your project?

Try the scraper on a $3.50 trial with 750 MB of residential traffic

Try the scraper on a $3.50 trial with 750 MB of residential traffic

Frequently asked questions

It is NodeMaven’s free, open-source Google search scraper for Python. It collects Google SERP results through a real Chrome browser and NodeMaven residential proxies. In hybrid mode, the default, Chrome passes Google’s checks and results pages come over fast HTTP requests with its cookies. You can also run every search in the browser. Results come back as JSON

The code is fully free and open-source. The searches are not. Every search runs through NodeMaven residential proxies, and proxy traffic is paid. In hybrid mode, 1,000 searches use about 0.45 GB of residential traffic on long runs

A paid SERP API runs on its own servers and charges per search. This tool runs on your machine through NodeMaven residential proxies. You pay for traffic, and every run shows the cost per page. The HTTP API uses a familiar SERP API format

Install the scraper with the [nodemaven] extra, set NODEMAVEN_LOGIN and NODEMAVEN_PASSWORD, and add --nodemaven to your command. The session id is handled for you. Choose the proxy country with --country. Set the results language and country with --hl and --gl

A new proxy IP is warmed up first. The browser visits ordinary pages and stays on each for 20 to 45 seconds. Later searches on the same proxy IP take about 1.4 seconds. The scraper is not frozen

The page is labeled captcha in page_verdict. It is never returned as an empty result. The proxy IP is dropped and the query is retried on a new one. Google may also answer with normal-looking results for a different query. The scraper checks every page against the query, so those results never reach you. The check costs no extra requests or traffic

Yes. The mcp command starts an MCP server with one tool, google_search(query, pages, hl, gl). It returns organic results, related questions and searches, and the AI Overview and map places when Google shows them. Claude, Cursor and other MCP clients can call it

Organic results, ads, related questions, related searches, pagination and search metadata. AI Overview on English results pages, and places from the map on local queries. Knowledge panels, shopping and images are not parsed

Yes. A ready image is published to GitHub Container Registry with every release. Run doctor inside it first to check the setup.

It depends on your use and where you run it. You are responsible for using the tool within Google’s terms and local law

Learn more about Google SERP scraping

This site uses cookies to enhance your experience. By continuing, you agree to our use of cookies.