Skip to content

Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.

Google Maps Scraper Kit — free local Google Maps scraper, driven by Claude

Mahanaicoachfa805c6973ed
diff --git a/.claude/commands/scrape-batch.md b/.claude/commands/scrape-batch.md
new file mode 100644
index 0000000..6d4b0a1
--- /dev/null
+++ b/.claude/commands/scrape-batch.md
@@ -0,0 +1,19 @@
+---
+description: Scrape many Google Maps queries at once (batch) from a list of keywords
+argument-hint: <keywords-file> [--city "City, ST"]
+---
+The user wants a **batch** Google Maps scrape. Input: **$ARGUMENTS** (a keywords file, optionally a city).
+
+Use the `google-maps-scraper` skill. The API scrapes **many keywords in ONE job** (the `keywords` array),
+so batch = one job with all terms.
+
+1. Ensure the scraper is up (`curl -s http://localhost:8080/api/v1/jobs`; `docker compose up -d` if not).
+2. Read the keyword list (one per line; ignore blank / `#` lines). If a `--city` was given, geocode it once
+ for `lat`/`lon`; otherwise expect the location baked into each keyword (geocode the first one).
+3. **Warn, don't block:** a batch with many keywords is high-volume — print ONE short warning that this can
+ get the IP temporarily rate-limited and suggest proxies for large/repeated runs, then **proceed**.
+4. POST a single job with the full `keywords` array + required fields (`max_time` seconds, string `lat`/`lon`).
+ Use a generous `max_time` (e.g. `900`) since many keywords take longer. Poll in the BACKGROUND, download.
+5. Present results grouped/clean and save the full file to disk.
+
+Tip: `scripts/scrape.py --keywords-file <file> --city "<City, ST>"` already does all of this.
diff --git a/.claude/commands/scrape-jobs.md b/.claude/commands/scrape-jobs.md
new file mode 100644
index 0000000..d4142cc
--- /dev/null
+++ b/.claude/commands/scrape-jobs.md
@@ -0,0 +1,11 @@
+---
+description: List or clean up Google Maps scrape jobs
+argument-hint: "[list | delete <job-id>]"
+---
+Manage scrape jobs via the API. Request: **$ARGUMENTS** (default to "list" if empty).
+
+- **list** → `GET http://localhost:8080/api/v1/jobs`. Show a tidy table of `ID`, `Name`, `Status`, `Date` (newest first).
+- **delete <job-id>** → `DELETE http://localhost:8080/api/v1/jobs/<job-id>` to free disk space, then confirm it returned HTTP 200.
+- **download <job-id>** → `GET .../jobs/<job-id>/download` → save CSV and summarize row count.
+
+See the `google-maps-scraper` skill for the full API reference.
diff --git a/.claude/commands/scrape-setup.md b/.claude/commands/scrape-setup.md
new file mode 100644
index 0000000..66f78ba
--- /dev/null
+++ b/.claude/commands/scrape-setup.md
@@ -0,0 +1,18 @@
+---
+description: Set up and start the local Google Maps scraper (Docker)
+---
+Set up the Google Maps scraper on this computer:
+
+1. Confirm Docker is installed and running: `docker ps`. If it errors, tell the user to install/start **Docker Desktop** (link in `SETUP.md` §0) and stop here.
+2. From the kit root, start the scraper: `docker compose up -d`.
+3. Wait ~10s, then verify it's live: `curl http://localhost:8080/api/v1/jobs` should return a JSON array (often `[]`).
+4. Report success and tell the user they can now run `/scrape <business> in <city>` — or just ask in natural language (e.g. "scrape gyms in Miami").
+5. **Show this over-use warning** (always print it once setup succeeds):
+ > ⚠️ **Before you scrape:** this hits Google Maps for real. Light use is fine, but running big jobs
+ > back-to-back, very high `depth`, many keywords, or scheduled runs **without proxies** can get your
+ > IP **temporarily rate-limited/blocked by Google** (usually clears in minutes–hours; jobs start
+ > returning empty/failed meanwhile). It won't ban your Google account. Stay safe: one job at a time,
+ > start at `depth 5`, and add **proxies** for large or repeated runs. Scraping Maps is also against
+ > Google's ToS — use responsibly and follow data laws (GDPR/CCPA) for any contact info.
+
+If a step fails, consult `SETUP.md` §6 (Troubleshooting).
diff --git a/.claude/commands/scrape.md b/.claude/commands/scrape.md
new file mode 100644
index 0000000..a945f50
--- /dev/null
+++ b/.claude/commands/scrape.md
@@ -0,0 +1,19 @@
+---
+description: Scrape Google Maps business listings for a query + city (drives the google-maps-scraper skill)
+argument-hint: <business type> in <city, ST> [depth]
+---
+The user wants to scrape Google Maps business listings. Query: **$ARGUMENTS**
+
+Use the `google-maps-scraper` skill. Execute:
+
+1. **Health-check** the API: `curl -s http://localhost:8080/api/v1/jobs`. If it's down, run `docker compose up -d` from the kit root, wait ~10s, and retry.
+2. **Parse** the business type and city from the query. Look up the city's **latitude/longitude** (as strings).
+3. **Create the job** (`POST /api/v1/jobs`) with the required fields — `keywords` (location baked into each term), `lat`/`lon` (strings), `max_time` (seconds, default `300`), and `depth` (use the number in the query, else `5`).
+4. **Poll to completion in the BACKGROUND** (the Bash tool blocks foreground `sleep` — use `run_in_background: true` and read the output file when notified). Then download the CSV.
+5. **Present ONLY money-useful lead fields** in a clean markdown table: `name (title), phone, emails, website, category, address, review_rating, review_count`. **Drop everything else** — especially `latitude`/`longitude`, IDs, hours, images, review blobs. Save to a file on disk; do NOT dump the full 34-column CSV into chat. (Include dropped fields only if the user explicitly asks.)
+
+**Warn, don't block:** if the request is high-volume (high `depth`, many keywords, or repeated runs),
+print ONE short warning that over-use can get their IP temporarily rate-limited by Google and suggest
+adding proxies — then **proceed with the scrape anyway**. Do not gate or demand confirmation just for a
+big job. Only refuse outright for clearly abusive use (surveilling individuals, spam/harassment). Also
+remind the user about PII / Google ToS if they'll store or contact the data.
diff --git a/.claude/settings.json b/.claude/settings.json
new file mode 100644
index 0000000..15e01bd
--- /dev/null
+++ b/.claude/settings.json
@@ -0,0 +1,21 @@
+{
+ "$comment": "Pre-approves the safe, local commands this kit uses so Claude can scrape on autopilot without a permission prompt each time. All entries are local-only (localhost / this folder's scripts / docker). Nothing here can reach the public internet except the scraper's own outbound calls.",
+ "permissions": {
+ "allow": [
+ "Bash(docker compose up:*)",
+ "Bash(docker compose down:*)",
+ "Bash(docker compose stop:*)",
+ "Bash(docker compose start:*)",
+ "Bash(docker compose ps:*)",
+ "Bash(docker ps:*)",
+ "Bash(curl http://localhost:8080:*)",
+ "Bash(curl -s http://localhost:8080:*)",
+ "Bash(curl http://127.0.0.1:8080:*)",
+ "Bash(./scripts/scrape.sh:*)",
+ "Bash(bash scripts/scrape.sh:*)",
+ "Bash(python3 scripts/scrape.py:*)",
+ "Bash(python scripts/scrape.py:*)"
+ ],
+ "deny": []
+ }
+}
diff --git a/.claude/skills/google-maps-scraper/SKILL.md b/.claude/skills/google-maps-scraper/SKILL.md
new file mode 100644
index 0000000..8dd9500
--- /dev/null
+++ b/.claude/skills/google-maps-scraper/SKILL.md
@@ -0,0 +1,144 @@
+---
+name: google-maps-scraper
+description: Scrape Google Maps business listings (name, address, phone, website, rating, reviews, lat/lng, hours, emails) via the local gosom google-maps-scraper REST API. Use when the user wants local-business / lead-gen data, "a list of [businesses] in [place]", or to enrich places with contact info. NOT for Instagram/TikTok/YouTube or any social-media scraping.
+---
+
+# Google Maps Scraper
+
+Drive the local Google Maps scraper API to turn a business-type + location into clean, structured rows.
+
+## Mental model
+The scraper runs as a local Docker container exposing a REST API at `http://localhost:8080` (no auth —
+localhost only). A "scrape" is an **async job**: you create it, poll until it's done, then download a CSV.
+One job can run many keywords. Each result has up to **34 fields**.
+
+## Step 0 — Make sure it's running
+```bash
+curl -s http://localhost:8080/api/v1/jobs >/dev/null 2>&1 && echo UP || echo DOWN
+```
+If `DOWN`: `docker compose up -d` (from the kit root), wait ~10s, retry. If Docker isn't installed, point
+the user to `SETUP.md`.
+
+## Step 1 — Create a job (`POST /api/v1/jobs`)
+**Required fields — the API returns `422` without them:**
+- `keywords` — array of search strings. **Bake the location into each term**: `"plumbers in Denver CO"`.
+- `lat`, `lon` — **strings**, the city's coordinates: `"39.7392"`, `"-104.9903"`.
+- `max_time` — integer **seconds** (max wall-clock for the job), e.g. `300`. (Sent as seconds; the API stores it as nanoseconds internally — just send seconds.)
+
+**Recommended fields:**
+- `depth` (default 10) — how far to scroll → roughly how many listings per keyword. Start at `5`.
+- `lang` `"en"`, `zoom` `15` (city level), `radius` `10000` (meters), `fast_mode` `false`, `email` `false`.
+
+```bash
+curl -s -X POST http://localhost:8080/api/v1/jobs \
+ -H "Content-Type: application/json" \
+ -d '{"name":"job","keywords":["coffee shops in Austin TX"],"lang":"en","zoom":15,
+ "lat":"30.2672","lon":"-97.7431","fast_mode":false,"radius":10000,
+ "depth":5,"email":false,"max_time":300}'
+# → {"id":"<uuid>"} (HTTP 201; note: lowercase "id")
+```
+
+## Step 2 — Poll until done (`GET /api/v1/jobs/{id}`)
+The response field is `"Status"` (capital S): `working` → `ok` (success) or `failed`.
+
+> ⚠️ **Claude harness rule:** the Bash tool **blocks foreground `sleep`**. Run the poll loop as a
+> **background** Bash command (`run_in_background: true`) and read its output file when notified.
+> Do NOT poll with a foreground `sleep`.
+
+Background poll snippet (parses the value safely — match up to the closing quote, don't anchor on `$`):
+```bash
+ID="<uuid>"
+for i in $(seq 1 40); do
+ S=$(curl -s "http://localhost:8080/api/v1/jobs/$ID" | grep -oE '"Status":"[^"]*"' | head -1 | cut -d'"' -f4)
+ echo "status=$S"
+ [ "$S" = ok ] && { echo DONE; break; }
+ [ "$S" = failed ] && { echo FAILED; break; }
+ sleep 15
+done
+```
+
+## Step 3 — Download + parse (`GET /api/v1/jobs/{id}/download`)
+```bash
+curl -s "http://localhost:8080/api/v1/jobs/$ID/download" -o results.csv
+```
+CSV columns (34): `input_id, link, title, category, address, open_hours, popular_times, website, phone,
+plus_code, review_count, review_rating, reviews_per_rating, latitude, longitude, cid, status,
+descriptions, reviews_link, thumbnail, timezone, price_range, data_id, place_id, images, reservations,
+order_online, menu, owner, complete_address, about, user_reviews, user_reviews_extended, emails`.
+
+### ⭐ Output ONLY money-useful lead fields (default)
+The raw CSV has 34 columns and most are noise. **By default, return ONLY these lead fields** — the data you
+actually use to contact and qualify a lead — and **drop everything else**:
+
+> `title` (name), `phone`, `emails`, `website`, `category`, `address`, `review_rating`, `review_count`
+
+**DROP by default** (do not show these unless the user explicitly asks): `latitude`/`longitude` (no use for
+outreach), `link`, `plus_code`, `cid`, `data_id`, `place_id`, `open_hours`, `popular_times`,
+`reviews_per_rating`, `reviews_link`, `thumbnail`, `images`, `timezone`, `price_range`, `status`,
+`input_id`, `complete_address`, `reservations`, `order_online`, `menu`, `owner`, `about`, `descriptions`,
+`user_reviews`, `user_reviews_extended`.
+
+`scripts/scrape.py` already returns exactly this lead set (use `--full` to keep all columns, or
+`--fields "a,b,c"` to customize). If you call the API directly, **strip to the lead fields yourself** before
+presenting — never dump the full 34-column row at the user.
+
+**Shortcuts (prefer these for common cases):**
+- One keyword: `scripts/scrape.sh "<keyword>" <lat> <lon> [depth]` (bash) — create→poll→download in one go.
+- Auto-geocode (no coords): `python3 scripts/scrape.py "<keyword>" --city "<City, ST>" [--depth N]` — resolves
+ lat/lon via OpenStreetMap Nominatim. You can also geocode the city yourself and pass coords.
+- **Batch (many keywords, ONE job):** `python3 scripts/scrape.py --keywords-file <file> --city "<City, ST>"`.
+ The API takes a `keywords` array, so put all terms in a single job rather than firing many jobs.
+Do the manual curl flow only for custom job bodies (e.g. setting `proxies`).
+
+## Other endpoints
+- `GET /api/v1/jobs` — list jobs. `DELETE /api/v1/jobs/{id}` — delete a job + free disk.
+- Browser UI + OpenAPI docs: `http://localhost:8080` and `http://localhost:8080/api/docs`.
+
+## Best practices (from the upstream docs)
+- **Depth:** higher `depth` = more results but slower and more block-prone. Start low (5), raise as needed.
+- **One job at a time** locally. Many concurrent jobs without proxies → throttling/blocks by Google.
+- **Email extraction (`email:true`)** visits each business's website → *significantly* slower. Only when needed.
+- **`fast_mode:true`** returns reduced data, up to ~21 results/query, faster — good for quick lookups.
+- **Proxies:** for large/repeated jobs, set `"proxies"` (array of `socks5://`/`http://`/`https://` URLs,
+ auth supported). The scraper has built-in rotation. This is the main defense against rate-limiting.
+- **`zoom`/`radius`** control the search area around `lat`/`lon`. Widen `radius` if results are too few.
+- **Extended reviews** are available but heavy — don't enable unless the user asks for review text.
+
+## Rate limits, bans & proxies (read before large jobs)
+This hits Google Maps for real. The upstream project's only formal note is a disclaimer:
+> "Please use this scraper responsibly and in accordance with applicable laws and regulations. Unauthorized scraping may violate terms of service."
+
+There is **no published hard threshold** for bans, so be conservative:
+- Google may **temporarily rate-limit / block your IP** if you scrape too fast or too much. It clears in
+ minutes–hours and does **not** ban your Google account — but jobs start failing meanwhile.
+- **Block signals to watch:** jobs returning `failed`, empty or unusually short results, or a sudden drop
+ in row counts vs. a prior identical run. If you see these, **back off** (pause, lower depth) or add proxies.
+- **Concurrency ↔ blocking** (upstream): *"Higher concurrency … can increase blocking or failures,
+ especially without proxies. Start with the default for a first run."* Reference throughput ≈ **120
+ places/min** at `-c 8 -depth 1`. Locally, run **one job at a time** and start at `depth 5`.
+
+**When to add proxies** (upstream: *"For larger scraping jobs, proxies help avoid rate limiting"*):
+large jobs, many keywords, repeated/scheduled runs, or after you see block signals.
+- Set the `"proxies"` array in the job body. Types: `socks5`, `socks5h`, `http`, `https`.
+- Format: `protocol://user:pass@host:port` (auth optional), e.g.
+ `"proxies": ["socks5://user:pass@host:port", "http://host2:port2"]`. The scraper rotates them automatically.
+
+## Safety & guardrails
+- **WARN, don't block.** When a request is large/high-volume (high `depth`, many keywords, repeated runs,
+ or `email:true`), **proceed with it** but first print ONE short warning about temporary IP-block risk and
+ suggest proxies. Do **not** gate, force-stop, or demand confirmation just because a normal scrape is big.
+ Refuse outright **only** for clearly abusive/illegal use (surveilling individuals, spam/harassment).
+- **Never expose** the API beyond `localhost` without an auth proxy; never print any API key if one exists.
+- **PII:** scraped emails/phones are personal data. If the user will store or contact them, remind them to
+ comply with GDPR/CCPA/CAN-SPAM (lawful basis, opt-outs, suppression). Don't help with spam/harassment.
+- **Google ToS:** scraping Maps is against Google's Terms — keep volume modest, treat output as leads to
+ verify, don't resell raw Google data. Refuse uses aimed at surveilling individuals.
+- **Output hygiene:** don't dump huge CSVs into chat — save the file to disk, summarize counts, show a few
+ sample rows. Dedupe by `place_id`/`cid` before storing.
+- **Disk:** results pile up in the Docker volume; offer to `DELETE` old jobs periodically.
+
+## Troubleshooting
+- `422 missing max time` → add `max_time` (seconds). `422 missing geo coordinates` → add string `lat`/`lon`.
+- Stuck `working` → lower `depth` / raise `max_time` / IP throttled (add proxies or wait).
+- Empty CSV → keyword too narrow or geo wrong → widen `radius`, fix coordinates.
+- Connection refused → container down → `docker compose up -d`.
diff --git a/.env.example b/.env.example
new file mode 100644
index 0000000..9b4b0bd
--- /dev/null
+++ b/.env.example
@@ -0,0 +1,10 @@
+# Copy this to ".env" and adjust if needed: cp .env.example .env
+#
+# This local setup binds the scraper to 127.0.0.1 only, so NO API key is required.
+# (The scraper image has no built-in authentication — localhost binding is the boundary.)
+
+SCRAPER_BASE_URL=http://localhost:8080
+
+# Only needed if you later expose the scraper beyond localhost behind an auth proxy
+# (e.g. Caddy/nginx that checks an X-API-Key header). Leave blank for local use.
+SCRAPER_API_KEY=
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..d76c569
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,18 @@
+# Secrets
+.env
+
+# Scrape outputs (may contain personal data — never commit)
+*.csv
+results-*.json
+out/
+output/
+gmaps-output/
+data/
+
+# Python build cruft
+__pycache__/
+*.pyc
+
+# OS / editor noise
+.DS_Store
+Thumbs.db
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 0000000..d501cf7
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,42 @@
+# CLAUDE.md — Google Maps Scraper Kit
+
+You are working inside the **Google Maps Scraper Kit**. This folder runs a local Google Maps scraper
+and you drive it for the user.
+
+## What this project is
+A Docker-based local deployment of the open-source `gosom/google-maps-scraper`, exposed as a REST API
+at **`http://localhost:8080`**, plus a skill that teaches you to use it. It scrapes **Google Maps
+business listings only** (not social media).
+
+## When the user asks to scrape businesses / get a list of places / lead-gen data
+**Load and follow the `google-maps-scraper` skill** (`.claude/skills/google-maps-scraper/SKILL.md`).
+It has the exact API flow, the required fields, and the best-practice + safety rules. Don't improvise
+the API calls from memory — use the skill.
+
+## Slash commands
+This kit ships commands in `.claude/commands/` that map to the skill: `/scrape` (single query),
+`/scrape-batch` (many queries, one job), `/scrape-setup` (start the scraper), `/scrape-jobs` (list/delete).
+Local commands are pre-approved in `.claude/settings.json` so you can run them without prompting the user.
+
+## Rate limits — warn, don't block
+If a request is large/high-volume (high `depth`, many keywords, repeated runs, or `email:true`), **proceed**
+but first print ONE short warning that over-use can get the user's IP temporarily rate-limited by Google,
+and suggest proxies. Do NOT gate or demand confirmation for a normal big scrape. Refuse only clearly
+abusive use (surveilling individuals, spam). Full guidance is in the skill.
+
+## Hard facts you must not get wrong
+- API base: `http://localhost:8080` — **no auth** (it's localhost-only). Never expose it publicly.
+- Create a job: `POST /api/v1/jobs`. **Required** body fields or it 422s:
+ `max_time` (in **seconds**, e.g. `300`), `lat` & `lon` (as **strings**), plus `keywords` (array).
+- The create response returns `{"id":"<uuid>"}` (lowercase `id`). Status checks use `GET /api/v1/jobs/{id}` whose field is `"Status"` and becomes `"ok"` when done.
+- **Poll in the background.** The Bash tool blocks foreground `sleep`; run the poll loop as a
+ background command (`run_in_background: true`) and read its output file when notified.
+- Before a big job (high depth / many keywords), **confirm with the user** — cost is time + IP-block risk.
+- Scraped emails/phones are **personal data**; don't dump huge CSVs into chat — summarize + save to disk.
+
+## Setup
+If the container isn't running: `docker compose up -d`, then verify `curl http://localhost:8080/api/v1/jobs`.
+Full setup is in `SETUP.md`.
+
+## Attribution
+This wraps `gosom/google-maps-scraper` (MIT, © Georgios Komninos). Keep `CREDITS.md` + `LICENSE` intact.
diff --git a/CREDITS.md b/CREDITS.md
new file mode 100644
index 0000000..c4fe689
--- /dev/null
+++ b/CREDITS.md
@@ -0,0 +1,45 @@
+# Credits & Attribution
+
+This kit is a **wrapper** around an existing open-source project. The actual scraping engine is **not**
+written by this kit's author — full credit goes to the original author.
+
+## Upstream project
+
+- **Project:** google-maps-scraper
+- **Author:** **Georgios Komninos** ([@gosom](https://github.com/gosom))
+- **Repository:** https://github.com/gosom/google-maps-scraper
+- **License:** MIT License — Copyright (c) 2023 Georgios Komninos
+- **Docker image used:** `gosom/google-maps-scraper`
+
+This kit pulls and runs the official published Docker image. It does **not** modify or redistribute the
+upstream source code — it only adds a Docker Compose setup, helper scripts, documentation, and a Claude
+skill around it. All scraping capability and field extraction come from the upstream project.
+
+## Upstream license (reproduced as required by MIT)
+
+```
+MIT License
+
+Copyright (c) 2023 Georgios Komninos
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
+
+If you found this useful, please **⭐ star the upstream repo** to support the original author:
+https://github.com/gosom/google-maps-scraper
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..9ec7d22
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,26 @@
+MIT License
+
+Copyright (c) 2026 Mahan (github.com/mahanjafari-dev)
+
+This license covers the "Google Maps Scraper Kit" wrapper (Docker Compose setup,
+scripts, documentation, and Claude skill) ONLY. The underlying scraping engine,
+"google-maps-scraper", is a separate work by Georgios Komninos, licensed under the
+MIT License (Copyright (c) 2023 Georgios Komninos) — see CREDITS.md.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..e25e929
--- /dev/null
+++ b/README.md
@@ -0,0 +1,151 @@
+# Google Maps Scraper Kit 🗺️
+
+<p align="center">
+ <img src="assets/banner.svg" alt="Google Maps Scraper Kit — run a free Google Maps scraper locally, let Claude drive it on autopilot" width="100%">
+</p>
+
+A **plug-and-play kit** that runs a free, open-source Google Maps scraper **on your own computer**
+and lets **Claude** drive it on autopilot — give Claude a city + a business type, get back a clean
+list of businesses (name, address, phone, website, rating, reviews, lat/lng, emails).
+
+Built as a thin wrapper around [**`gosom/google-maps-scraper`**](https://github.com/gosom/google-maps-scraper)
+by **Georgios Komninos** (MIT). See [CREDITS.md](CREDITS.md). This kit adds: a one-command Docker setup,
+ready-to-run scripts, and a **Claude skill** so Claude knows exactly how to use the API the right way.
+
+> ⚠️ **Before you scrape — over-use can get your IP temporarily blocked.**
+> This runs a real scraper against Google Maps. Light use is fine, but big jobs back-to-back, very high
+> `depth`, many keywords, or scheduled runs **without proxies** can make Google **temporarily rate-limit or
+> block your IP** (usually clears in minutes–hours; jobs return empty/failed meanwhile). It does **not** ban
+> your Google account. Stay safe: **one job at a time, start at `depth 5`, add proxies for large/repeated
+> runs.** Scraping Maps is against Google's ToS — use responsibly and follow data law (GDPR/CCPA) for any
+> contact data. Full guidance: [Rate limits, bans & responsible use](#️-rate-limits-bans--responsible-use).
+
+---
+
+## What you get
+
+- 🐳 **One-command local setup** — `docker compose up -d` runs the scraper at `http://localhost:8080`.
+- 🤖 **A Claude skill** — open this folder in Claude Code and just say *"scrape coffee shops in Austin"*. Claude does create → poll → download → clean results, following best practices automatically.
+- 🛠️ **Standalone scripts** — `scripts/scrape.sh` (bash) and `scripts/scrape.py` (Python, no dependencies) if you'd rather not use Claude.
+- 🔒 **Safe by default** — the scraper binds to `127.0.0.1` only, secrets are git-ignored, and the skill enforces rate-limit / legal / data-handling guardrails.
+
+## What it scrapes (and what it doesn't)
+
+- ✅ **Google Maps business listings** → a clean **lead list** by default: `name, phone, email, website, category, address, rating, review count`. The scraper captures ~34 raw fields, but the kit **strips the noise** (geo coordinates, IDs, hours, images, review blobs) so you only get data you can actually use for outreach. Want everything? `scrape.py --full`.
+- ❌ **Not** a social-media scraper. It cannot touch Instagram / TikTok / YouTube.
+
+## Why use this instead of asking an AI to "just scrape Google Maps"
+
+Tested head-to-head: a chat AI fetching `maps.google.com` directly hits a **consent wall** and
+**JavaScript-rendered** content, so it returns **~0 structured results**. This kit reliably returns
+**dozens–hundreds of clean rows per query**. For Google Maps, it's the difference between *can't* and *can*.
+
+---
+
+## Quick start (3 steps)
+
+```bash
+# 1. Start the scraper (needs Docker Desktop)
+docker compose up -d
+
+# 2. Check it's alive — should print a JSON list (probably empty [])
+curl http://localhost:8080/api/v1/jobs
+
+# 3a. Scrape with the script…
+./scripts/scrape.sh "coffee shops in Austin TX" 30.2672 -97.7431 5
+
+# 3b. …or open this folder in Claude Code and say:
+# "Scrape gyms in Miami and give me phones + websites."
+```
+
+Full instructions: **[SETUP.md](SETUP.md)**. How Claude uses it: **[.claude/skills/google-maps-scraper/SKILL.md](.claude/skills/google-maps-scraper/SKILL.md)**.
+
+---
+
+## Invoking it in Claude Code
+
+### Slash commands
+| Command | What it does |
+|---|---|
+| `/scrape <business> in <city, ST> [depth]` | Run a scrape and get a clean table |
+| `/scrape-batch <keywords-file> [--city "City, ST"]` | Scrape many queries in one job |
+| `/scrape-setup` | Start the local scraper + health check |
+| `/scrape-jobs [list \| delete <id>]` | List or delete jobs |
+
+Example: `/scrape dentists in Denver CO 10`
+
+### Or just talk to it — the skill auto-triggers on phrases like:
+- "scrape coffee shops in Austin"
+- "find all gyms in Miami with phone numbers"
+- "build me a lead list of dentists in Denver, CO"
+- "pull Google Maps listings for plumbers in Phoenix"
+
+It intentionally **does not** trigger for social media (Instagram/TikTok) — that's not what it does.
+
+---
+
+## ⚠️ Rate limits, bans & responsible use
+
+The upstream project's guidance (and ours): *"Please use this scraper responsibly and in accordance with
+applicable laws and regulations. Unauthorized scraping may violate terms of service."* There's **no
+published ban threshold**, so be conservative.
+
+**What "over-use" looks like** and the risk:
+- Big jobs back-to-back, very high `depth`, many keywords at once, or scheduled runs **without proxies**.
+- Risk: Google **temporarily rate-limits/blocks your IP** (minutes–hours, then clears). Your Google
+ **account is not banned**. While blocked, jobs come back `failed` or with empty/short results.
+
+**Block signals to watch:** jobs returning `failed`, suddenly empty results, or far fewer rows than an
+identical earlier run. If you see these, **slow down** — pause, lower `depth`, or add proxies.
+
+**Stay safe:**
+- One job at a time; start at `depth 5` and raise only as needed.
+- Turn on `email` extraction only when you need emails (it's much slower).
+- Treat results as **leads to verify**, not a redistributable dataset. Don't resell raw Google data.
+- Scraped phones/emails are **personal data** → follow GDPR/CCPA/CAN-SPAM if you store or contact them.
+
+**When to add proxies** (*"For larger scraping jobs, proxies help avoid rate limiting"*): large jobs, many
+keywords, repeated/scheduled runs, or after you hit block signals. Set the `proxies` array in the job body:
+```json
+"proxies": ["socks5://user:pass@host:port", "http://host2:port2"]
+```
+Types: `socks5`, `socks5h`, `http`, `https` (auth optional). The scraper rotates them automatically.
+For reference, upstream measures ~120 places/min at `-c 8 -depth 1`.
+
+> Claude's rule here is **warn, don't block**: if you ask for a large scrape, it'll flag the risk and
+> suggest proxies, then run it anyway. It only refuses clearly abusive use.
+
+---
+
+## What's inside
+
+```
+google-maps-scraper-kit/
+├── README.md ← you are here
+├── SETUP.md ← detailed, copy-paste setup (start here)
+├── CLAUDE.md ← auto-loaded instructions for Claude Code
+├── CREDITS.md ← attribution to the upstream author (gosom)
+├── LICENSE ← MIT (this kit)
+├── docker-compose.yml ← runs the scraper locally on :8080
+├── .env.example ← config template
+├── .gitignore
+├── scripts/
+│ ├── scrape.sh ← one-shot bash scraper (single keyword)
+│ └── scrape.py ← Python scraper: single, batch, + auto-geocoding (stdlib only)
+├── examples/
+│ ├── queries.example.json ← reference job body + coordinate cheatsheet
+│ └── queries.example.txt ← batch keyword list (one per line)
+└── .claude/
+ ├── settings.json ← pre-approves local commands so Claude runs on autopilot
+ ├── commands/ ← /scrape, /scrape-batch, /scrape-setup, /scrape-jobs
+ └── skills/
+ └── google-maps-scraper/
+ └── SKILL.md ← THE skill that teaches Claude to use the API well
+```
+
+## Credits & license
+
+This kit orchestrates **[google-maps-scraper](https://github.com/gosom/google-maps-scraper)** by
+**Georgios Komninos**, used under the MIT License. The kit's own code is MIT licensed too. Full
+attribution in **[CREDITS.md](CREDITS.md)**. Please respect Google's Terms of Service and applicable
+data-protection law (GDPR/CCPA) when using scraped data — see the Safety section in the skill.
diff --git a/SETUP.md b/SETUP.md
new file mode 100644
index 0000000..1a43dd8
--- /dev/null
+++ b/SETUP.md
@@ -0,0 +1,187 @@
+# SETUP — Google Maps Scraper Kit
+
+This guide sets up the whole system **on your computer** from scratch. It's written so that a person
+*or* Claude can follow it exactly. Total time: ~5 minutes (most of it is Docker pulling the image).
+
+---
+
+## 0. Prerequisites
+
+Install these first (one-time):
+
+| Tool | Why | Get it |
+|---|---|---|
+| **Docker Desktop** | runs the scraper container | https://www.docker.com/products/docker-desktop |
+| **Claude Code** *(optional but recommended)* | lets Claude drive the scraper for you | https://claude.com/claude-code |
+| **git** *(optional)* | only if you'll push this to GitHub | https://git-scm.com |
+
+Verify Docker is running:
+```bash
+docker --version # should print a version
+docker ps # should NOT error (means the daemon is up)
+```
+
+> **Platform notes:**
+> - **Apple Silicon (M1/M2/M3):** the scraper image is `linux/amd64` and runs under emulation. If the
+> container won't start, uncomment the `platform: linux/amd64` line in `docker-compose.yml`.
+> - **Windows:** use `scripts/scrape.py` (run with `py` or `python3`). The bash script `scrape.sh` needs
+> **WSL2** or **Git Bash**.
+
+---
+
+## 1. Get the kit
+
+If someone shared this folder with you, just `cd` into it:
+```bash
+cd google-maps-scraper-kit
+```
+(Or clone it from GitHub if it's hosted: `git clone <repo-url> && cd google-maps-scraper-kit`.)
+
+Create your local config (optional — defaults work as-is):
+```bash
+cp .env.example .env
+```
+
+---
+
+## 2. Start the scraper
+
+```bash
+docker compose up -d
+```
+
+First run pulls the `gosom/google-maps-scraper` image (~once, a few hundred MB). When it finishes:
+
+```bash
+docker ps | grep gmaps-scraper # should show it "Up"
+curl http://localhost:8080/api/v1/jobs # should print [] (empty list) or existing jobs
+```
+
+You can also open the built-in UI and API docs in a browser:
+- Web UI: **http://localhost:8080**
+- OpenAPI docs: **http://localhost:8080/api/docs**
+
+✅ If `curl` returns a JSON array, the system is live.
+
+> ⚠️ **Now that it's running — read this once.** This hits Google Maps for real. Light use is fine, but
+> big jobs back-to-back, very high `depth`, many keywords, or scheduled runs **without proxies** can get
+> your **IP temporarily rate-limited/blocked by Google** (clears in minutes–hours; jobs return empty/failed
+> meanwhile). It does **not** ban your Google account. **Stay safe:** one job at a time, start at `depth 5`,
+> and add **proxies** for large or repeated runs. Scraping Maps is against Google's ToS — use responsibly
+> and follow data law (GDPR/CCPA) for any contact data. (Full guidance in README → *Rate limits, bans &
+> responsible use*.)
+
+---
+
+## 3. Run your first scrape (without Claude)
+
+Use the included script. Arguments: `"<keyword>" <lat> <lon> [depth]`
+
+```bash
+./scripts/scrape.sh "coffee shops in Austin TX" 30.2672 -97.7431 5
+```
+
+or the Python version (no dependencies needed):
+
+```bash
+python3 scripts/scrape.py "gyms in Miami FL" 25.7617 -80.1918 --depth 5
+```
+
+The script will **create the job → poll until done → download the CSV → print a summary** and save
+the full results to a file.
+
+**Don't want to look up coordinates?** Let it geocode the city for you:
+```bash
+python3 scripts/scrape.py "cafes in Austin TX" --city "Austin, TX" --depth 5
+```
+
+**Scrape many queries at once** (one job, many keywords) from a file:
+```bash
+python3 scripts/scrape.py --keywords-file examples/queries.example.txt --city "Denver, CO"
+```
+
+---
+
+## 4. Use it with Claude (the autopilot way) 🤖
+
+This is the point of the kit. Two parts make it work:
+
+1. **[CLAUDE.md](CLAUDE.md)** — auto-loaded when you open this folder in Claude Code. It tells Claude
+ what this project is and the key rules.
+2. **[.claude/skills/google-maps-scraper/SKILL.md](.claude/skills/google-maps-scraper/SKILL.md)** — a
+ **skill** Claude loads on demand. It teaches Claude the exact API flow + best practices.
+
+### How to connect it to Claude
+There's nothing to "connect" — the scraper is a local HTTP API at `http://localhost:8080`, and Claude
+calls it with its normal `curl`/Bash ability. Just:
+
+```bash
+cd google-maps-scraper-kit
+claude # start Claude Code inside this folder
+```
+
+Then talk to it naturally:
+> "Make sure the scraper is running, then scrape **dentists in Denver CO**, depth 10, and give me a table of name, phone, website."
+
+Claude will: check the container is up (start it if not) → build the correct job (with the required
+`max_time`, `lat`, `lon`) → poll in the background → download → hand you clean rows. It follows the
+safety/best-practice rules in the skill automatically.
+
+> ℹ️ **Advanced (optional):** prefer a Model Context Protocol server instead of curl? You can wrap the
+> API as an MCP tool, but it's unnecessary for a local single-user setup — the skill + local API is
+> simpler and just as capable.
+
+---
+
+### Slash-command shortcuts
+Inside Claude Code you can also use these commands (type `/` to see them):
+
+| Command | What it does |
+|---|---|
+| `/scrape <business> in <city, ST> [depth]` | Run a scrape, get a clean table |
+| `/scrape-batch <keywords-file> [--city "City, ST"]` | Scrape many queries in one job |
+| `/scrape-setup` | Start the scraper + health check |
+| `/scrape-jobs [list \| delete <id>]` | List or delete jobs |
+
+…or just ask in plain English: *"scrape gyms in Miami with phone numbers."* (The scraper's commands and
+`localhost` calls are pre-approved in `.claude/settings.json`, so Claude won't prompt for each one.)
+
+---
+
+## 5. Stop / clean up
+
+```bash
+docker compose stop # pause (keeps data + image)
+docker compose down # stop + remove container (keeps named volumes/data)
+docker compose down -v # ALSO delete scraped data volumes (full reset)
+```
+
+Delete old jobs to free disk (results accumulate in the data volume):
+```bash
+curl -X DELETE http://localhost:8080/api/v1/jobs/<job-id>
+```
+
+---
+
+## 6. Troubleshooting
+
+| Symptom | Fix |
+|---|---|
+| `curl: connection refused` on :8080 | Container not up. `docker compose up -d`, then `docker ps`. |
+| `422 missing max time` | Job body needs `max_time` (in **seconds**, e.g. `300`). |
+| `422 missing geo coordinates` | Job body needs `lat` and `lon` as **strings**, e.g. `"30.2672"`. |
+| Job stuck `working` forever | Lower `depth`, or the IP is being throttled by Google — wait, or add proxies (see skill). |
+| Empty CSV | Keyword too narrow or wrong geo — widen `radius` or fix `lat`/`lon`. |
+| Docker pull is slow | Normal on first run; it caches after that. |
+
+---
+
+## 7. Best practices & safety (summary)
+
+The full rules live in the skill, but the essentials:
+- **Always** send `max_time` (seconds) and `lat`/`lon` (strings).
+- Start with **low `depth`** (5) and **one job at a time**. Raise only when needed.
+- Turn on **email extraction** only when you need emails — it's much slower.
+- For big jobs, add **proxies** (the scraper has built-in rotation) to avoid IP blocks.
+- Scraped **phones/emails are personal data** — comply with GDPR/CCPA/CAN-SPAM, and respect Google's ToS.
+- Never commit result files (they're git-ignored for you).
diff --git a/assets/banner.svg b/assets/banner.svg
new file mode 100644
index 0000000..fd01448
--- /dev/null
+++ b/assets/banner.svg
@@ -0,0 +1,83 @@
+<svg width="1280" height="360" viewBox="0 0 1280 360" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Google Maps Scraper Kit">
+ <defs>
+ <linearGradient id="bg" x1="0" y1="0" x2="1" y2="1">
+ <stop offset="0" stop-color="#0b1220"/>
+ <stop offset="1" stop-color="#111c33"/>
+ </linearGradient>
+ <linearGradient id="pin" x1="0" y1="0" x2="0" y2="1">
+ <stop offset="0" stop-color="#34d399"/>
+ <stop offset="1" stop-color="#059669"/>
+ </linearGradient>
+ <filter id="shadow" x="-40%" y="-40%" width="180%" height="180%">
+ <feDropShadow dx="0" dy="8" stdDeviation="12" flood-color="#000" flood-opacity="0.40"/>
+ </filter>
+ <style>
+ .t { font-family: -apple-system,Segoe UI,Roboto,Helvetica,Arial,sans-serif; }
+ .mono { font-family: ui-monospace,SFMono-Regular,Menlo,Consolas,monospace; }
+ </style>
+ </defs>
+
+ <!-- background -->
+ <rect width="1280" height="360" fill="url(#bg)"/>
+
+ <!-- faint map grid + roads -->
+ <g stroke="#1e293b" stroke-width="1" opacity="0.5">
+ <path d="M0 72 H1280 M0 144 H1280 M0 216 H1280 M0 288 H1280"/>
+ <path d="M160 0 V360 M360 0 V360 M560 0 V360 M760 0 V360 M960 0 V360 M1160 0 V360"/>
+ </g>
+ <g stroke="#10b981" stroke-width="3" opacity="0.16" fill="none" stroke-linecap="round">
+ <path d="M-20 270 C 200 250, 280 110, 520 135 S 980 110, 1300 50"/>
+ <path d="M-20 80 C 260 100, 420 225, 720 215 S 1100 270, 1300 230"/>
+ </g>
+
+ <!-- map pin with magnifier -->
+ <g transform="translate(140,182) scale(0.82)" filter="url(#shadow)">
+ <path d="M0,-95 C 52,-95 96,-54 96,-2 C 96,58 18,112 0,150 C -18,112 -96,58 -96,-2 C -96,-54 -52,-95 0,-95 Z" fill="url(#pin)"/>
+ <circle cx="0" cy="-6" r="40" fill="#0b1220"/>
+ <circle cx="-6" cy="-13" r="19" fill="none" stroke="#34d399" stroke-width="6"/>
+ <line x1="9" y1="6" x2="27" y2="24" stroke="#34d399" stroke-width="8" stroke-linecap="round"/>
+ </g>
+
+ <!-- title + subtitle (kept clear of the card on the right) -->
+ <text x="300" y="128" class="t" font-weight="800" font-size="44" fill="#f8fafc">Google Maps Scraper Kit</text>
+ <text x="302" y="170" class="t" font-weight="400" font-size="21" fill="#94a3b8">Run a free Google&#160;Maps scraper locally, driven by Claude.</text>
+
+ <!-- chips -->
+ <g transform="translate(300,200)">
+ <g>
+ <rect x="0" y="0" rx="22" ry="22" width="170" height="42" fill="#0f213b" stroke="#1f3a5f"/>
+ <circle cx="24" cy="21" r="6" fill="#38bdf8"/>
+ <text x="40" y="28" class="t" font-weight="600" font-size="19" fill="#cbd5e1">Local Docker</text>
+ </g>
+ <g transform="translate(186,0)">
+ <rect x="0" y="0" rx="22" ry="22" width="158" height="42" fill="#0f213b" stroke="#1f3a5f"/>
+ <circle cx="24" cy="21" r="6" fill="#a78bfa"/>
+ <text x="40" y="28" class="t" font-weight="600" font-size="19" fill="#cbd5e1">Claude skill</text>
+ </g>
+ <g transform="translate(360,0)">
+ <rect x="0" y="0" rx="22" ry="22" width="158" height="42" fill="#0f213b" stroke="#1f3a5f"/>
+ <circle cx="24" cy="21" r="6" fill="#34d399"/>
+ <text x="40" y="28" class="t" font-weight="600" font-size="19" fill="#cbd5e1">Clean leads</text>
+ </g>
+ </g>
+
+ <!-- footer url -->
+ <text x="302" y="296" class="mono" font-size="17" fill="#64748b">github.com/Mahanaicoach/google-maps-scraper-kit</text>
+
+ <!-- lead-list card (right side, no overlap with text) -->
+ <g transform="translate(972,86)" filter="url(#shadow)">
+ <rect x="0" y="0" rx="16" ry="16" width="268" height="188" fill="#0f1a30" stroke="#24324d"/>
+ <text x="20" y="34" class="t" font-size="13" letter-spacing="2" fill="#64748b">LEAD&#160;LIST</text>
+ <g>
+ <circle cx="28" cy="72" r="7" fill="#34d399"/>
+ <rect x="46" y="64" rx="5" width="150" height="14" fill="#334155"/>
+ <rect x="46" y="84" rx="5" width="104" height="9" fill="#1f2c44"/>
+ <circle cx="28" cy="116" r="7" fill="#34d399"/>
+ <rect x="46" y="108" rx="5" width="120" height="14" fill="#334155"/>
+ <rect x="46" y="128" rx="5" width="140" height="9" fill="#1f2c44"/>
+ <circle cx="28" cy="160" r="7" fill="#34d399"/>
+ <rect x="46" y="152" rx="5" width="160" height="14" fill="#334155"/>
+ <rect x="46" y="172" rx="5" width="96" height="9" fill="#1f2c44"/>
+ </g>
+ </g>
+</svg>
diff --git a/docker-compose.yml b/docker-compose.yml
new file mode 100644
index 0000000..f090f2d
--- /dev/null
+++ b/docker-compose.yml
@@ -0,0 +1,24 @@
+# Runs the open-source Google Maps scraper (gosom/google-maps-scraper) locally.
+# Web/API mode on http://localhost:8080. The image has NO built-in auth, so we bind
+# to 127.0.0.1 ONLY — that localhost binding is the security boundary. Do not change
+# the port mapping to 0.0.0.0 / a public interface without putting an auth proxy in front.
+
+services:
+ google-maps-scraper:
+ # Pinned to a known-good release for reproducibility (bump deliberately).
+ # The published image is linux/amd64. On Apple Silicon (M1/M2/M3) it runs under
+ # emulation; if it won't start, UNCOMMENT the 'platform' line below.
+ image: gosom/google-maps-scraper:v1.15.0
+ # platform: linux/amd64
+ container_name: gmaps-scraper
+ restart: unless-stopped
+ command: ["-web", "-data-folder", "/gmapsdata"]
+ ports:
+ - "127.0.0.1:8080:8080" # localhost only — see warning above
+ volumes:
+ - gmaps_data:/gmapsdata # job results persist here
+ - gmaps_cache:/opt # Playwright browser cache (faster restarts)
+
+volumes:
+ gmaps_data:
+ gmaps_cache:
diff --git a/examples/queries.example.json b/examples/queries.example.json
new file mode 100644
index 0000000..4103d40
--- /dev/null
+++ b/examples/queries.example.json
@@ -0,0 +1,31 @@
+{
+ "_comment": "Reference request body for POST /api/v1/jobs. Copy the 'job' object and edit. Required fields: keywords, lat, lon (strings), max_time (seconds).",
+ "job": {
+ "name": "my scrape",
+ "keywords": ["coffee shops in Austin TX"],
+ "lang": "en",
+ "zoom": 15,
+ "lat": "30.2672",
+ "lon": "-97.7431",
+ "fast_mode": false,
+ "radius": 10000,
+ "depth": 5,
+ "email": false,
+ "max_time": 300,
+ "proxies": null
+ },
+ "keyword_ideas": [
+ "dentists in Denver CO",
+ "plumbers in Phoenix AZ",
+ "yoga studios in Portland OR",
+ "barbershops in Brooklyn NY",
+ "wedding photographers in Nashville TN"
+ ],
+ "city_coordinates_cheatsheet": {
+ "Austin, TX": ["30.2672", "-97.7431"],
+ "Miami, FL": ["25.7617", "-80.1918"],
+ "Denver, CO": ["39.7392", "-104.9903"],
+ "New York, NY": ["40.7128", "-74.0060"],
+ "Los Angeles, CA": ["34.0522", "-118.2437"]
+ }
+}
diff --git a/examples/queries.example.txt b/examples/queries.example.txt
new file mode 100644
index 0000000..2ba52c5
--- /dev/null
+++ b/examples/queries.example.txt
@@ -0,0 +1,7 @@
+# Batch keywords — one per line. Lines starting with # are ignored.
+# Bake the location into each line, OR pass --city to geocode one place for all of them.
+# Run: python3 scripts/scrape.py --keywords-file examples/queries.example.txt --city "Denver, CO"
+dentists in Denver CO
+orthodontists in Denver CO
+dental hygienists in Denver CO
+pediatric dentists in Denver CO
diff --git a/scripts/scrape.py b/scripts/scrape.py
new file mode 100755
index 0000000..10bd964
--- /dev/null
+++ b/scripts/scrape.py
@@ -0,0 +1,163 @@
+#!/usr/bin/env python3
+"""One-shot Google Maps scrape using only the Python standard library (no pip installs).
+
+Single keyword:
+ python3 scripts/scrape.py "gyms in Miami FL" 25.7617 -80.1918 --depth 5
+
+Auto-geocode (no coordinates needed):
+ python3 scripts/scrape.py "cafes in Austin TX" --city "Austin, TX" --depth 5
+
+Batch (one job, many keywords) from a file (one keyword per line):
+ python3 scripts/scrape.py --keywords-file examples/queries.example.txt --city "Denver, CO"
+
+Notes:
+- Required by the API: keywords, lat/lon (strings), max_time (SECONDS). This script fills them in.
+- Geocoding uses OpenStreetMap Nominatim (free, no key). Please be gentle: it allows ~1 request/sec
+ and requires a descriptive User-Agent (set below). Don't loop it aggressively.
+"""
+import argparse, csv, io, json, os, sys, time, urllib.request, urllib.parse, urllib.error
+
+BASE = os.environ.get("SCRAPER_BASE_URL", "http://localhost:8080")
+KEY = os.environ.get("SCRAPER_API_KEY", "")
+# Money-useful LEAD fields only — what you actually use to contact/qualify a lead.
+# Everything else (geo coordinates, IDs, hours, images, reviews blobs…) is dropped by default.
+LEAD = ["title", "phone", "emails", "website", "category", "address", "review_rating", "review_count"]
+UA = "google-maps-scraper-kit/1.0 (https://github.com/Mahanaicoach/google-maps-scraper-kit)"
+
+
+def req(method, path, body=None):
+ headers = {"Content-Type": "application/json", "User-Agent": UA}
+ if KEY:
+ headers["X-API-Key"] = KEY
+ data = json.dumps(body).encode() if body is not None else None
+ r = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
+ with urllib.request.urlopen(r, timeout=60) as resp:
+ return resp.status, resp.read()
+
+
+def geocode(place):
+ """City/place name -> ('lat','lon') strings via Nominatim, or None."""
+ q = urllib.parse.urlencode({"format": "json", "limit": 1, "q": place})
+ url = f"https://nominatim.openstreetmap.org/search?{q}"
+ r = urllib.request.Request(url, headers={"User-Agent": UA})
+ try:
+ with urllib.request.urlopen(r, timeout=30) as resp:
+ hits = json.loads(resp.read())
+ time.sleep(1) # respect Nominatim's ~1 req/sec policy
+ if hits:
+ return str(hits[0]["lat"]), str(hits[0]["lon"])
+ except Exception as e:
+ print(f" (geocoding failed: {e})", file=sys.stderr)
+ return None
+
+
+def collect_keywords(a):
+ kws = []
+ if a.keyword:
+ kws.append(a.keyword)
+ kws.extend(a.also or [])
+ if a.keywords_file:
+ with open(a.keywords_file) as f:
+ kws.extend(line.strip() for line in f if line.strip() and not line.startswith("#"))
+ # de-dupe, keep order
+ seen, out = set(), []
+ for k in kws:
+ if k not in seen:
+ seen.add(k); out.append(k)
+ return out
+
+
+def main():
+ ap = argparse.ArgumentParser(description="Scrape Google Maps business listings.")
+ ap.add_argument("keyword", nargs="?", help='e.g. "coffee shops in Austin TX"')
+ ap.add_argument("lat", nargs="?", help="latitude (optional if --city given)")
+ ap.add_argument("lon", nargs="?", help="longitude (optional if --city given)")
+ ap.add_argument("--keyword", dest="also", action="append", help="extra keyword (repeatable)")
+ ap.add_argument("--keywords-file", help="file with one keyword per line (batch)")
+ ap.add_argument("--city", help="city/place to auto-geocode for lat/lon")
+ ap.add_argument("--depth", type=int, default=5)
+ ap.add_argument("--email", action="store_true", help="also extract emails (much slower)")
+ ap.add_argument("--max-time", type=int, default=600, help="job time limit in SECONDS")
+ ap.add_argument("--out", default=None, help="write parsed JSON results here")
+ ap.add_argument("--full", action="store_true", help="keep ALL raw columns (default: lean lead fields)")
+ ap.add_argument("--fields", help="comma-separated columns to keep (overrides the default lead set)")
+ a = ap.parse_args()
+
+ keywords = collect_keywords(a)
+ if not keywords:
+ sys.exit("✗ No keywords. Pass a keyword, --keyword, or --keywords-file.")
+
+ # Resolve coordinates: explicit lat/lon, else geocode --city, else geocode first keyword.
+ lat, lon = a.lat, a.lon
+ if not (lat and lon):
+ place = a.city or keywords[0]
+ print(f"▶ Geocoding \"{place}\"…")
+ coords = geocode(place)
+ if not coords:
+ sys.exit("✗ Could not resolve coordinates. Pass lat/lon or a clearer --city.")
+ lat, lon = coords
+ print(f" → {lat}, {lon}")
+
+ # --- WARN, don't block: flag high-volume jobs (proceed anyway) ---
+ if a.depth >= 15 or len(keywords) >= 10 or a.email:
+ print("⚠️ Heads-up: this is a large/slow job (high depth, many keywords, or email extraction).")
+ print(" Running it back-to-back without proxies can get your IP temporarily rate-limited by")
+ print(" Google (clears in minutes–hours; returns empty/failed jobs meanwhile). For big or")
+ print(" repeated runs, add proxies. Proceeding…\n")
+
+ # Health check
+ try:
+ req("GET", "/api/v1/jobs")
+ except Exception as e:
+ sys.exit(f"✗ Scraper not reachable at {BASE} — run 'docker compose up -d' first.\n ({e})")
+
+ body = {"name": "scrape-py", "keywords": keywords, "lang": "en", "zoom": 15,
+ "lat": str(lat), "lon": str(lon), "fast_mode": False, "radius": 10000,
+ "depth": a.depth, "email": a.email, "max_time": a.max_time}
+ print(f"▶ Creating job: {len(keywords)} keyword(s) @ {lat},{lon} depth={a.depth} email={a.email}")
+ for k in keywords:
+ print(f" • {k}")
+ try:
+ _, raw = req("POST", "/api/v1/jobs", body)
+ except urllib.error.HTTPError as e:
+ sys.exit(f"✗ Create failed: HTTP {e.code} — {e.read().decode()[:200]}")
+ job_id = json.loads(raw).get("id")
+ if not job_id:
+ sys.exit("✗ No job id returned.")
+ print(f" job id: {job_id}")
+
+ print("▶ Polling…")
+ status = None
+ for i in range(120):
+ _, raw = req("GET", f"/api/v1/jobs/{job_id}")
+ status = json.loads(raw).get("Status")
+ print(f"\r status: {str(status):<10} (attempt {i + 1})", end="", flush=True)
+ if status == "ok":
+ print(); break
+ if status == "failed":
+ sys.exit("\n✗ Job failed. If this keeps happening you may be rate-limited — wait or add proxies.")
+ time.sleep(8)
+ else:
+ sys.exit("\n✗ Timed out.")
+
+ _, raw = req("GET", f"/api/v1/jobs/{job_id}/download")
+ rows = list(csv.DictReader(io.StringIO(raw.decode("utf-8", "replace"))))
+ if a.full:
+ fields = list(rows[0].keys()) if rows else LEAD
+ elif a.fields:
+ fields = [c.strip() for c in a.fields.split(",") if c.strip()]
+ else:
+ fields = LEAD
+ results = [{k: r.get(k, "") for k in fields} for r in rows]
+ print(f"✓ Done — {len(results)} businesses (fields: {', '.join(fields)}).")
+
+ out = a.out or f"results-{job_id[:8]}.json"
+ with open(out, "w") as f:
+ json.dump(results, f, indent=2, ensure_ascii=False)
+ print(f" saved → {out}")
+ for r in results[:5]:
+ print(f" • {r.get('title','')} | {r.get('phone','')} | {r.get('emails','')} | {r.get('website','')}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/scripts/scrape.sh b/scripts/scrape.sh
new file mode 100755
index 0000000..a1e633b
--- /dev/null
+++ b/scripts/scrape.sh
@@ -0,0 +1,69 @@
+#!/usr/bin/env bash
+# One-shot Google Maps scrape: create job -> poll -> download -> summarize.
+# Usage: ./scripts/scrape.sh "<keyword>" <lat> <lon> [depth]
+# Example: ./scripts/scrape.sh "coffee shops in Austin TX" 30.2672 -97.7431 5
+set -euo pipefail
+
+BASE="${SCRAPER_BASE_URL:-http://localhost:8080}"
+KEY="${SCRAPER_API_KEY:-}"
+AUTH=(); [ -n "$KEY" ] && AUTH=(-H "X-API-Key: $KEY")
+
+KEYWORD="${1:-}"; LAT="${2:-}"; LON="${3:-}"; DEPTH="${4:-5}"
+if [ -z "$KEYWORD" ] || [ -z "$LAT" ] || [ -z "$LON" ]; then
+ echo "Usage: $0 \"<keyword>\" <lat> <lon> [depth]" >&2
+ echo "Example: $0 \"coffee shops in Austin TX\" 30.2672 -97.7431 5" >&2
+ exit 1
+fi
+
+# Is the scraper up?
+if ! curl -s -m 5 "${AUTH[@]}" "$BASE/api/v1/jobs" >/dev/null 2>&1; then
+ echo "✗ Scraper not reachable at $BASE — run 'docker compose up -d' first." >&2
+ exit 1
+fi
+
+echo "▶ Creating job: \"$KEYWORD\" @ $LAT,$LON depth=$DEPTH"
+BODY=$(printf '{"name":"scrape-cli","keywords":["%s"],"lang":"en","zoom":15,"lat":"%s","lon":"%s","fast_mode":false,"radius":10000,"depth":%s,"email":false,"max_time":600}' \
+ "$KEYWORD" "$LAT" "$LON" "$DEPTH")
+ID=$(curl -s -X POST "$BASE/api/v1/jobs" "${AUTH[@]}" -H "Content-Type: application/json" -d "$BODY" \
+ | grep -oE '[a-f0-9-]{36}' | head -1 || true)
+[ -n "$ID" ] || { echo "✗ Failed to create job (check required fields)." >&2; exit 1; }
+echo " job id: $ID"
+
+echo "▶ Polling (up to ~10 min)…"
+for i in $(seq 1 60); do
+ # '|| true' keeps the script alive under 'set -euo pipefail' when grep finds no match yet
+ S=$(curl -s "${AUTH[@]}" "$BASE/api/v1/jobs/$ID" | grep -oE '"Status":"[^"]*"' | head -1 | cut -d'"' -f4 || true)
+ printf '\r status: %-10s (attempt %d)' "${S:-?}" "$i"
+ [ "$S" = ok ] && { echo; break; }
+ [ "$S" = failed ] && { echo; echo "✗ Job failed." >&2; exit 1; }
+ sleep 10
+done
+[ "$S" = ok ] || { echo; echo "✗ Timed out waiting for job." >&2; exit 1; }
+
+OUT="results-$(echo "$KEYWORD" | tr ' /' '__' | tr -cd 'A-Za-z0-9_-').csv"
+RAW="$(mktemp)"
+echo "▶ Downloading…"
+curl -s "${AUTH[@]}" "$BASE/api/v1/jobs/$ID/download" -o "$RAW"
+
+# Trim the raw 34-column dump down to money-useful LEAD fields only (drops geo coords, IDs, hours, images…).
+if command -v python3 >/dev/null 2>&1; then
+ python3 - "$RAW" "$OUT" <<'PY'
+import csv, sys
+LEAD = ["title", "phone", "emails", "website", "category", "address", "review_rating", "review_count"]
+with open(sys.argv[1], newline='') as f:
+ rows = list(csv.DictReader(f))
+with open(sys.argv[2], "w", newline='') as f:
+ w = csv.DictWriter(f, fieldnames=LEAD); w.writeheader()
+ for r in rows:
+ w.writerow({k: r.get(k, "") for k in LEAD})
+print(f"✓ Done — {len(rows)} leads saved to {sys.argv[2]}")
+print(" (name, phone, email, website, category, address, rating, reviews)\n")
+print("Preview (name | phone | email | website):")
+for r in rows[:6]:
+ print(f" • {r.get('title','')} | {r.get('phone','')} | {r.get('emails','')} | {r.get('website','')}")
+PY
+ rm -f "$RAW"
+else
+ mv "$RAW" "$OUT"
+ echo "✓ Done — raw CSV saved to $OUT (install python3 to auto-trim to lead fields only)"
+fi