Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.
Python agent-skillsb2b-leadsclaude-codeclaude-skillscodex-skillsemail-extractoremail-scrapergoogle-mapsgoogle-maps-extractorgoogle-maps-leadsgoogle-maps-scrapergoogle-maps-scraper-pythongoogle-maps-scrapinggooglemaps-scraperheadless-chromelead-generationlead-scraperno-api-keypythonweb-scraping
fix: read the host port from .env in compose and both scripts (#9)
Boulanouar Walidd08a23836aa9
diff --git a/.env.example b/.env.exampleindex 9b4b0bd..bbac372 100644--- a/.env.example+++ b/.env.example@@ -3,7 +3,14 @@# This local setup binds the scraper to 127.0.0.1 only, so NO API key is required.# (The scraper image has no built-in authentication — localhost binding is the boundary.)-SCRAPER_BASE_URL=http://localhost:8080+# Host port for the scraper. Change it if 8080 is already taken on your machine, then run+# "docker compose up -d" again. Compose and both scripts read it from this file.+# The port inside the container stays 8080.+SCRAPER_PORT=8080++# Only set this if the scraper runs somewhere other than http://localhost:<SCRAPER_PORT>.+# When set, the scripts use it and ignore SCRAPER_PORT.+# SCRAPER_BASE_URL=http://localhost:8080# Only needed if you later expose the scraper beyond localhost behind an auth proxy# (e.g. Caddy/nginx that checks an X-API-Key header). Leave blank for local use.diff --git a/SETUP.md b/SETUP.mdindex fe6a177..8f23d14 100644--- a/SETUP.md+++ b/SETUP.md@@ -168,6 +168,7 @@ curl -X DELETE http://localhost:8080/api/v1/jobs/<job-id>| Symptom | Fix ||---|---|| `curl: connection refused` on :8080 | Container not up. `docker compose up -d`, then `docker ps`. |+| `port is already allocated` / something else already uses 8080 | Set `SCRAPER_PORT=8081` in `.env`, then `docker compose up -d`. Compose publishes the scraper on that host port and `scripts/scrape.sh` and `scripts/scrape.py` read the same file, so nothing else needs editing. The port inside the container stays 8080 (do not change the right-hand side of the mapping). If your `.env` still has a `SCRAPER_BASE_URL` line, delete it or update its port: it overrides `SCRAPER_PORT`. The slash commands and the skill show `localhost:8080` in their examples, so tell Claude the port you picked. || `422 missing max time` | Job body needs `max_time` (in **seconds**, e.g. `300`). || `422 missing geo coordinates` | Job body needs `lat` and `lon` as **strings**, e.g. `"30.2672"`. || Job stuck `working` forever | Lower `depth`, or the IP is being throttled by Google — wait, or add proxies (see skill). |diff --git a/docker-compose.yml b/docker-compose.ymlindex 0483727..8bd1e06 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -1,5 +1,5 @@# Runs the open-source Google Maps scraper (gosom/google-maps-scraper) locally.-# Web/API mode on http://localhost:8080. The image has NO built-in auth, so we bind+# Web/API mode on http://localhost:8080 (host port: SCRAPER_PORT in .env, default 8080). The image has NO built-in auth, so we bind# to 127.0.0.1 ONLY — that localhost binding is the security boundary. Do not change# the port mapping to 0.0.0.0 / a public interface without putting an auth proxy in front.@@ -17,7 +17,8 @@ services:restart: unless-stoppedcommand: ["-web", "-data-folder", "/gmapsdata"]ports:- - "127.0.0.1:8080:8080" # localhost only — see warning above+ # Host port comes from SCRAPER_PORT in .env. The container side stays 8080.+ - "127.0.0.1:${SCRAPER_PORT:-8080}:8080" # localhost only — see warning abovevolumes:- gmaps_data:/gmapsdata # job results persist here- gmaps_cache:/opt # Playwright browser cache (faster restarts)diff --git a/scripts/scrape.py b/scripts/scrape.pyindex 97652b7..a3219aa 100755--- a/scripts/scrape.py+++ b/scripts/scrape.py@@ -18,7 +18,25 @@ Notes:import argparse, csv, io, json, os, re, sys, time, urllib.request, urllib.parse, urllib.errorfrom concurrent.futures import ThreadPoolExecutor-BASE = os.environ.get("SCRAPER_BASE_URL", "http://localhost:8080")+ENV_KEYS = ("SCRAPER_BASE_URL", "SCRAPER_PORT", "SCRAPER_API_KEY")+++def load_env_file():+ """Read SCRAPER_* from the kit's .env, the same file docker compose reads, so a remapped+ port needs one edit. Variables already set in the environment win over the file."""+ path = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, ".env")+ try:+ with open(path) as f:+ for line in f:+ k, sep, v = line.strip().partition("=")+ if sep and k in ENV_KEYS and k not in os.environ:+ os.environ[k] = v.strip().strip("\"'")+ except OSError:+ pass+++load_env_file()+BASE = os.environ.get("SCRAPER_BASE_URL") or f"http://localhost:{os.environ.get('SCRAPER_PORT') or '8080'}"KEY = os.environ.get("SCRAPER_API_KEY", "")# Money-useful LEAD fields only — what you actually use to contact/qualify a lead.# Everything else (geo coordinates, IDs, hours, images, reviews blobs…) is dropped by default.diff --git a/scripts/scrape.sh b/scripts/scrape.shindex aef4ffa..5be0f5c 100755--- a/scripts/scrape.sh+++ b/scripts/scrape.sh@@ -4,7 +4,18 @@# Example: ./scripts/scrape.sh "coffee shops in Austin TX" 30.2672 -97.7431 5set -euo pipefail-BASE="${SCRAPER_BASE_URL:-http://localhost:8080}"+# Read SCRAPER_* from the kit's .env, the same file docker compose reads, so a remapped+# port needs one edit. Variables already exported in the shell win over the file.+ENV_FILE="$(cd "$(dirname "$0")/.." && pwd)/.env"+if [ -f "$ENV_FILE" ]; then+ while IFS='=' read -r k v || [ -n "$k" ]; do+ case "$k" in SCRAPER_BASE_URL|SCRAPER_PORT|SCRAPER_API_KEY) ;; *) continue ;; esac+ v="${v%$'\r'}"; v="${v#\"}"; v="${v%\"}"; v="${v#\'}"; v="${v%\'}"+ [ -n "${!k+x}" ] || export "$k=$v"+ done < "$ENV_FILE"+fi++BASE="${SCRAPER_BASE_URL:-http://localhost:${SCRAPER_PORT:-8080}}"KEY="${SCRAPER_API_KEY:-}"# NOTE: empty-array expansion needs the ${a[@]+...} guard — plain "${AUTH[@]}" is an# unbound-variable error under 'set -u' on bash 3.2 (stock macOS), which is the