Skip to content

Free, open-source Google Maps scraper for lead generation (Python, no API key, no Docker). Exports name, phone, email, website, rating and Instagram/Facebook/LinkedIn/TikTok/YouTube/X links to CSV. 100 places with emails in 44 s; --grid beats Google's 120 cap. MX email check, ad-pixel detection. Skill for Claude Code, Codex and Cursor.

fix: read the host port from .env in compose and both scripts (#9)

Boulanouar Walidd08a23836aa9
diff --git a/.env.example b/.env.example
index 9b4b0bd..bbac372 100644
--- a/.env.example
+++ b/.env.example
@@ -3,7 +3,14 @@
# This local setup binds the scraper to 127.0.0.1 only, so NO API key is required.
# (The scraper image has no built-in authentication — localhost binding is the boundary.)
-SCRAPER_BASE_URL=http://localhost:8080
+# Host port for the scraper. Change it if 8080 is already taken on your machine, then run
+# "docker compose up -d" again. Compose and both scripts read it from this file.
+# The port inside the container stays 8080.
+SCRAPER_PORT=8080
+
+# Only set this if the scraper runs somewhere other than http://localhost:<SCRAPER_PORT>.
+# When set, the scripts use it and ignore SCRAPER_PORT.
+# SCRAPER_BASE_URL=http://localhost:8080
# Only needed if you later expose the scraper beyond localhost behind an auth proxy
# (e.g. Caddy/nginx that checks an X-API-Key header). Leave blank for local use.
diff --git a/SETUP.md b/SETUP.md
index fe6a177..8f23d14 100644
--- a/SETUP.md
+++ b/SETUP.md
@@ -168,6 +168,7 @@ curl -X DELETE http://localhost:8080/api/v1/jobs/<job-id>
| Symptom | Fix |
|---|---|
| `curl: connection refused` on :8080 | Container not up. `docker compose up -d`, then `docker ps`. |
+| `port is already allocated` / something else already uses 8080 | Set `SCRAPER_PORT=8081` in `.env`, then `docker compose up -d`. Compose publishes the scraper on that host port and `scripts/scrape.sh` and `scripts/scrape.py` read the same file, so nothing else needs editing. The port inside the container stays 8080 (do not change the right-hand side of the mapping). If your `.env` still has a `SCRAPER_BASE_URL` line, delete it or update its port: it overrides `SCRAPER_PORT`. The slash commands and the skill show `localhost:8080` in their examples, so tell Claude the port you picked. |
| `422 missing max time` | Job body needs `max_time` (in **seconds**, e.g. `300`). |
| `422 missing geo coordinates` | Job body needs `lat` and `lon` as **strings**, e.g. `"30.2672"`. |
| Job stuck `working` forever | Lower `depth`, or the IP is being throttled by Google — wait, or add proxies (see skill). |
diff --git a/docker-compose.yml b/docker-compose.yml
index 0483727..8bd1e06 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,5 +1,5 @@
# Runs the open-source Google Maps scraper (gosom/google-maps-scraper) locally.
-# Web/API mode on http://localhost:8080. The image has NO built-in auth, so we bind
+# Web/API mode on http://localhost:8080 (host port: SCRAPER_PORT in .env, default 8080). The image has NO built-in auth, so we bind
# to 127.0.0.1 ONLY — that localhost binding is the security boundary. Do not change
# the port mapping to 0.0.0.0 / a public interface without putting an auth proxy in front.
@@ -17,7 +17,8 @@ services:
restart: unless-stopped
command: ["-web", "-data-folder", "/gmapsdata"]
ports:
- - "127.0.0.1:8080:8080" # localhost only — see warning above
+ # Host port comes from SCRAPER_PORT in .env. The container side stays 8080.
+ - "127.0.0.1:${SCRAPER_PORT:-8080}:8080" # localhost only — see warning above
volumes:
- gmaps_data:/gmapsdata # job results persist here
- gmaps_cache:/opt # Playwright browser cache (faster restarts)
diff --git a/scripts/scrape.py b/scripts/scrape.py
index 97652b7..a3219aa 100755
--- a/scripts/scrape.py
+++ b/scripts/scrape.py
@@ -18,7 +18,25 @@ Notes:
import argparse, csv, io, json, os, re, sys, time, urllib.request, urllib.parse, urllib.error
from concurrent.futures import ThreadPoolExecutor
-BASE = os.environ.get("SCRAPER_BASE_URL", "http://localhost:8080")
+ENV_KEYS = ("SCRAPER_BASE_URL", "SCRAPER_PORT", "SCRAPER_API_KEY")
+
+
+def load_env_file():
+ """Read SCRAPER_* from the kit's .env, the same file docker compose reads, so a remapped
+ port needs one edit. Variables already set in the environment win over the file."""
+ path = os.path.join(os.path.dirname(os.path.abspath(__file__)), os.pardir, ".env")
+ try:
+ with open(path) as f:
+ for line in f:
+ k, sep, v = line.strip().partition("=")
+ if sep and k in ENV_KEYS and k not in os.environ:
+ os.environ[k] = v.strip().strip("\"'")
+ except OSError:
+ pass
+
+
+load_env_file()
+BASE = os.environ.get("SCRAPER_BASE_URL") or f"http://localhost:{os.environ.get('SCRAPER_PORT') or '8080'}"
KEY = os.environ.get("SCRAPER_API_KEY", "")
# Money-useful LEAD fields only — what you actually use to contact/qualify a lead.
# Everything else (geo coordinates, IDs, hours, images, reviews blobs…) is dropped by default.
diff --git a/scripts/scrape.sh b/scripts/scrape.sh
index aef4ffa..5be0f5c 100755
--- a/scripts/scrape.sh
+++ b/scripts/scrape.sh
@@ -4,7 +4,18 @@
# Example: ./scripts/scrape.sh "coffee shops in Austin TX" 30.2672 -97.7431 5
set -euo pipefail
-BASE="${SCRAPER_BASE_URL:-http://localhost:8080}"
+# Read SCRAPER_* from the kit's .env, the same file docker compose reads, so a remapped
+# port needs one edit. Variables already exported in the shell win over the file.
+ENV_FILE="$(cd "$(dirname "$0")/.." && pwd)/.env"
+if [ -f "$ENV_FILE" ]; then
+ while IFS='=' read -r k v || [ -n "$k" ]; do
+ case "$k" in SCRAPER_BASE_URL|SCRAPER_PORT|SCRAPER_API_KEY) ;; *) continue ;; esac
+ v="${v%$'\r'}"; v="${v#\"}"; v="${v%\"}"; v="${v#\'}"; v="${v%\'}"
+ [ -n "${!k+x}" ] || export "$k=$v"
+ done < "$ENV_FILE"
+fi
+
+BASE="${SCRAPER_BASE_URL:-http://localhost:${SCRAPER_PORT:-8080}}"
KEY="${SCRAPER_API_KEY:-}"
# NOTE: empty-array expansion needs the ${a[@]+...} guard — plain "${AUTH[@]}" is an
# unbound-variable error under 'set -u' on bash 3.2 (stock macOS), which is the