Skip to content

Autonomous OODA-loop pentesting agent. Hermes brain, Kali + Playwright tools over MCP, ATT&CK as the Orient language.

feat(kali): add dns_recon (DNSDumpster) and webtech_scan (whatweb+webanalyze+wpscan) tools

G0dsp6a72b14c2de0
diff --git a/.env.example b/.env.example
index 1c5d967..38f7b3a 100644
--- a/.env.example
+++ b/.env.example
@@ -123,3 +123,15 @@ PROWLER_STATUS=
PROWLER_REGIONS=
PROWLER_SERVICES=
PROWLER_TIMEOUT=
+
+# --- Recon / fingerprint APIs (OPTIONAL) --------------------------------------
+# DNSDumpster API key for the `dns_recon` tool (DNS + attack-surface recon: subdomains,
+# IPs, ASN/PTR and HTTP/HTTPS banners for a domain). Get a free key from your account at
+# https://dnsdumpster.com/ (free tier: 50 records/domain; Plus raises limits + pagination).
+# Without it, `dns_recon` fails fail-closed and reports the missing key.
+DNSDUMPSTER_API_KEY=
+
+# WPScan API token for the `webtech_scan` tool. OPTIONAL: wpscan enumerates WordPress
+# (core/plugins/theme versions) WITHOUT a token; the token only adds known-vulnerability
+# (CVE) data to the results. Get one at https://wpscan.com/profile (free tier: 25 req/day).
+WPSCAN_API_TOKEN=
diff --git a/docker-compose.yml b/docker-compose.yml
index 194137e..f8f5469 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -51,6 +51,12 @@ services:
PROWLER_REGIONS: "${PROWLER_REGIONS-${AWS_REGION:-us-east-1}}"
PROWLER_SERVICES: "${PROWLER_SERVICES-iam s3}"
PROWLER_TIMEOUT: "${PROWLER_TIMEOUT:-1800}"
+ # --- Recon / fingerprint APIs (optional) ---
+ # DNSDumpster API key for the `dns_recon` tool; WPScan token for `webtech_scan`
+ # (optional: enables CVE data). Both empty = those tools run without them
+ # (dns_recon fails fail-closed without a key; webtech_scan still enumerates).
+ DNSDUMPSTER_API_KEY: "${DNSDUMPSTER_API_KEY:-}"
+ WPSCAN_API_TOKEN: "${WPSCAN_API_TOKEN:-}"
# --- Scope-guard (OPT-IN, off by default) ---
# The scope-guard validates the DESTINATION of kali_run/kali_exec BEFORE executing,
# deterministically (code, not prompt). It activates ONLY if a scope file exists at
diff --git a/docker/kali/Dockerfile b/docker/kali/Dockerfile
index fc74e8a..c8eadf6 100644
--- a/docker/kali/Dockerfile
+++ b/docker/kali/Dockerfile
@@ -75,8 +75,39 @@ RUN apt-get update \
sqlmap \
gobuster \
impacket-scripts \
+ wpscan \
&& rm -rf /var/lib/apt/lists/*
+# webanalyze: motor de fingerprints de Wappalyzer (port en Go; el CLI oficial de Wappalyzer
+# está descontinuado). Se usa junto a whatweb + wpscan en la tool `webtech_scan` para cruzar
+# tecnologías/versiones. No está en apt: se baja el binario de release y sus fingerprints
+# (`technologies.json`) a una ruta FIJA (`/opt/webanalyze`), que `webtech_scan` pasa con
+# `-apps`. Versión fijada; checksum verificado contra el `checksums.txt` de la release.
+ARG WEBANALYZE_VERSION=v0.4.6
+RUN set -eux; \
+ arch="$(dpkg --print-architecture)"; \
+ case "$arch" in \
+ amd64) wa_arch="x86_64" ;; \
+ arm64) wa_arch="arm64" ;; \
+ *) echo "[kali-mcp] webanalyze: arch $arch no soportada; se reportará ausente." && exit 0 ;; \
+ esac; \
+ base="https://github.com/rverton/webanalyze/releases/download/${WEBANALYZE_VERSION}"; \
+ ver="${WEBANALYZE_VERSION#v}"; \
+ tarball="webanalyze_Linux_${wa_arch}.tar.gz"; \
+ cd /tmp; \
+ curl -fsSL -o "$tarball" "${base}/${tarball}"; \
+ curl -fsSL -o checksums.txt "${base}/webanalyze_${ver}_checksums.txt"; \
+ grep " ${tarball}\$" checksums.txt | sha256sum -c -; \
+ tar -xzf "$tarball" webanalyze; \
+ install -m 0755 webanalyze /usr/local/bin/webanalyze; \
+ mkdir -p /opt/webanalyze; \
+ ( cd /opt/webanalyze && webanalyze -update ); \
+ test -f /opt/webanalyze/technologies.json; \
+ rm -f /tmp/"$tarball" /tmp/checksums.txt /tmp/webanalyze
+
+# Ruta fija de los fingerprints de webanalyze que lee `webtech_scan` (tool `webtech_scan`).
+ENV WEBANALYZE_APPS=/opt/webanalyze/technologies.json
+
WORKDIR /opt/kali-mcp
# Virtualenv dedicado con Python 3.13 (compatible con Prowler). Ponerlo primero en
@@ -138,6 +169,15 @@ COPY docker/kali/aws_iam_simulate.py ./aws_iam_simulate.py
# Runner de SHELL CRUDA (tool `kali_exec`): ejecuta un comando arbitrario de la Kali vía
# `sh -c` reutilizando tool_streaming (STDLIB PURA). Lo importa kali_mcp_server.
COPY docker/kali/shell_exec.py ./shell_exec.py
+# Cliente de la API de DNSDumpster (tool `dns_recon`): recon DNS + superficie de ataque de
+# un dominio (hosts/IPs/ASN/PTR/banners) con rate limit y paginación. STDLIB PURA (urllib).
+# Lo importa kali_mcp_server (import dnsdumpster_client); sin este COPY el servidor aborta
+# al arrancar con ModuleNotFoundError.
+COPY docker/kali/dnsdumpster_client.py ./dnsdumpster_client.py
+# Fingerprint web por cruce whatweb+webanalyze+wpscan (tool `webtech_scan`): fusiona las
+# versiones que ve cada herramienta y marca conflictos. STDLIB PURA. Lo importa
+# kali_mcp_server (import webtech_scan as webtech); DEBE estar en la imagen.
+COPY docker/kali/webtech_scan.py ./webtech_scan.py
# Store de hallazgos por engagement (tools `record_finding`/`read_findings`): append
# estructurado a un JSONL sobre el plano de artefactos (STDLIB PURA, reutiliza
# artifact_store para confinar la ruta). Lo importa kali_mcp_server.
diff --git a/docker/kali/dnsdumpster_client.py b/docker/kali/dnsdumpster_client.py
new file mode 100644
index 0000000..b688469
--- /dev/null
+++ b/docker/kali/dnsdumpster_client.py
@@ -0,0 +1,369 @@
+"""Cliente de la API de DNSDumpster (STDLIB PURA) — recon DNS + superficie de ataque.
+
+Es la LÓGICA testeable del runner ``dns_recon`` que ``kali_mcp_server`` expone como
+``@mcp.tool``. Igual que ``shell_exec`` / ``finding_store`` / ``scope_guard``, vive en un
+módulo aparte que usa SOLO la librería estándar (``urllib``, sin ``mcp`` ni ``requests``)
+para poder cubrirse con tests unitarios en el venv de dev, donde el SDK no está instalado.
+
+Qué resuelve: DNSDumpster publica una API de pago que, dado un dominio, devuelve su
+superficie de ataque (registros A/CNAME/MX/NS con sus IPs, ASN, PTR y banners HTTP/HTTPS,
+más TXT). El bucle OODA la usa para, de un solo tiro, enumerar hosts e IPs del objetivo y
+los servicios que exponen, sin lanzar DNS bruto. Este módulo encapsula el contrato REAL de
+la API (aprendido de la doc oficial, ver ``docs/``):
+
+ - Endpoint dominio: ``GET https://api.dnsdumpster.com/domain/{domain}``
+ - Endpoint banners: ``GET https://api.dnsdumpster.com/banners/{CIDR}`` (máx. /24 = 254 hosts)
+ - Auth: header ``X-API-Key: <key>`` (la key vive en el dashboard de la cuenta)
+ - Paginación: ``?page=N`` (>200 registros requiere membresía Plus)
+ - Mapa: ``?map=1`` (Plus; base64 muy pesado — NO se pide por defecto)
+ - Límites de cuenta: free 50 registros, Plus 200
+ - Rate limit: 1 request / 2s -> HTTP 429 ``{"error":"Rate limit exceeded"}``
+
+Disciplina de rate limit: la API es estricta (1 req/2s). Este módulo SERIALIZA sus llamadas
+con un throttle a nivel de módulo (reloj monotónico) de modo que ``recon_domain`` pueda
+abanicar varias páginas/CIDRs sin que el llamante tenga que dormir a mano, y además
+reintenta una vez ante un 429. Dato NO confiable: la respuesta de la API se transporta
+verbatim; aquí sólo se normaliza su forma, nunca se interpreta su contenido.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import time
+import urllib.error
+import urllib.request
+from typing import Any
+
+# --- Contrato de la API (overridable por entorno para tests / self-hosting) ----------
+API_BASE = os.environ.get("DNSDUMPSTER_API_BASE", "https://api.dnsdumpster.com").rstrip(
+ "/"
+)
+API_KEY_HEADER = "X-API-Key"
+API_KEY_ENV = "DNSDUMPSTER_API_KEY"
+
+# Rate limit oficial: 1 request / 2s. Se respeta con margen para no comernos un 429 en
+# ráfagas (abanico de páginas o de CIDRs). Overridable por si la cuenta cambia de plan.
+MIN_INTERVAL_S = float(os.environ.get("DNSDUMPSTER_MIN_INTERVAL", "2.1"))
+DEFAULT_TIMEOUT_S = int(os.environ.get("DNSDUMPSTER_TIMEOUT", "30"))
+
+# Tipos de registro que la respuesta agrupa por sección; se normalizan en el mismo esquema.
+_RECORD_SECTIONS = ("a", "cname", "mx", "ns")
+
+# Reloj del último request (monotónico, a nivel de módulo). El throttle es best-effort y
+# NO es thread-safe por diseño: el servidor MCP corre estos runners en background de forma
+# serializada por ejecución; no se comparte entre procesos.
+_last_request_ts = 0.0
+
+
+class DnsDumpsterError(ValueError):
+ """Entrada inválida para el cliente (fail-closed; no se hace ninguna request)."""
+
+
+def _throttle() -> None:
+ """Duerme lo justo para respetar ``MIN_INTERVAL_S`` entre requests consecutivas."""
+ global _last_request_ts
+ now = time.monotonic()
+ wait = MIN_INTERVAL_S - (now - _last_request_ts)
+ if wait > 0:
+ time.sleep(wait)
+ _last_request_ts = time.monotonic()
+
+
+def _validate_domain(domain: Any) -> str:
+ """Valida que ``domain`` sea un string no vacío y sin esquema/espacios.
+
+ NO resuelve ni comprueba que exista: sólo rechaza fail-closed lo que no sirve como
+ path de la API (no-string, vacío, con ``/`` o espacios) para no construir una URL rota.
+ """
+ if not isinstance(domain, str) or not domain.strip():
+ raise DnsDumpsterError("domain inválido: se espera un dominio no vacío.")
+ d = domain.strip().lower()
+ if d.startswith(("http://", "https://")):
+ d = d.split("://", 1)[1]
+ d = d.strip("/")
+ if "/" in d or " " in d or not d:
+ raise DnsDumpsterError(f"domain inválido: {domain!r}")
+ return d
+
+
+def _resolve_key(api_key: str | None) -> str:
+ """Devuelve la API key explícita o la del entorno, o lanza fail-closed."""
+ key = (api_key or os.environ.get(API_KEY_ENV, "")).strip()
+ if not key:
+ raise DnsDumpsterError(
+ f"Falta la API key de DNSDumpster: pásala o exporta {API_KEY_ENV}."
+ )
+ return key
+
+
+def _get(path: str, api_key: str, timeout: int) -> dict[str, Any]:
+ """GET autenticado a ``API_BASE + path`` con throttle y manejo de errores HTTP.
+
+ Returns un dict de estado uniforme (nunca lanza por un error HTTP; fail-closed con
+ ``status='failed'``):
+ - ``{'status':'ok','data':<json>}``
+ - ``{'status':'rate_limited','error':...}`` (HTTP 429)
+ - ``{'status':'unauthorized','error':...}`` (HTTP 401/403 — key inválida/plan)
+ - ``{'status':'failed','error':...,'http_status':n}`` (otros)
+ """
+ url = f"{API_BASE}{path}"
+ req = urllib.request.Request(url, headers={API_KEY_HEADER: api_key}, method="GET")
+ _throttle()
+ try:
+ # nosec B310: el esquema es https FIJO (API_BASE) y el dominio se valida
+ # (sin esquema propio ni '/'), así que no hay apertura de file:// ni esquemas raros.
+ with urllib.request.urlopen(
+ req, timeout=timeout
+ ) as resp: # noqa: S310 - https fijo
+ raw = resp.read().decode("utf-8", errors="replace")
+ return {"status": "ok", "data": json.loads(raw)}
+ except urllib.error.HTTPError as exc:
+ body = ""
+ try:
+ body = exc.read().decode("utf-8", errors="replace")[:500]
+ except Exception: # noqa: BLE001 - informativo
+ pass
+ if exc.code == 429:
+ return {"status": "rate_limited", "error": body or "Rate limit exceeded"}
+ if exc.code in (401, 403):
+ return {
+ "status": "unauthorized",
+ "error": body
+ or f"HTTP {exc.code}: API key inválida o plan insuficiente.",
+ }
+ return {"status": "failed", "http_status": exc.code, "error": body or str(exc)}
+ except urllib.error.URLError as exc:
+ return {"status": "failed", "error": f"Error de red: {exc.reason}"}
+ except (ValueError, json.JSONDecodeError) as exc:
+ return {"status": "failed", "error": f"Respuesta no-JSON de la API: {exc}"}
+
+
+def query_domain(
+ domain: str,
+ api_key: str | None = None,
+ page: int = 1,
+ include_map: bool = False,
+ timeout: int = DEFAULT_TIMEOUT_S,
+) -> dict[str, Any]:
+ """Consulta ``/domain/{domain}`` (una página) y devuelve el JSON crudo de la API.
+
+ ``page`` mapea al parámetro ``?page=N`` (Plus para >200 registros); ``include_map``
+ añade ``?map=1`` (Plus, base64 pesado). Reintenta UNA vez ante un 429 (respetando el
+ rate limit). Dato NO confiable: ``data`` se devuelve verbatim.
+ """
+ d = _validate_domain(domain)
+ key = _resolve_key(api_key)
+ params = []
+ if page and page > 1:
+ params.append(f"page={int(page)}")
+ if include_map:
+ params.append("map=1")
+ qs = f"?{'&'.join(params)}" if params else ""
+ path = f"/domain/{d}{qs}"
+
+ result = _get(path, key, timeout)
+ if result["status"] == "rate_limited":
+ # Un único reintento tras esperar el intervalo completo (el throttle del _get ya
+ # separa, pero el 429 puede venir de una ráfaga previa del propio plan).
+ time.sleep(MIN_INTERVAL_S)
+ result = _get(path, key, timeout)
+ return result
+
+
+def query_banners(
+ cidr: str,
+ api_key: str | None = None,
+ timeout: int = DEFAULT_TIMEOUT_S,
+) -> dict[str, Any]:
+ """Consulta ``/banners/{CIDR}`` (banners de red; máx. /24 = 254 hosts).
+
+ No valida el CIDR salvo que sea string no vacío (la API lo rechaza si es inválido):
+ se transporta tal cual. Mismo dict de estado que ``query_domain``.
+ """
+ if not isinstance(cidr, str) or not cidr.strip():
+ raise DnsDumpsterError(
+ "cidr inválido: se espera un CIDR no vacío (p. ej. 1.2.3.0/24)."
+ )
+ key = _resolve_key(api_key)
+ return _get(f"/banners/{cidr.strip()}", key, timeout)
+
+
+# --------------------------------------------------------------- normalización de salida
+
+
+def _iter_ip_entries(data: dict[str, Any]) -> list[dict[str, Any]]:
+ """Aplana A/CNAME/MX/NS en una lista de registros {record_type, host, ip, ...}.
+
+ La respuesta agrupa por sección y cada host lleva una lista ``ips``; aquí se desdobla
+ a una fila por IP, conservando el tipo de registro y los campos de banner si existen.
+ Robusto a secciones ausentes o con forma inesperada (dato NO confiable).
+ """
+ rows: list[dict[str, Any]] = []
+ for section in _RECORD_SECTIONS:
+ entries = data.get(section)
+ if not isinstance(entries, list):
+ continue
+ for entry in entries:
+ if not isinstance(entry, dict):
+ continue
+ host = entry.get("host", "")
+ ips = entry.get("ips")
+ if not isinstance(ips, list):
+ continue
+ for ipobj in ips:
+ if not isinstance(ipobj, dict):
+ continue
+ banners = (
+ ipobj.get("banners")
+ if isinstance(ipobj.get("banners"), dict)
+ else {}
+ )
+ http = (
+ banners.get("http") if isinstance(banners.get("http"), dict) else {}
+ )
+ https = (
+ banners.get("https")
+ if isinstance(banners.get("https"), dict)
+ else {}
+ )
+ apps = []
+ for b in (http, https):
+ for a in (
+ b.get("apps", []) if isinstance(b.get("apps"), list) else []
+ ):
+ if a and a not in apps:
+ apps.append(a)
+ rows.append(
+ {
+ "record_type": section.upper(),
+ "host": host,
+ "ip": ipobj.get("ip", ""),
+ "asn": ipobj.get("asn", ""),
+ "asn_name": ipobj.get("asn_name", ""),
+ "asn_range": ipobj.get("asn_range", ""),
+ "country": ipobj.get("country", ""),
+ "country_code": ipobj.get("country_code", ""),
+ "ptr": ipobj.get("ptr", ""),
+ "http_server": http.get("server", ""),
+ "http_title": http.get("title", ""),
+ "https_cn": https.get("cn", ""),
+ "https_server": https.get("server", ""),
+ "apps": apps,
+ }
+ )
+ return rows
+
+
+def extract_surface(data: dict[str, Any]) -> dict[str, Any]:
+ """Normaliza el JSON de ``/domain`` en una superficie de ataque consumible.
+
+ Devuelve ``{hosts:[...], ips:[...], cidrs:[...], txt:[...], counts:{...}}`` donde:
+ - ``hosts`` : una fila por (host, IP) con ASN/PTR/banners (``_iter_ip_entries``).
+ - ``ips`` : lista DEDUPLICADA de IPs vistas (para abanicar escaneos aguas abajo).
+ - ``cidrs`` : lista DEDUPLICADA de ``asn_range`` (para el endpoint de banners).
+ - ``txt`` : registros TXT verbatim (SPF/verificaciones).
+ - ``counts`` : conteos rápidos (hosts, ips, por tipo de registro).
+ """
+ if not isinstance(data, dict):
+ return {"hosts": [], "ips": [], "cidrs": [], "txt": [], "counts": {}}
+ hosts = _iter_ip_entries(data)
+ ips: list[str] = []
+ cidrs: list[str] = []
+ for row in hosts:
+ if row["ip"] and row["ip"] not in ips:
+ ips.append(row["ip"])
+ if row["asn_range"] and row["asn_range"] not in cidrs:
+ cidrs.append(row["asn_range"])
+ txt = (
+ [t for t in data.get("txt", []) if isinstance(t, str)]
+ if isinstance(data.get("txt"), list)
+ else []
+ )
+ by_type: dict[str, int] = {}
+ for row in hosts:
+ by_type[row["record_type"]] = by_type.get(row["record_type"], 0) + 1
+ return {
+ "hosts": hosts,
+ "ips": ips,
+ "cidrs": cidrs,
+ "txt": txt,
+ "counts": {"hosts": len(hosts), "ips": len(ips), "by_record_type": by_type},
+ }
+
+
+def recon_domain(
+ domain: str,
+ api_key: str | None = None,
+ max_pages: int = 1,
+ with_banners: bool = False,
+ timeout: int = DEFAULT_TIMEOUT_S,
+) -> dict[str, Any]:
+ """Recon de alto nivel: pagina ``/domain``, normaliza y (opcional) abanica banners.
+
+ Flujo:
+ 1) Consulta ``/domain`` hasta ``max_pages`` (parando cuando una página no aporta
+ hosts nuevos; respeta el rate limit vía el throttle de ``_get``).
+ 2) Normaliza con ``extract_surface`` (hosts, IPs únicas, CIDRs, TXT).
+ 3) Si ``with_banners`` y la cuenta lo permite, consulta ``/banners/{CIDR}`` por cada
+ CIDR único hallado y adjunta la salida cruda (dato NO confiable, verbatim).
+
+ Returns ``{'status':'done', 'domain':..., 'surface':..., 'pages':n, 'banners':[...]}``
+ o un dict de error de ``query_domain`` (``unauthorized``/``rate_limited``/``failed``)
+ si la PRIMERA página falla. Las páginas/banners posteriores que fallen se anotan en
+ ``warnings`` sin abortar (lo ya recolectado sigue siendo útil para el bucle OODA).
+ """
+ d = _validate_domain(domain)
+ key = _resolve_key(api_key)
+
+ merged: dict[str, Any] = {}
+ warnings: list[str] = []
+ pages_read = 0
+ prev_host_count = -1
+
+ for page in range(1, max(1, int(max_pages)) + 1):
+ res = query_domain(d, key, page=page, timeout=timeout)
+ if res["status"] != "ok":
+ if page == 1:
+ return res # el primer fallo es fatal: no hay nada que devolver
+ warnings.append(f"page {page}: {res['status']} - {res.get('error', '')}")
+ break
+ data = res["data"]
+ pages_read += 1
+ if page == 1:
+ merged = data
+ else:
+ # Concatena las secciones de registro de las páginas siguientes.
+ for section in _RECORD_SECTIONS:
+ if isinstance(data.get(section), list):
+ merged.setdefault(section, [])
+ merged[section].extend(data[section])
+ surface_so_far = extract_surface(merged)
+ if surface_so_far["counts"]["hosts"] == prev_host_count:
+ break # la página no aportó hosts nuevos: no seguimos paginando
+ prev_host_count = surface_so_far["counts"]["hosts"]
+
+ surface = extract_surface(merged)
+
+ banners: list[dict[str, Any]] = []
+ if with_banners:
+ for cidr in surface["cidrs"]:
+ bres = query_banners(cidr, key, timeout=timeout)
+ if bres["status"] == "ok":
+ banners.append({"cidr": cidr, "data": bres["data"]})
+ else:
+ warnings.append(
+ f"banners {cidr}: {bres['status']} - {bres.get('error', '')}"
+ )
+
+ out: dict[str, Any] = {
+ "status": "done",
+ "domain": d,
+ "pages": pages_read,
+ "surface": surface,
+ }
+ if with_banners:
+ out["banners"] = banners
+ if warnings:
+ out["warnings"] = warnings
+ return out
diff --git a/docker/kali/kali_mcp_server.py b/docker/kali/kali_mcp_server.py
index e908044..edd8d0d 100644
--- a/docker/kali/kali_mcp_server.py
+++ b/docker/kali/kali_mcp_server.py
@@ -56,6 +56,18 @@ import aws_simulate
# no hay gate de autorización (es responsabilidad del operador).
import shell_exec
+# Cliente de la API de DNSDumpster (STDLIB PURA, sin `mcp`/`requests`): recon DNS +
+# superficie de ataque (hosts, IPs, ASN, PTR, banners) de un dominio, con rate limit
+# (1 req/2s) y paginación. La tool `dns_recon` es una capa fina sobre `recon_domain`.
+import dnsdumpster_client
+
+# Fingerprinting web por CRUCE de whatweb + webanalyze + wpscan (STDLIB PURA): fusiona las
+# versiones que ven las tres herramientas y marca conflictos. La tool `webtech_scan`
+# delega en `run_webtech`. Cada binario es opcional: si no está, se omite (no se finge).
+# Se importa con alias para que la tool `@mcp.tool def webtech_scan(...)` no sombree al
+# módulo (ambos se llamarían igual en el espacio de nombres global).
+import webtech_scan as webtech
+
# Store de hallazgos por engagement (STDLIB PURA): append estructurado a un JSONL sobre
# el plano de artefactos (``/artifacts``). Las tools `record_finding`/`read_findings` son
# una capa fina sobre ``finding_store.append``/``read``. Es el registro durable del que
@@ -159,6 +171,19 @@ KALI_RUN_TOOL_NAME = "kali_run"
RECORD_FINDING_TOOL_NAME = "record_finding"
READ_FINDINGS_STORE_TOOL_NAME = "read_findings"
+# Tool de recon DNS + superficie de ataque vía la API de DNSDumpster (BLOQUEANTE como
+# `kali_run`): dado un dominio, enumera hosts/subdominios, IPs únicas, ASN/PTR y banners
+# HTTP/HTTPS de un tiro, respetando el rate limit de la API (1 req/2s). Pensada para
+# alimentar el abanico de escaneos aguas abajo (p. ej. `webtech_scan` por cada host/IP).
+# Requiere `DNSDUMPSTER_API_KEY` en el entorno (o pasada como argumento).
+DNS_RECON_TOOL_NAME = "dns_recon"
+
+# Tool de fingerprint web por CRUCE de 3 herramientas (BLOQUEANTE): corre whatweb +
+# webanalyze + wpscan contra una URL y fusiona qué tecnologías/versiones ve cada una,
+# marcando conflictos de versión. wpscan sólo aplica a WordPress (si no lo es, lo reporta
+# sin error). Cada binario es opcional: el que no esté se omite y se anota en `skipped`.
+WEBTECH_SCAN_TOOL_NAME = "webtech_scan"
+
# El endpoint MCP se monta en la raíz para que KALI_MCP_URL=http://kali:8900
# (sin path) funcione tal cual con el cliente (streamable-HTTP del SDK oficial).
mcp = FastMCP("kali-mcp", host=HOST, port=PORT, streamable_http_path="/")
@@ -197,12 +222,23 @@ def server_info() -> dict[str, Any]:
# Disponibilidad REAL por collector (which de prowler/scout/cloudsplaining); no se
# finge en arm64: lo que no esté se reporta ausente y su collection falla claro.
collectors = collector_runner.available_collectors()
+ # Disponibilidad REAL de los binarios de recon/fingerprint (no se finge en arm64 ni en
+ # imágenes recortadas): lo que no esté se reporta ausente y su tool lo omite/avisa.
+ recon_tools = {
+ "whatweb": shutil.which(webtech.WHATWEB_BIN) is not None,
+ "webanalyze": shutil.which(webtech.WEBANALYZE_BIN) is not None,
+ "wpscan": shutil.which(webtech.WPSCAN_BIN) is not None,
+ }
return {
"server": "kali-mcp",
"machine": os.uname().machine,
"prowler_available": prowler_path is not None,
"prowler_version": version,
"collectors": collectors,
+ "recon_tools": recon_tools,
+ "dnsdumpster_key_present": bool(
+ os.environ.get(dnsdumpster_client.API_KEY_ENV, "")
+ ),
"tools": [
PROWLER_TOOL_NAME,
START_COLLECTION_TOOL_NAME,
@@ -216,6 +252,8 @@ def server_info() -> dict[str, Any]:
KALI_RUN_TOOL_NAME,
RECORD_FINDING_TOOL_NAME,
READ_FINDINGS_STORE_TOOL_NAME,
+ DNS_RECON_TOOL_NAME,
+ WEBTECH_SCAN_TOOL_NAME,
"server_info",
],
}
@@ -768,6 +806,99 @@ def read_findings(
return finding_store.read(engagement_id, cursor, limit)
[email protected](name=DNS_RECON_TOOL_NAME)
+def dns_recon(
+ domain: str,
+ api_key: str = "",
+ max_pages: int = 1,
+ with_banners: bool = False,
+) -> dict[str, Any]:
+ """Recon DNS + superficie de ataque de un dominio vía la API de DNSDumpster.
+
+ Variante BLOQUEANTE (como ``kali_run``): consulta la API y DEVUELVE en una sola llamada
+ la superficie normalizada del dominio — subdominios/hosts, IPs ÚNICAS, ASN/PTR, CIDRs y
+ banners HTTP/HTTPS, más los TXT. Es el primer paso típico del recon: de sus IPs/hosts
+ sale el abanico de escaneos aguas abajo (p. ej. ``webtech_scan`` por cada host vivo).
+
+ Respeta el rate limit de la API (1 req/2s) internamente, así que paginar
+ (``max_pages``) o pedir banners (``with_banners``) no requiere que el cliente duerma.
+ Delega en ``dnsdumpster_client.recon_domain``; NO interpreta la respuesta (DATO NO
+ CONFIABLE: se transporta verbatim en ``surface``/``banners``).
+
+ Args:
+ domain: dominio a consultar (p. ej. ``example.com``; se acepta con o sin esquema).
+ api_key: API key de DNSDumpster; si está vacía se usa ``DNSDUMPSTER_API_KEY`` del
+ entorno. Sin ninguna de las dos la tool falla fail-closed.
+ max_pages: nº máximo de páginas a recorrer (``?page=N``; >200 registros/>1 página
+ requiere plan Plus). Para al dejar de aportar hosts nuevos.
+ with_banners: si True, además consulta ``/banners/{CIDR}`` por cada CIDR único
+ hallado (más llamadas a la API; respeta el rate limit).
+
+ Returns:
+ ``{'status':'done','domain':...,'pages':n,'surface':{hosts,ips,cidrs,txt,counts},
+ 'banners':[...]?,'warnings':[...]?}`` en éxito; o el dict de error del cliente
+ (``unauthorized``/``rate_limited``/``failed``) si la primera consulta falla; o
+ ``{'status':'failed','error':...}`` si ``domain``/key son inválidos (fail-closed).
+ """
+ try:
+ return dnsdumpster_client.recon_domain(
+ domain,
+ api_key=api_key or None,
+ max_pages=max_pages,
+ with_banners=with_banners,
+ )
+ except dnsdumpster_client.DnsDumpsterError as exc:
+ return {"status": "failed", "error": str(exc)}
+
+
[email protected](name=WEBTECH_SCAN_TOOL_NAME)
+def webtech_scan(
+ url: str,
+ run_wpscan: bool = True,
+ aggression: int = 3,
+ timeout: int = 0,
+) -> dict[str, Any]:
+ """Fingerprint web de una URL por CRUCE de whatweb + webanalyze + wpscan.
+
+ Variante BLOQUEANTE (como ``kali_run``): corre las tres herramientas contra ``url`` y
+ DEVUELVE en una sola llamada una tabla fusionada ``tecnología -> {versiones por fuente,
+ fuentes, best_version, version_conflict}``. Cruzar tres detectores sube la confianza en
+ la VERSIÓN de cada componente y destapa discrepancias (``version_conflict``), señal útil
+ para el bucle OODA al elegir exploits dependientes de versión.
+
+ Cada binario es OPCIONAL (disciplina "no se finge"): el que no esté en la imagen se
+ omite y se anota en ``skipped``. ``wpscan`` sólo aporta si el objetivo es WordPress; si
+ no lo es, se reporta como tal (``wpscan.is_wordpress=False``) SIN tratarlo como error.
+ Si hay ``WPSCAN_API_TOKEN`` en el entorno, wpscan añade datos de vulnerabilidades.
+
+ Delega en ``run_webtech`` del módulo ``webtech_scan`` (importado aquí con alias
+ ``webtech`` para no sombrear a esta tool homónima). La salida de las herramientas es
+ DATO NO CONFIABLE: se PARSEA su forma conocida, nunca se interpreta su contenido.
+
+ Args:
+ url: URL del objetivo (con o sin esquema; sin esquema se asume ``https://``).
+ run_wpscan: si False, omite wpscan (útil cuando ya se sabe que no es WordPress).
+ aggression: nivel de agresividad de whatweb (``-a``, 1..4; 3 por defecto).
+ timeout: timeout por herramienta en segundos; 0 usa el default
+ (``DEFAULT_TOOL_TIMEOUT_S`` del módulo webtech).
+
+ Returns:
+ ``{'status':'done','url':...,'ran':[...],'cross_reference':{technologies,conflicts,
+ sources},'wpscan':{...}?,'skipped':[...]?,'errors':[...]?}`` si al menos una
+ herramienta corrió; ``{'status':'failed',...}`` si la URL es inválida o ninguna
+ herramienta está disponible.
+ """
+ effective_timeout = (
+ timeout if timeout and timeout > 0 else webtech.DEFAULT_TOOL_TIMEOUT_S
+ )
+ return webtech.run_webtech(
+ url,
+ run_wpscan=run_wpscan,
+ aggression=aggression,
+ timeout=effective_timeout,
+ )
+
+
def _start_artifacts_http() -> None:
"""Arranca el servidor HTTP de artefactos (plano de DATOS) en un hilo demonio.
diff --git a/docker/kali/tests/fixtures/dnsdumpster_example.json b/docker/kali/tests/fixtures/dnsdumpster_example.json
new file mode 100644
index 0000000..68f154a
--- /dev/null
+++ b/docker/kali/tests/fixtures/dnsdumpster_example.json
@@ -0,0 +1,93 @@
+{
+ "a": [
+ {
+ "host": "www.example.com",
+ "ips": [
+ {
+ "ip": "192.0.2.10",
+ "asn": "64500",
+ "asn_name": "EXAMPLE-AS",
+ "asn_range": "192.0.2.0/24",
+ "country": "United States",
+ "country_code": "US",
+ "ptr": "www.example.com",
+ "banners": {
+ "ip": "192.0.2.10",
+ "http": {"apps": ["nginx"], "server": "nginx", "title": "Example Domain"},
+ "https": {
+ "alt_n": ["example.com", "www.example.com"],
+ "apps": ["nginx", "PHP"],
+ "cn": "example.com",
+ "server": "nginx",
+ "title": "Example Domain"
+ }
+ }
+ }
+ ]
+ },
+ {
+ "host": "blog.example.com",
+ "ips": [
+ {
+ "ip": "192.0.2.11",
+ "asn": "64500",
+ "asn_name": "EXAMPLE-AS",
+ "asn_range": "192.0.2.0/24",
+ "country": "United States",
+ "country_code": "US",
+ "ptr": "",
+ "banners": {}
+ }
+ ]
+ }
+ ],
+ "cname": [],
+ "mx": [
+ {
+ "host": "mail.example.com",
+ "ips": [
+ {
+ "ip": "198.51.100.20",
+ "asn": "64501",
+ "asn_name": "MAIL-AS",
+ "asn_range": "198.51.100.0/24",
+ "country": "United States",
+ "country_code": "US",
+ "ptr": "mail.example.com"
+ }
+ ]
+ }
+ ],
+ "ns": [
+ {
+ "host": "ns1.example.net",
+ "ips": [
+ {
+ "ip": "203.0.113.30",
+ "asn": "64502",
+ "asn_name": "NS-AS",
+ "asn_range": "203.0.113.0/24",
+ "country": "United States",
+ "country_code": "US",
+ "ptr": "ns1.example.net"
+ }
+ ]
+ },
+ {
+ "host": "ns2.example.net",
+ "ips": [
+ {
+ "ip": "203.0.113.31",
+ "asn": "64502",
+ "asn_name": "NS-AS",
+ "asn_range": "203.0.113.0/24",
+ "country": "United States",
+ "country_code": "US",
+ "ptr": "ns2.example.net"
+ }
+ ]
+ }
+ ],
+ "total_a_recs": 2,
+ "txt": ["v=spf1 -all", "google-site-verification=EXAMPLEtoken"]
+}
diff --git a/docker/kali/tests/fixtures/webanalyze_example.json b/docker/kali/tests/fixtures/webanalyze_example.json
new file mode 100644
index 0000000..90b661b
--- /dev/null
+++ b/docker/kali/tests/fixtures/webanalyze_example.json
@@ -0,0 +1,36 @@
+{
+ "hostname": "https://www.example.com",
+ "matches": [
+ {
+ "app": {
+ "cats": ["1", "11"],
+ "category_names": ["CMS", "Blogs"],
+ "website": "https://wordpress.org",
+ "implies": ["PHP", "MySQL"]
+ },
+ "app_name": "WordPress",
+ "matches": [["meta generator"]],
+ "version": "6.4.8"
+ },
+ {
+ "app": {
+ "cats": ["59"],
+ "category_names": ["JavaScript libraries"],
+ "website": "https://jquery.com"
+ },
+ "app_name": "jQuery",
+ "matches": [],
+ "version": "3.6.0"
+ },
+ {
+ "app": {
+ "cats": ["22"],
+ "category_names": ["Web servers"],
+ "website": "https://nginx.org"
+ },
+ "app_name": "Nginx",
+ "matches": [],
+ "version": ""
+ }
+ ]
+}
diff --git a/docker/kali/tests/fixtures/whatweb_example.json b/docker/kali/tests/fixtures/whatweb_example.json
new file mode 100644
index 0000000..8b45812
--- /dev/null
+++ b/docker/kali/tests/fixtures/whatweb_example.json
@@ -0,0 +1,24 @@
+[
+ {
+ "target": "https://example.com",
+ "http_status": 301,
+ "request_config": {"headers": {"User-Agent": "WhatWeb/0.6.4"}},
+ "plugins": {
+ "HTTPServer": {"string": ["nginx"]},
+ "RedirectLocation": {"string": ["https://www.example.com/"]}
+ }
+ },
+ {
+ "target": "https://www.example.com/",
+ "http_status": 200,
+ "request_config": {"headers": {"User-Agent": "WhatWeb/0.6.4"}},
+ "plugins": {
+ "HTTPServer": {"string": ["nginx"]},
+ "MetaGenerator": {"string": ["WordPress 6.4.8"]},
+ "jQuery": {"version": ["3.6.0"]},
+ "PHP": {"version": ["8.1.2"]},
+ "HTML5": {},
+ "Title": {"string": ["Example Domain"]}
+ }
+ }
+]
diff --git a/docker/kali/tests/fixtures/wpscan_not_wordpress.json b/docker/kali/tests/fixtures/wpscan_not_wordpress.json
new file mode 100644
index 0000000..551dbe1
--- /dev/null
+++ b/docker/kali/tests/fixtures/wpscan_not_wordpress.json
@@ -0,0 +1,7 @@
+{
+ "db_files_updated": [],
+ "db_update_started": true,
+ "db_update_finished": true,
+ "target_url": "https://www.example.com/",
+ "scan_aborted": "The remote website is up, but does not seem to be running WordPress."
+}
diff --git a/docker/kali/tests/fixtures/wpscan_wordpress.json b/docker/kali/tests/fixtures/wpscan_wordpress.json
new file mode 100644
index 0000000..21d4105
--- /dev/null
+++ b/docker/kali/tests/fixtures/wpscan_wordpress.json
@@ -0,0 +1,34 @@
+{
+ "target_url": "https://blog.example.com/",
+ "effective_url": "https://blog.example.com/",
+ "version": {
+ "number": "6.4.8",
+ "status": "insecure",
+ "confidence": 100,
+ "interesting_entries": ["https://blog.example.com/, Match: 'WordPress 6.4.8'"]
+ },
+ "main_theme": {
+ "slug": "twentytwentyfour",
+ "style_name": "Twenty Twenty-Four",
+ "version": {"number": "1.2", "confidence": 80}
+ },
+ "plugins": {
+ "contact-form-7": {
+ "slug": "contact-form-7",
+ "version": {"number": "5.9.3", "confidence": 100}
+ },
+ "revslider": {
+ "slug": "revslider",
+ "version": {"number": "6.6.0", "confidence": 90}
+ },
+ "wordpress-seo": {
+ "slug": "wordpress-seo",
+ "version": {"number": "22.6", "confidence": 100}
+ }
+ },
+ "interesting_findings": [
+ {"url": "https://blog.example.com/xmlrpc.php", "type": "xmlrpc", "to_s": "XML-RPC seems to be enabled"},
+ {"url": "https://blog.example.com/readme.html", "type": "readme", "to_s": "WordPress readme found"}
+ ],
+ "scan_aborted": null
+}
diff --git a/docker/kali/tests/test_dnsdumpster_client.py b/docker/kali/tests/test_dnsdumpster_client.py
new file mode 100644
index 0000000..83ff712
--- /dev/null
+++ b/docker/kali/tests/test_dnsdumpster_client.py
@@ -0,0 +1,219 @@
+"""Tests stdlib-only del cliente DNSDumpster (`dnsdumpster_client.py`).
+
+Ejercitan la LÓGICA del cliente SIN tocar la red (``_get`` se sustituye por un fake o se
+monkeypatchea ``urllib.request.urlopen``). Fijan que:
+ - ``_validate_domain`` normaliza (quita esquema/barras) y rechaza fail-closed la basura;
+ - ``_resolve_key`` toma la key explícita o la del entorno y falla sin ella;
+ - ``extract_surface`` aplana A/CNAME/MX/NS a filas (host, IP, ASN, banners), deduplica
+ IPs y CIDRs y recoge TXT — comprobado contra un fixture SINTÉTICO (example.com, IPs de
+ rango documentación TEST-NET) con la forma real de la API;
+ - el manejo de estados HTTP: 200->ok, 429->rate_limited (con reintento), 401/403->
+ unauthorized, otros->failed, cuerpo no-JSON->failed;
+ - ``recon_domain`` devuelve el error de la primera página si falla, y agrega páginas.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+import dnsdumpster_client as dd # noqa: E402
+
+FIXTURES = Path(__file__).resolve().parent / "fixtures"
+
+
+def _load(name: str) -> dict:
+ return json.loads((FIXTURES / name).read_text(encoding="utf-8"))
+
+
+# --------------------------------------------------------------- validación de entrada
+
+
+ "raw,expected",
+ [
+ ("example.com", "example.com"),
+ ("https://example.com/", "example.com"),
+ ("HTTP://Example.COM", "example.com"),
+ (" example.com ", "example.com"),
+ ],
+)
+def test_validate_domain_normaliza(raw, expected):
+ assert dd._validate_domain(raw) == expected
+
+
[email protected]("bad", ["", " ", None, 123, "a b.com", "foo.com/path"])
+def test_validate_domain_rechaza(bad):
+ with pytest.raises(dd.DnsDumpsterError):
+ dd._validate_domain(bad)
+
+
+def test_resolve_key_explicita_o_entorno(monkeypatch):
+ assert dd._resolve_key("abc") == "abc"
+ monkeypatch.setenv(dd.API_KEY_ENV, "fromenv")
+ assert dd._resolve_key(None) == "fromenv"
+ monkeypatch.delenv(dd.API_KEY_ENV, raising=False)
+ with pytest.raises(dd.DnsDumpsterError):
+ dd._resolve_key(None)
+
+
+# --------------------------------------------------------------- normalización de salida
+
+
+def test_extract_surface_fixture():
+ data = _load("dnsdumpster_example.json")
+ surface = dd.extract_surface(data)
+ # El fixture trae hosts en A/MX/NS; se aplanan a una fila por IP.
+ assert surface["counts"]["hosts"] == 5
+ assert len(surface["ips"]) == len(set(surface["ips"])) # deduplicadas
+ assert all(ip for ip in surface["ips"]) # sin vacíos
+ # Debe haber registros A y NS (el fixture usa ambos).
+ types = surface["counts"]["by_record_type"]
+ assert "A" in types and "NS" in types
+ # Las filas llevan los campos normalizados esperados.
+ row = surface["hosts"][0]
+ assert set(["record_type", "host", "ip", "asn", "ptr", "apps"]).issubset(row)
+ # TXT se recoge verbatim (el fixture publica SPF + verificación).
+ assert "v=spf1 -all" in surface["txt"]
+
+
+def test_extract_surface_banners_y_apps():
+ data = {
+ "a": [
+ {
+ "host": "web.example.com",
+ "ips": [
+ {
+ "ip": "192.0.2.4",
+ "asn": "64500",
+ "asn_name": "EXAMPLE",
+ "asn_range": "192.0.2.0/24",
+ "country": "US",
+ "country_code": "US",
+ "ptr": "web.example.com",
+ "banners": {
+ "http": {
+ "apps": ["Nginx"],
+ "server": "nginx",
+ "title": "Home",
+ },
+ "https": {
+ "apps": ["Nginx", "PHP"],
+ "cn": "example.com",
+ "server": "nginx",
+ },
+ },
+ }
+ ],
+ }
+ ],
+ "txt": ["v=spf1 -all"],
+ }
+ surface = dd.extract_surface(data)
+ assert surface["ips"] == ["192.0.2.4"]
+ assert surface["cidrs"] == ["192.0.2.0/24"]
+ row = surface["hosts"][0]
+ assert row["http_server"] == "nginx"
+ assert row["https_cn"] == "example.com"
+ assert "PHP" in row["apps"] and "Nginx" in row["apps"] # apps fusionadas http+https
+
+
+def test_extract_surface_robusto_a_basura():
+ # Secciones ausentes, tipos inesperados: no revienta, devuelve vacío coherente.
+ assert dd.extract_surface({})["ips"] == []
+ assert dd.extract_surface({"a": "no-es-lista"})["hosts"] == []
+ assert dd.extract_surface(None)["hosts"] == []
+
+
+# --------------------------------------------------------------- manejo HTTP (fake urlopen)
+
+
+class _FakeResp:
+ def __init__(self, body: str):
+ self._body = body.encode("utf-8")
+
+ def read(self):
+ return self._body
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *a):
+ return False
+
+
+def test_get_ok(monkeypatch):
+ monkeypatch.setattr(dd, "_throttle", lambda: None)
+ monkeypatch.setattr(
+ dd.urllib.request, "urlopen", lambda req, timeout: _FakeResp('{"a":[]}')
+ )
+ res = dd._get("/domain/x", "k", 10)
+ assert res["status"] == "ok" and res["data"] == {"a": []}
+
+
+def _make_httperror(code: int, body: str):
+ import urllib.error
+
+ err = urllib.error.HTTPError("http://x", code, "err", {}, None)
+ err.read = lambda: body.encode("utf-8") # type: ignore[assignment]
+ return err
+
+
+def test_get_status_codes(monkeypatch):
+ monkeypatch.setattr(dd, "_throttle", lambda: None)
+
+ def raise_code(code, body=""):
+ def _urlopen(req, timeout):
+ raise _make_httperror(code, body)
+
+ return _urlopen
+
+ monkeypatch.setattr(
+ dd.urllib.request, "urlopen", raise_code(429, '{"error":"Rate limit exceeded"}')
+ )
+ assert dd._get("/domain/x", "k", 10)["status"] == "rate_limited"
+
+ monkeypatch.setattr(dd.urllib.request, "urlopen", raise_code(403))
+ assert dd._get("/domain/x", "k", 10)["status"] == "unauthorized"
+
+ monkeypatch.setattr(dd.urllib.request, "urlopen", raise_code(500, "boom"))
+ assert dd._get("/domain/x", "k", 10)["status"] == "failed"
+
+
+def test_get_cuerpo_no_json(monkeypatch):
+ monkeypatch.setattr(dd, "_throttle", lambda: None)
+ monkeypatch.setattr(
+ dd.urllib.request,
+ "urlopen",
+ lambda req, timeout: _FakeResp("<html>no json</html>"),
+ )
+ assert dd._get("/domain/x", "k", 10)["status"] == "failed"
+
+
+# --------------------------------------------------------------- recon_domain (alto nivel)
+
+
+def test_recon_domain_primer_fallo_es_fatal(monkeypatch):
+ monkeypatch.setattr(
+ dd,
+ "query_domain",
+ lambda *a, **k: {"status": "unauthorized", "error": "bad key"},
+ )
+ out = dd.recon_domain("example.com", api_key="k")
+ assert out["status"] == "unauthorized"
+
+
+def test_recon_domain_ok_normaliza(monkeypatch):
+ data = _load("dnsdumpster_example.json")
+ monkeypatch.setattr(
+ dd, "query_domain", lambda *a, **k: {"status": "ok", "data": data}
+ )
+ out = dd.recon_domain("example.com", api_key="k", max_pages=1)
+ assert out["status"] == "done"
+ assert out["domain"] == "example.com"
+ assert out["surface"]["counts"]["hosts"] == 5
diff --git a/docker/kali/tests/test_server_recon_tools.py b/docker/kali/tests/test_server_recon_tools.py
new file mode 100644
index 0000000..3609b99
--- /dev/null
+++ b/docker/kali/tests/test_server_recon_tools.py
@@ -0,0 +1,67 @@
+"""Tests de FORMA del cableado de `dns_recon` / `webtech_scan` en el servidor MCP.
+
+Restricción del entorno (igual que ``test_kali_run``): ``kali_mcp_server.py`` exige
+``mcp<2`` (FastMCP 1.x) y en el venv dev no está, así que NO se importa el servidor: se
+inspecciona su TEXTO fuente. La LÓGICA de ambas tools vive en ``dnsdumpster_client`` /
+``webtech_scan`` y se cubre en sus propios tests; aquí sólo se fija que el servidor:
+ - importa los módulos (``webtech_scan`` con ALIAS para no sombrear a la tool homónima);
+ - declara las constantes de nombre de tool y las registra en ``server_info``;
+ - expone wrappers ``@mcp.tool`` que DELEGAN en la lógica (no la reimplementan);
+ - reporta la disponibilidad REAL de binarios y la presencia de la API key.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+_SERVER_PATH = Path(__file__).resolve().parent.parent / "kali_mcp_server.py"
+
+
+def _source() -> str:
+ return _SERVER_PATH.read_text(encoding="utf-8")
+
+
+def test_imports_de_modulos():
+ source = _source()
+ assert "import dnsdumpster_client" in source
+ # El módulo webtech_scan se importa con alias para no colisionar con la tool.
+ assert "import webtech_scan as webtech" in source
+
+
+def test_constantes_de_tool():
+ source = _source()
+ assert 'DNS_RECON_TOOL_NAME = "dns_recon"' in source
+ assert 'WEBTECH_SCAN_TOOL_NAME = "webtech_scan"' in source
+
+
+def test_wrappers_registrados_y_delegan():
+ source = _source()
+ assert "@mcp.tool(name=DNS_RECON_TOOL_NAME)" in source
+ assert "def dns_recon(" in source
+ assert "dnsdumpster_client.recon_domain(" in source
+
+ assert "@mcp.tool(name=WEBTECH_SCAN_TOOL_NAME)" in source
+ assert "def webtech_scan(" in source
+ # Delega usando el ALIAS del módulo, no el nombre de la función (evita el shadowing).
+ assert "webtech.run_webtech(" in source
+
+
+def test_server_info_lista_tools_y_disponibilidad():
+ source = _source()
+ assert "DNS_RECON_TOOL_NAME," in source
+ assert "WEBTECH_SCAN_TOOL_NAME," in source
+ assert '"recon_tools":' in source
+ assert '"dnsdumpster_key_present":' in source
+ # La disponibilidad se calcula con shutil.which sobre los binarios del módulo webtech.
+ assert "shutil.which(webtech.WHATWEB_BIN)" in source
+ assert "shutil.which(webtech.WPSCAN_BIN)" in source
+
+
+def test_no_hay_shadowing_del_modulo():
+ # Dentro del cuerpo de la tool NO debe usarse `webtech_scan.` (sería la función, no el
+ # módulo): toda referencia al módulo va por el alias `webtech.`.
+ source = _source()
+ body_start = source.index("def webtech_scan(")
+ body = source[body_start:]
+ assert "webtech_scan.run_webtech" not in body
+ assert "webtech_scan.DEFAULT_TOOL_TIMEOUT_S" not in body
diff --git a/docker/kali/tests/test_webtech_scan.py b/docker/kali/tests/test_webtech_scan.py
new file mode 100644
index 0000000..d82c97f
--- /dev/null
+++ b/docker/kali/tests/test_webtech_scan.py
@@ -0,0 +1,162 @@
+"""Tests stdlib-only del fingerprinting web por cruce (`webtech_scan.py`).
+
+Ejercitan los PARSERS puros y el CRUCE contra fixtures SINTÉTICOS (example.com) con la
+forma real de cada herramienta: whatweb (``--log-json``), webanalyze (``-output json``) y
+wpscan (``--format json``, tanto "no es WordPress" como WordPress detectado). Fijan que:
+ - ``parse_whatweb`` extrae plugins+versiones del array ``--log-json``;
+ - ``parse_webanalyze`` extrae app_name/version/categorías del ``-output json``;
+ - ``parse_wpscan`` distingue "no es WordPress" (``scan_aborted``) de WP detectado
+ (core+plugins+tema con versión) y normaliza a ``techs``;
+ - ``cross_reference`` fusiona por nombre, agrega versiones por fuente, y marca
+ ``version_conflict`` cuando dos fuentes discrepan y acuerdo cuando coinciden;
+ - ``run_webtech`` falla fail-closed con URL inválida y omite (no finge) una herramienta
+ ausente.
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+
+import webtech_scan as wt # noqa: E402
+
+FIXTURES = Path(__file__).resolve().parent / "fixtures"
+
+
+def _read(name: str) -> str:
+ return (FIXTURES / name).read_text(encoding="utf-8")
+
+
+# --------------------------------------------------------------- parsers
+
+
+def test_parse_whatweb_fixture():
+ techs = wt.parse_whatweb(_read("whatweb_example.json"))
+ names = {t["name"] for t in techs}
+ assert "HTTPServer" in names and "MetaGenerator" in names
+ assert all(t["source"] == "whatweb" for t in techs)
+ # El fixture trae plugins versionados (jQuery/PHP): su versión se extrae.
+ by_name = {t["name"]: t["version"] for t in techs}
+ assert by_name.get("jQuery") == "3.6.0"
+
+
+def test_parse_whatweb_versiones():
+ raw = '[{"plugins": {"jQuery": {"version": ["3.4.1"]}, "PHP": {"version": ["8.1.2"]}, "HTML5": {}}}]'
+ techs = {t["name"]: t["version"] for t in wt.parse_whatweb(raw)}
+ assert techs["jQuery"] == "3.4.1"
+ assert techs["PHP"] == "8.1.2"
+ assert techs["HTML5"] == "" # sin versión pero presente
+
+
+def test_parse_whatweb_json_roto():
+ assert wt.parse_whatweb("no json") == []
+ assert wt.parse_whatweb("") == []
+
+
+def test_parse_webanalyze_fixture():
+ techs = wt.parse_webanalyze(_read("webanalyze_example.json"))
+ names = {t["name"] for t in techs}
+ assert "WordPress" in names
+ assert all(t["source"] == "webanalyze" for t in techs)
+ # Las categorías se conservan (WordPress -> CMS) y la versión se extrae.
+ wp = next(t for t in techs if t["name"] == "WordPress")
+ assert wp["version"] == "6.4.8"
+ assert any("CMS" in c for c in wp.get("categories", []))
+
+
+def test_parse_wpscan_no_es_wordpress_fixture():
+ res = wt.parse_wpscan(_read("wpscan_not_wordpress.json"))
+ assert res["is_wordpress"] is False
+ assert (
+ "WordPress" in res["aborted_reason"] or "does not seem" in res["aborted_reason"]
+ )
+ assert res["techs"] == []
+
+
+def test_parse_wpscan_wordpress_detectado():
+ res = wt.parse_wpscan(_read("wpscan_wordpress.json"))
+ assert res["is_wordpress"] is True
+ assert res["wp_version"] == "6.4.8"
+ assert "twentytwentyfour" in res["theme"]
+ slugs = {p["slug"] for p in res["plugins"]}
+ assert {"contact-form-7", "revslider", "wordpress-seo"}.issubset(slugs)
+ # normaliza core+plugins+tema a techs con prefijo plugin:/theme:
+ tech_names = {t["name"] for t in res["techs"]}
+ assert "WordPress" in tech_names
+ assert "plugin:contact-form-7" in tech_names
+ cf7 = next(t for t in res["techs"] if t["name"] == "plugin:contact-form-7")
+ assert cf7["version"] == "5.9.3"
+
+
+# --------------------------------------------------------------- cruce de versiones
+
+
+def test_cross_reference_acuerdo():
+ # Dos fuentes ven jQuery con la MISMA versión: sin conflicto, best = esa versión.
+ a = [{"name": "jQuery", "version": "3.4.1", "source": "whatweb"}]
+ b = [{"name": "jquery", "version": "3.4.1", "source": "webanalyze"}]
+ out = wt.cross_reference(a, b)
+ jq = next(t for t in out["technologies"] if t["name"].lower() == "jquery")
+ assert jq["version_conflict"] is False
+ assert jq["best_version"] == "3.4.1"
+ assert set(jq["sources"]) == {"whatweb", "webanalyze"}
+ assert out["conflicts"] == []
+
+
+def test_cross_reference_conflicto():
+ # Dos fuentes discrepan en la versión de jQuery: se marca conflicto y se listan ambas.
+ a = [{"name": "jQuery", "version": "3.4.1", "source": "whatweb"}]
+ b = [{"name": "jQuery", "version": "3.5.0", "source": "webanalyze"}]
+ out = wt.cross_reference(a, b)
+ jq = next(t for t in out["technologies"] if t["name"].lower() == "jquery")
+ assert jq["version_conflict"] is True
+ assert set(out["conflicts"][0]["versions"]) == {"3.4.1", "3.5.0"}
+
+
+def test_cross_reference_orden_por_num_fuentes():
+ # La tech vista por más fuentes va primero.
+ a = [{"name": "WordPress", "version": "6.4.8", "source": "whatweb"}]
+ b = [{"name": "WordPress", "version": "6.4.8", "source": "wpscan"}]
+ c = [{"name": "Nginx", "version": "1.25.3", "source": "whatweb"}]
+ out = wt.cross_reference(a, b, c)
+ assert out["technologies"][0]["name"] == "WordPress" # 2 fuentes > 1
+
+
+def test_cross_reference_tres_fuentes():
+ ww = wt.parse_whatweb(_read("whatweb_example.json"))
+ wa = wt.parse_webanalyze(_read("webanalyze_example.json"))
+ wp = wt.parse_wpscan(_read("wpscan_not_wordpress.json"))["techs"]
+ out = wt.cross_reference(ww, wa, wp)
+ assert "whatweb" in out["sources"] and "webanalyze" in out["sources"]
+ assert len(out["technologies"]) > 0
+ # whatweb y webanalyze comparten el nombre "jQuery" con la misma versión -> se fusiona
+ # en una sola entrada vista por dos fuentes, sin conflicto.
+ jq = next(t for t in out["technologies"] if t["name"].lower() == "jquery")
+ assert set(jq["sources"]) == {"whatweb", "webanalyze"}
+ assert jq["version_conflict"] is False
+ assert jq["best_version"] == "3.6.0"
+
+
+# --------------------------------------------------------------- orquestador
+
+
+def test_run_webtech_url_invalida():
+ out = wt.run_webtech("")
+ assert out["status"] == "failed"
+
+
+def test_run_webtech_sin_binarios(monkeypatch):
+ # Si ninguna herramienta está disponible, falla claro (no finge).
+ monkeypatch.setattr(wt, "_which", lambda b: None)
+ out = wt.run_webtech("https://example.com")
+ assert out["status"] == "failed"
+ assert set(out["skipped"]) >= {"whatweb", "webanalyze"}
+
+
+def test_run_webtech_normaliza_url(monkeypatch):
+ # Sin esquema -> se asume https; con ninguna tool disponible seguimos validando la URL.
+ monkeypatch.setattr(wt, "_which", lambda b: None)
+ out = wt.run_webtech("example.com")
+ assert out["url"] == "https://example.com"
diff --git a/docker/kali/webtech_scan.py b/docker/kali/webtech_scan.py
new file mode 100644
index 0000000..f1c999b
--- /dev/null
+++ b/docker/kali/webtech_scan.py
@@ -0,0 +1,464 @@
+"""Fingerprinting web por CRUCE de whatweb + webanalyze + wpscan (STDLIB PURA).
+
+Es la LÓGICA testeable del runner ``webtech_scan`` que ``kali_mcp_server`` expone como
+``@mcp.tool``. Igual que ``shell_exec`` / ``dnsdumpster_client`` / ``scope_guard``, vive en
+un módulo aparte con SOLO la librería estándar (sin ``mcp``) para cubrirse con tests
+unitarios en el venv de dev.
+
+Qué resuelve: tres herramientas ven la pila tecnológica de un sitio con sesgos distintos.
+Ninguna sola es de fiar para la VERSIÓN de un componente; cruzarlas sube la confianza y
+destapa discrepancias. Este módulo ejecuta las tres (cada una OPCIONAL: si el binario no
+está se omite y se reporta, nunca se finge) y FUSIONA sus salidas en una tabla única
+``tecnología -> {versiones por fuente, fuentes que la vieron, acuerdo/conflicto}``:
+
+ - whatweb : fingerprint amplio y rápido; versiones en ``plugins.<x>.version``.
+ - webanalyze : motor de fingerprints de Wappalyzer (port en Go; el CLI oficial de
+ Wappalyzer está descontinuado). Categorías + versión por app.
+ - wpscan : especialista WordPress. Si el objetivo NO es WordPress lo dice
+ (``scan_aborted``) y se reporta como tal (NO es un error); si lo es,
+ aporta versión del core, plugins/temas con versión y hallazgos.
+
+Cruce de versiones: por cada tecnología se recogen TODAS las versiones observadas por
+fuente. Si dos fuentes dan versiones distintas para el mismo componente se marca
+``version_conflict`` (señal para el bucle OODA de que la versión no es concluyente). La
+versión "mejor" es la más específica concordante.
+
+Dato NO confiable: la salida de las tres herramientas se transporta verbatim; aquí sólo se
+PARSEA su forma conocida, nunca se interpreta su contenido. Las funciones ``parse_*`` y
+``cross_reference`` son PURAS (string JSON -> dict) y son el núcleo cubierto por tests; el
+orquestador ``run_webtech`` ejecuta los binarios y delega en ellas.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import subprocess
+import tempfile
+from typing import Any
+
+# Timeout por herramienta (segundos). whatweb/webanalyze son rápidos; wpscan puede tardar.
+DEFAULT_TOOL_TIMEOUT_S = int(os.environ.get("WEBTECH_TOOL_TIMEOUT", "300"))
+
+# Nombres de binario (overridable por entorno para contenedores con rutas no estándar).
+WHATWEB_BIN = os.environ.get("WHATWEB_BIN", "whatweb")
+WEBANALYZE_BIN = os.environ.get("WEBANALYZE_BIN", "webanalyze")
+WPSCAN_BIN = os.environ.get("WPSCAN_BIN", "wpscan")
+
+# webanalyze lee sus fingerprints de ``technologies.json`` en el CWD por defecto. En el
+# contenedor el fichero se baja a una ruta fija (ver Dockerfile) y se expone aquí para
+# pasarlo con ``-apps``; si está vacío, webanalyze usa su búsqueda por defecto.
+WEBANALYZE_APPS = os.environ.get("WEBANALYZE_APPS", "")
+
+
+class WebTechError(ValueError):
+ """Entrada inválida para el runner (fail-closed; no se ejecuta nada)."""
+
+
+def _validate_url(url: Any) -> str:
+ """Valida que ``url`` sea http(s) no vacía, o lanza ``WebTechError``."""
+ if not isinstance(url, str) or not url.strip():
+ raise WebTechError("url inválida: se espera una URL http(s) no vacía.")
+ u = url.strip()
+ if not u.startswith(("http://", "https://")):
+ u = "https://" + u
+ return u
+
+
+# --------------------------------------------------------------- parsers (PUROS)
+
+
+def parse_whatweb(raw: str) -> list[dict[str, Any]]:
+ """Parsea el ``--log-json`` de whatweb en ``[{name, version, source='whatweb'}]``.
+
+ whatweb emite un ARRAY (una entrada por salto de URL); cada una lleva ``plugins`` como
+ dict ``nombre -> {version:[...], string:[...]}``. Se extrae nombre+versión de cada
+ plugin. Robusto a JSON vacío/erróneo (devuelve lista vacía): dato NO confiable.
+ """
+ out: list[dict[str, Any]] = []
+ seen: set[tuple[str, str]] = set()
+ try:
+ docs = json.loads(raw)
+ except (ValueError, json.JSONDecodeError):
+ return out
+ if isinstance(docs, dict):
+ docs = [docs]
+ if not isinstance(docs, list):
+ return out
+ for doc in docs:
+ plugins = doc.get("plugins") if isinstance(doc, dict) else None
+ if not isinstance(plugins, dict):
+ continue
+ for name, info in plugins.items():
+ versions = info.get("version", []) if isinstance(info, dict) else []
+ if not isinstance(versions, list):
+ versions = [versions]
+ versions = [str(v) for v in versions if v not in (None, "", [])]
+ if versions:
+ for v in versions:
+ key = (name, v)
+ if key not in seen:
+ seen.add(key)
+ out.append({"name": name, "version": v, "source": "whatweb"})
+ else:
+ key = (name, "")
+ if key not in seen:
+ seen.add(key)
+ out.append({"name": name, "version": "", "source": "whatweb"})
+ return out
+
+
+def parse_webanalyze(raw: str) -> list[dict[str, Any]]:
+ """Parsea el ``-output json`` de webanalyze en ``[{name, version, categories, source}]``.
+
+ webanalyze emite un objeto ``{hostname, matches:[{app_name, version, app:{category_names}}]}``
+ (o un array de ellos si se le pasan varios hosts). Robusto a formas inesperadas.
+ """
+ out: list[dict[str, Any]] = []
+ seen: set[tuple[str, str]] = set()
+ try:
+ data = json.loads(raw)
+ except (ValueError, json.JSONDecodeError):
+ return out
+ docs = data if isinstance(data, list) else [data]
+ for doc in docs:
+ matches = doc.get("matches") if isinstance(doc, dict) else None
+ if not isinstance(matches, list):
+ continue
+ for m in matches:
+ if not isinstance(m, dict):
+ continue
+ name = m.get("app_name") or m.get("app") or ""
+ if not isinstance(name, str) or not name:
+ continue
+ version = str(m.get("version") or "")
+ cats = []
+ app = m.get("app")
+ if isinstance(app, dict) and isinstance(app.get("category_names"), list):
+ cats = [str(c) for c in app["category_names"]]
+ key = (name, version)
+ if key not in seen:
+ seen.add(key)
+ out.append(
+ {
+ "name": name,
+ "version": version,
+ "categories": cats,
+ "source": "webanalyze",
+ }
+ )
+ return out
+
+
+def parse_wpscan(raw: str) -> dict[str, Any]:
+ """Parsea el ``--format json`` de wpscan en un resumen WordPress estructurado.
+
+ Devuelve ``{is_wordpress, aborted_reason, wp_version, theme, plugins:[...],
+ interesting_count, techs:[...]}`` donde ``techs`` normaliza core + plugins + tema a
+ ``[{name, version, source='wpscan'}]`` para el cruce. Si wpscan abortó porque el sitio
+ no es WordPress, ``is_wordpress=False`` y ``aborted_reason`` lleva el motivo (NO es un
+ error). Robusto a JSON vacío/erróneo.
+ """
+ result: dict[str, Any] = {
+ "is_wordpress": False,
+ "aborted_reason": "",
+ "wp_version": "",
+ "theme": "",
+ "plugins": [],
+ "interesting_count": 0,
+ "techs": [],
+ }
+ try:
+ data = json.loads(raw)
+ except (ValueError, json.JSONDecodeError):
+ result["aborted_reason"] = "salida de wpscan no parseable"
+ return result
+ if not isinstance(data, dict):
+ return result
+
+ aborted = data.get("scan_aborted")
+ if aborted:
+ result["aborted_reason"] = str(aborted)
+ return result
+
+ techs: list[dict[str, Any]] = []
+ version = data.get("version")
+ if isinstance(version, dict) and version.get("number"):
+ result["is_wordpress"] = True
+ result["wp_version"] = str(version["number"])
+ techs.append(
+ {"name": "WordPress", "version": result["wp_version"], "source": "wpscan"}
+ )
+
+ main_theme = data.get("main_theme")
+ if isinstance(main_theme, dict):
+ tname = main_theme.get("slug") or main_theme.get("style_name") or ""
+ tver = (
+ str(main_theme.get("version", {}).get("number", ""))
+ if isinstance(main_theme.get("version"), dict)
+ else ""
+ )
+ if tname:
+ result["is_wordpress"] = True
+ result["theme"] = f"{tname} {tver}".strip()
+ techs.append(
+ {"name": f"theme:{tname}", "version": tver, "source": "wpscan"}
+ )
+
+ plugins = data.get("plugins")
+ if isinstance(plugins, dict):
+ for slug, pinfo in plugins.items():
+ if not isinstance(pinfo, dict):
+ continue
+ result["is_wordpress"] = True
+ pver = ""
+ if isinstance(pinfo.get("version"), dict):
+ pver = str(pinfo["version"].get("number", ""))
+ result["plugins"].append({"slug": slug, "version": pver})
+ techs.append(
+ {"name": f"plugin:{slug}", "version": pver, "source": "wpscan"}
+ )
+
+ interesting = data.get("interesting_findings")
+ if isinstance(interesting, list):
+ result["interesting_count"] = len(interesting)
+ # Un target vivo con findings pero sin version/plugins detectados sigue siendo WP
+ # si wpscan no abortó: lo marcamos para no reportar un falso "no es WordPress".
+ if interesting and not result["is_wordpress"] and not result["aborted_reason"]:
+ result["is_wordpress"] = True
+
+ result["techs"] = techs
+ return result
+
+
+def _norm(name: str) -> str:
+ """Clave de fusión case-insensitive y sin espacios de borde para un nombre de tech."""
+ return name.strip().lower()
+
+
+def cross_reference(*tech_lists: list[dict[str, Any]]) -> dict[str, Any]:
+ """Fusiona varias listas ``[{name, version, source}]`` cruzando versiones por fuente.
+
+ Devuelve ``{technologies:[...], conflicts:[...], sources:[...]}`` donde cada entrada de
+ ``technologies`` es::
+
+ {name, sources:[...], versions:{source: [v,...]}, best_version, version_conflict}
+
+ ``version_conflict`` es True si dos fuentes reportan versiones NO vacías distintas para
+ el mismo componente (señal de versión no concluyente). ``best_version`` es la versión
+ concordante más específica (la más larga entre las observadas; vacío si ninguna fuente
+ dio versión). El cruce es por nombre normalizado (case-insensitive).
+ """
+ merged: dict[str, dict[str, Any]] = {}
+ all_sources: list[str] = []
+
+ for tech_list in tech_lists:
+ for tech in tech_list:
+ name = tech.get("name", "")
+ if not name:
+ continue
+ source = tech.get("source", "?")
+ version = str(tech.get("version", "") or "")
+ if source not in all_sources:
+ all_sources.append(source)
+ key = _norm(name)
+ entry = merged.setdefault(
+ key,
+ {"name": name, "sources": [], "versions": {}, "categories": []},
+ )
+ if source not in entry["sources"]:
+ entry["sources"].append(source)
+ entry["versions"].setdefault(source, [])
+ if version and version not in entry["versions"][source]:
+ entry["versions"][source].append(version)
+ cats = tech.get("categories")
+ if isinstance(cats, list):
+ for c in cats:
+ if c and c not in entry["categories"]:
+ entry["categories"].append(c)
+
+ technologies: list[dict[str, Any]] = []
+ conflicts: list[dict[str, Any]] = []
+ for entry in merged.values():
+ distinct_versions = sorted(
+ {v for versions in entry["versions"].values() for v in versions}
+ )
+ conflict = len(distinct_versions) > 1
+ best = max(distinct_versions, key=len) if distinct_versions else ""
+ tech_out = {
+ "name": entry["name"],
+ "sources": entry["sources"],
+ "versions": entry["versions"],
+ "best_version": best,
+ "version_conflict": conflict,
+ }
+ if entry["categories"]:
+ tech_out["categories"] = entry["categories"]
+ technologies.append(tech_out)
+ if conflict:
+ conflicts.append(
+ {
+ "name": entry["name"],
+ "versions": distinct_versions,
+ "by_source": entry["versions"],
+ }
+ )
+
+ # Orden estable: primero las vistas por más fuentes, luego alfabético.
+ technologies.sort(key=lambda t: (-len(t["sources"]), t["name"].lower()))
+ return {
+ "technologies": technologies,
+ "conflicts": conflicts,
+ "sources": all_sources,
+ }
+
+
+# --------------------------------------------------------------- orquestador (ejecuta)
+
+
+def _which(bin_name: str) -> str | None:
+ return shutil.which(bin_name)
+
+
+def _run(cmd: list[str], timeout: int) -> tuple[int, str, str]:
+ """Ejecuta ``cmd`` sin shell con timeout; devuelve ``(rc, stdout, stderr)``.
+
+ No lanza por returncode ni por timeout: un fallo de la herramienta es información, no
+ una excepción. Dato NO confiable: stdout/stderr verbatim.
+ """
+ try:
+ proc = subprocess.run( # noqa: S603 - binarios fijos, args construidos aquí
+ cmd, capture_output=True, text=True, timeout=timeout
+ )
+ return proc.returncode, proc.stdout or "", proc.stderr or ""
+ except subprocess.TimeoutExpired:
+ return -1, "", f"timeout tras {timeout}s"
+ except FileNotFoundError:
+ return -1, "", "binario no encontrado"
+
+
+def run_webtech(
+ url: str,
+ run_wpscan: bool = True,
+ aggression: int = 3,
+ timeout: int = DEFAULT_TOOL_TIMEOUT_S,
+) -> dict[str, Any]:
+ """Ejecuta whatweb + webanalyze + (opcional) wpscan y CRUZA sus versiones.
+
+ Cada herramienta es opcional: si su binario no está, se omite y se anota en ``skipped``
+ (disciplina "no se finge"). Flujo por herramienta: ejecutar -> capturar JSON (fichero o
+ stdout) -> parsear con la función pura correspondiente. Finalmente ``cross_reference``
+ fusiona las tres fuentes.
+
+ Returns ``{'status':'done', url, cross_reference:{...}, wpscan:{...}, ran:[...],
+ skipped:[...], errors:[...]}``. ``status`` es ``'done'`` aunque alguna herramienta
+ falle (lo recolectado por las demás sigue siendo útil); sólo es ``'failed'`` si la URL
+ es inválida o NINGUNA herramienta pudo ejecutarse.
+ """
+ try:
+ u = _validate_url(url)
+ except WebTechError as exc:
+ return {"status": "failed", "url": url, "error": str(exc)}
+ ran: list[str] = []
+ skipped: list[str] = []
+ errors: list[str] = []
+ whatweb_techs: list[dict[str, Any]] = []
+ webanalyze_techs: list[dict[str, Any]] = []
+ wpscan_summary: dict[str, Any] = {}
+
+ workdir = tempfile.mkdtemp(prefix="webtech-")
+ try:
+ # --- whatweb (--log-json a fichero) ---
+ if _which(WHATWEB_BIN):
+ out_f = os.path.join(workdir, "whatweb.json")
+ rc, _so, se = _run(
+ [
+ WHATWEB_BIN,
+ "--quiet",
+ f"--log-json={out_f}",
+ "-a",
+ str(aggression),
+ u,
+ ],
+ timeout,
+ )
+ try:
+ with open(out_f, encoding="utf-8", errors="replace") as fh:
+ whatweb_techs = parse_whatweb(fh.read())
+ ran.append("whatweb")
+ except OSError:
+ errors.append(f"whatweb: sin salida (rc={rc}) {se[:120]}")
+ else:
+ skipped.append("whatweb")
+
+ # --- webanalyze (-output json a stdout) ---
+ if _which(WEBANALYZE_BIN):
+ wa_cmd = [WEBANALYZE_BIN, "-host", u, "-output", "json", "-crawl", "0"]
+ if WEBANALYZE_APPS:
+ wa_cmd += ["-apps", WEBANALYZE_APPS]
+ rc, so, se = _run(wa_cmd, timeout)
+ webanalyze_techs = parse_webanalyze(so)
+ if webanalyze_techs or rc == 0:
+ ran.append("webanalyze")
+ else:
+ errors.append(f"webanalyze: sin matches (rc={rc}) {se[:120]}")
+ else:
+ skipped.append("webanalyze")
+
+ # --- wpscan (--format json a fichero) ---
+ if run_wpscan:
+ if _which(WPSCAN_BIN):
+ out_f = os.path.join(workdir, "wpscan.json")
+ cmd = [
+ WPSCAN_BIN,
+ "--url",
+ u,
+ "--format",
+ "json",
+ "--no-banner",
+ "--random-user-agent",
+ "-o",
+ out_f,
+ ]
+ token = os.environ.get("WPSCAN_API_TOKEN", "").strip()
+ if token:
+ cmd += ["--api-token", token]
+ rc, _so, se = _run(cmd, timeout)
+ try:
+ with open(out_f, encoding="utf-8", errors="replace") as fh:
+ wpscan_summary = parse_wpscan(fh.read())
+ ran.append("wpscan")
+ except OSError:
+ errors.append(f"wpscan: sin salida (rc={rc}) {se[:120]}")
+ else:
+ skipped.append("wpscan")
+
+ if not ran:
+ return {
+ "status": "failed",
+ "url": u,
+ "error": "ninguna herramienta de fingerprint disponible",
+ "skipped": skipped,
+ "errors": errors,
+ }
+
+ wpscan_techs = wpscan_summary.get("techs", []) if wpscan_summary else []
+ crossref = cross_reference(whatweb_techs, webanalyze_techs, wpscan_techs)
+
+ out: dict[str, Any] = {
+ "status": "done",
+ "url": u,
+ "ran": ran,
+ "cross_reference": crossref,
+ }
+ if wpscan_summary:
+ # resumen WP sin la lista ``techs`` (ya está fusionada en cross_reference)
+ out["wpscan"] = {k: v for k, v in wpscan_summary.items() if k != "techs"}
+ if skipped:
+ out["skipped"] = skipped
+ if errors:
+ out["errors"] = errors
+ return out
+ finally:
+ shutil.rmtree(workdir, ignore_errors=True)
diff --git a/docs/usage.md b/docs/usage.md
index fc97c48..cfdf8b1 100644
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -82,6 +82,48 @@ For CI scripts or plain `curl`, Hermes exposes an OpenAI-compatible API server d
(the same endpoints `vulnerator-mcp` wraps). Full reference — access modes, the runs flow,
`chat/completions`, and the request/response contract — in [`http-api.md`](http-api.md).
+# Recon & fingerprint tools
+
+The Kali MCP server (`kali-mcp`) ships two dedicated recon tools alongside the raw shell
+(`kali_run`/`kali_exec`). They are **blocking** tools: one call in, structured result out — no
+`started → poll` dance. `server_info` reports which binaries are present (`recon_tools`) and
+whether the DNSDumpster key is set (`dnsdumpster_key_present`).
+
+## `dns_recon` — DNS + attack surface (DNSDumpster API)
+
+Given a domain, enumerates its attack surface in one call: subdomains/hosts, **unique IPs**,
+ASN/PTR and HTTP/HTTPS banners, plus TXT records. It is the usual first recon step — its IPs and
+hosts feed the downstream fan-out (e.g. `webtech_scan` per live host).
+
+- **Key:** set `DNSDUMPSTER_API_KEY` in `.env` (free key at <https://dnsdumpster.com/>; the
+ installer offers to store it). Without a key the tool fails fail-closed and says so.
+- **Rate limit:** the API allows 1 request / 2 s; the client throttles internally and retries once
+ on HTTP 429, so paginating (`max_pages`) or fetching banners (`with_banners`) needs no manual
+ sleeps. Free tier returns up to 50 records/domain; pagination beyond 200 needs a Plus plan.
+- **Returns:** `{status, domain, pages, surface:{hosts, ips, cidrs, txt, counts}, banners?, warnings?}`.
+ The API response is untrusted data — it is normalized in shape, never interpreted.
+
+## `webtech_scan` — technology/version fingerprint by cross-reference
+
+Runs **whatweb + webanalyze + wpscan** against a URL and **merges** what each one sees into a single
+`technology → {versions per source, sources, best_version, version_conflict}` table. Cross-checking
+three detectors raises confidence in a component's version and flags disagreements
+(`version_conflict`) — a useful signal before choosing a version-dependent exploit.
+
+- **Optional binaries, no faking:** any tool missing from the image is skipped and listed under
+ `skipped` (the image installs all three). `wpscan` only contributes on WordPress; on a non-WP
+ site it reports `wpscan.is_wordpress=false` **as a normal result, not an error**.
+- **WPScan token (optional):** set `WPSCAN_API_TOKEN` in `.env` to add known-vulnerability (CVE)
+ data; wpscan still enumerates core/plugin/theme versions without it.
+- **Wappalyzer note:** the official Wappalyzer CLI is discontinued, so the Wappalyzer *fingerprint
+ engine* is provided via [`webanalyze`](https://github.com/rverton/webanalyze) (Go port). Its
+ fingerprint DB lives at `/opt/webanalyze/technologies.json` in the image.
+- **Cross-reference caveat:** merging is by (normalized) technology name. whatweb and webanalyze use
+ different naming vocabularies for some techs (e.g. whatweb encodes a CMS inside `MetaGenerator`'s
+ value while webanalyze names the app directly), so not every detection auto-merges between those
+ two. The cross-reference is most valuable where tools share a name (jQuery, nginx, PHP, WordPress,
+ …) and for surfacing version conflicts.
+
# Practice against a lab target (Juice Shop)
To practice the `web-pentest` skill without going out to the internet or touching real systems,
diff --git a/install.sh b/install.sh
index eabdd3f..b754d54 100755
--- a/install.sh
+++ b/install.sh
@@ -113,6 +113,36 @@ read_openrouter_key() {
export OPENROUTER_API_KEY
}
+# --- 2b. Recon / fingerprint API keys (OPTIONAL) -----------------------------
+# These are NOT required to bring the stack up. They enable two recon tools:
+# - DNSDUMPSTER_API_KEY -> the `dns_recon` tool (DNS + attack-surface recon).
+# - WPSCAN_API_TOKEN -> CVE data in the `webtech_scan` tool (wpscan works without it).
+# Resolution order per key: exported environment variable > interactive prompt (Enter to
+# skip) > empty. A non-interactive run just leaves whatever the environment provided.
+read_recon_keys() {
+ # DNSDumpster
+ if [ -z "${DNSDUMPSTER_API_KEY:-}" ] && [ -t 0 ]; then
+ echo
+ info "OPTIONAL: DNSDumpster API key for the 'dns_recon' tool (DNS + attack surface)."
+ info "Free key at https://dnsdumpster.com/ — press Enter to skip."
+ printf 'DNSDUMPSTER_API_KEY (optional): '
+ read -rs DNSDUMPSTER_API_KEY
+ echo
+ fi
+ export DNSDUMPSTER_API_KEY="${DNSDUMPSTER_API_KEY:-}"
+
+ # WPScan
+ if [ -z "${WPSCAN_API_TOKEN:-}" ] && [ -t 0 ]; then
+ echo
+ info "OPTIONAL: WPScan API token for 'webtech_scan' (adds CVE data; wpscan runs without it)."
+ info "Free token at https://wpscan.com/profile — press Enter to skip."
+ printf 'WPSCAN_API_TOKEN (optional): '
+ read -rs WPSCAN_API_TOKEN
+ echo
+ fi
+ export WPSCAN_API_TOKEN="${WPSCAN_API_TOKEN:-}"
+}
+
# --- 3. Secret generation ----------------------------------------------------
gen_secret() { openssl rand -hex 32; }
@@ -123,13 +153,18 @@ gen_secret() { openssl rand -hex 32; }
# avoiding any injection coming from the user-supplied API key.
build_env() {
local openrouter_key="$1" hermes_key="$2" mcp_token="$3"
+ # Optional recon keys (may be empty); passed through to .env as-is.
+ local dnsdumpster_key="${4:-}" wpscan_token="${5:-}"
- info "Building .env from .env.example (only the three required keys are filled in)..."
+ info "Building .env from .env.example (required keys filled in; optional recon keys if provided)..."
- awk -v ork="$openrouter_key" -v hk="$hermes_key" -v mt="$mcp_token" '
+ awk -v ork="$openrouter_key" -v hk="$hermes_key" -v mt="$mcp_token" \
+ -v ddk="$dnsdumpster_key" -v wpt="$wpscan_token" '
/^OPENROUTER_API_KEY=/ { print "OPENROUTER_API_KEY=" ork; next }
/^HERMES_API_SERVER_KEY=/ { print "HERMES_API_SERVER_KEY=" hk; next }
/^VULNERATOR_MCP_INBOUND_TOKEN=/ { print "VULNERATOR_MCP_INBOUND_TOKEN=" mt; next }
+ /^DNSDUMPSTER_API_KEY=/ { print "DNSDUMPSTER_API_KEY=" ddk; next }
+ /^WPSCAN_API_TOKEN=/ { print "WPSCAN_API_TOKEN=" wpt; next }
{ print }
' "$ENV_EXAMPLE" > "$ENV_FILE.tmp"
@@ -168,11 +203,15 @@ main() {
read_openrouter_key
[ -n "${OPENROUTER_API_KEY:-}" ] || die "OPENROUTER_API_KEY is empty. It is required in OpenRouter mode (the default)."
+ # Optional recon/fingerprint API keys (Enter to skip; empty is fine).
+ read_recon_keys
+
local hermes_key mcp_token
hermes_key="$(gen_secret)"
mcp_token="$(gen_secret)"
- build_env "$OPENROUTER_API_KEY" "$hermes_key" "$mcp_token"
+ build_env "$OPENROUTER_API_KEY" "$hermes_key" "$mcp_token" \
+ "${DNSDUMPSTER_API_KEY:-}" "${WPSCAN_API_TOKEN:-}"
if [ "$NO_UP" -eq 1 ]; then
echo
@@ -190,6 +229,8 @@ main() {
echo " - OPENROUTER_API_KEY (the key you provided)"
echo " - HERMES_API_SERVER_KEY (auto-generated, 32-byte hex)"
echo " - VULNERATOR_MCP_INBOUND_TOKEN (auto-generated, 32-byte hex)"
+ [ -n "${DNSDUMPSTER_API_KEY:-}" ] && echo " - DNSDUMPSTER_API_KEY (optional; enables dns_recon)"
+ [ -n "${WPSCAN_API_TOKEN:-}" ] && echo " - WPSCAN_API_TOKEN (optional; CVE data in webtech_scan)"
echo
info "Open the interactive TUI once the containers are healthy:"
echo " docker compose exec -it ooda-agent hermes chat"