Skip to content

Open source alternative to Semrush and Ahrefs

Site audit P0 (1/3): issue engine, incremental persistence, block detection (#362)

Ben Senescu1c74fded7b59
diff --git a/docs/site-audit-pm-research.md b/docs/site-audit-pm-research.md
new file mode 100644
index 00000000..f4107c55
--- /dev/null
+++ b/docs/site-audit-pm-research.md
@@ -0,0 +1,142 @@
+# OpenSEO Site Audit: 80/20 Feature Set & Crawl Architecture Proposal (rev. 3, founder decision applied)
+
+**Author:** PM research synthesis | **Date:** 2026-06-09 | **Inputs:** SEOnaut + LibreCrawl code reviews, OpenSEO baseline review, competitor/market research, architecture research, adversarial panel review. Revision notes: panel critiques accepted are folded in silently; where we overrode the panel, an explicit **Panel pushback** note marks the disagreement. **Rev. 3:** founder decision — build on the in-house Workers crawler, do not adopt DataForSEO OnPage; "we were blocked" flagging accepted as the blocking posture. §4 now weighs that decision's tradeoffs instead of recommending an engine.
+
+---
+
+## 1. How site audits actually work (primer)
+
+Every site audit tool is the same three-stage pipeline:
+
+1. **Crawl.** Start from a URL, fetch robots.txt and sitemaps, fetch pages, extract links, enqueue new same-site URLs (BFS), repeat until a page cap. Two engine choices matter: _plain HTTP fetch_ (fast, cheap, blind to JavaScript) vs _headless rendering_ (sees what Google sees, ~10x cost). JS rendering is table stakes in commercial tools; differentiation is fidelity and whether it's paywalled.
+2. **Check.** Per-page rules (missing title, noindex, broken canonical) plus cross-crawl rules (duplicate titles, redirect chains, orphan pages). The cross-page checks carry most of the value, and they're set-queries over crawl data, not parsing.
+3. **Report.** Severity grouping, plain-English explanations, drill-down to affected URLs, **export**, and diffs against previous crawls.
+
+**Desktop vs cloud.** Screaming Frog ($259/yr) and Sitebulb run on the user's laptop: residential IP, no bot-blocking, but no scheduling, RAM-limited crawls (the #1 SF complaint), and the machine must be on. Cloud tools (Ahrefs, Semrush, Moz) crawl datacenter IPs on a schedule — and **getting blocked is their #1 recurring complaint**: WAFs 403/challenge their bots, producing phantom 404s and false positives.
+
+**Why blocking happens.** WAFs score IP reputation (datacenter = suspicious) and fingerprint (no JS = bot). It hits every cloud auditor, including ours. The industry playbook: documented UAs, published allowlistable IPs, Cloudflare's Verified Bots program, and ownership verification to justify higher crawl rates (Ahrefs advertises dramatically higher rates for verified owners; the oft-quoted 30→30,000 URLs/min figure is unsourced in our fact base and Ahrefs' verification is its own, not GSC — treat as directional only).
+
+---
+
+## 2. Where OpenSEO's audit stands today
+
+Honest summary: **we have a crawler and a Lighthouse integration, but not an audit product.** We extract per-page fields and show filterable tables, but no rule engine names issues; the only "issues" users see are Lighthouse failures. SEOnaut ships ~88 issue types; LibreCrawl ~26; Screaming Frog ~300. We effectively ship ~2 as table styling.
+
+Structural problems, in priority order:
+
+- **The link graph is thrown away.** Link lists are extracted but only counts persist to D1, so broken links, orphans, redirect chains, and inlink analysis are impossible without schema change.
+- **Persistence is finalize-only and Workflow-shaped.** `runCrawlBatch` returns full page arrays (links, images) as durable step state (~1MiB/step cap — already borderline on link-dense sites), `allPages` accumulates everything in 128MB Worker memory, and `batchWriteResults` writes all rows inside the single finalize step (DB_BATCH_SIZE=100 → a 1M-row link graph would mean ~10,000 subrequests in one step, ~10x the limit). **Any link-graph feature requires restructuring persistence first.** This was deferred to P1 in draft 1; the panel correctly identified that as backwards — it is now the P0 prerequisite.
+- **Step budget caps crawl size — ledger, not vibes.** Workflows allows ~1,024 steps/instance. Today: 3 steps per 25-page batch → ~341 batches ≈ **~8.5k pages** fits (minus discovery/finalize overhead), so the existing 10,000-page clamp slightly exceeds the budget. The sharper constraint is Lighthouse strategy `all`: 2 steps per 10-URL batch ≈ 1,000 steps at 5k pages — over budget on its own. Politeness delays must live _inside_ steps (wall-clock waits are fine), never as smaller batches. See §4 for the post-refactor ledger and caps.
+- **No audit MCP tools** — keywords, SERP, backlinks, rank-tracker, GSC all have them; audit has zero (verified in `src/server/mcp/tools`). Ahrefs, meanwhile, **already ships site-audit MCP tools** (`site-audit-issues`, `site-audit-page-explorer`, etc.); Semrush has an MCP server too. We are behind here, not ahead — see §3.4 for what our actual wedge is.
+- **SSRF gap:** `url-policy.ts` validates only the start URL; discovered links are fetched unvalidated. Must be fixed in P0 for as long as the in-house crawler runs.
+
+Condensed check inventory: status/redirects, title/meta, canonicals, indexability, headings, images = **partial** (extracted, unflagged; redirects followed silently; header variants like `X-Robots-Tag` unchecked). Duplicates (30–35% of sites, Semrush study¹), broken links (52% of domains¹), redirect chains (~12%¹), orphans, security headers, hreflang, JS rendering, scheduling/diffs, issue UX, MCP, block resilience = **missing**. Lighthouse/CWV = **strong**, better than both OSS tools. _(¹ Prevalence figures from market research §3; primary sources to be attached as a footnoted appendix before this circulates externally.)_
+
+(Folklore correction stands: the 100k-page lifetime cap settles to actual crawled counts; only failed/stuck audits permanently pay their reservation.)
+
+---
+
+## 3. The 80/20 feature set
+
+**Pricing posture (decides everything downstream).** Audit is the most commoditized feature in SEO — Ahrefs Webmaster Tools and Screaming Frog ≤500 URLs are free. Per our SEO strategy (goal = signups), **audit is acquisition, not monetization**: default 50-page audits are free per site; credits are charged for large crawls, scheduling, and history. The free posture also reinforces the engine decision (§4): with the in-house crawler, a free audit's marginal cost is D1 writes and Workers compute largely inside quotas we already pay for — no per-audit vendor bill, no metered cost on failed runs (DataForSEO charges even for failed tasks).
+
+The prevalence evidence says a credible audit needs ~6 check families, not 300. P0 below runs on the **in-house crawler (founder decision, §4)**; crawler-side hardening is therefore mandatory P0 work, not contingency. Everything except the crawler-side items remains **engine-independent by design** (schema, SQL checks, UX, MCP) — that portability is the cheap hedge that keeps a future vendor or local-agent engine swap on the table.
+
+### P0 — the audit becomes real (~6–8 weeks)
+
+**0. Incremental persistence (prerequisite, was P1).** Write `audit_pages`/`audit_links` rows to D1 _inside each crawl-batch step_ (one extra `db.batch` per batch keeps subrequests bounded), return only slim summaries (id, url, status) as step state, drop the `allPages` accumulator. Finalize becomes: multipage SQL step → completion. Bonus: failed audits keep partial results. Also fix the SSRF gap (run discovered links through url-policy).
+
+**1. Issue engine + schema.** `audit_links` edge table (source, target, anchor, rel, internal) and `audit_issues` table. **Reporter contract (decided now, per panel):** reporters consume a normalized page record — `(record: AuditPageRecord) => boolean` where the record includes response headers captured at crawl time — _not_ SEOnaut's `(page, $, headers)` cheerio signature. Cheerio runs once in `crawlPage`; reporters are DOM-free. This is what makes the engine portable across our crawler, a future local agent, or a vendor feed if the §4 revisit triggers ever fire. Cross-page checks are D1 SQL passes (duplicates = `GROUP BY hash HAVING count > 1`, per SEOnaut's multipage reporters). **Effort: 2–3 weeks including #0's restructure** (was "1 week" — panel was right).
+
+**2. High-prevalence checks** (post-#1; thresholds cribbed from SEOnaut/LibreCrawl):
+
+- _Broken internal links + redirect chains/loops_: switch to `redirect: "manual"`, record hops, enqueue targets. Decision: redirect hops do **not** count against maxPages or billing (they're cheap HEAD-sized fetches; metering/progress semantics budgeted at +1 day). Broken-link checks fire **only for targets we actually fetched or HEAD-checked** — never inferred from absence on partial crawls. ~4 days.
+- _Duplicates_: body-hash group-by catches mirrored/parameterized duplicates only (any dynamic token defeats exact hashing); **duplicate titles/descriptions group-by is the workhorse** and carries the prevalence claim. Avoid LibreCrawl's O(n²) similarity scan. ~1–2 days.
+- _Title/meta length, missing/multiple H1, heading order_ (SEOnaut thresholds). ~1–2 days.
+- _Canonical + indexability with header fallbacks_ (`X-Robots-Tag`, `Link: rel=canonical` — SEOnaut's standout trick; headers are in the normalized record by design). ~2 days.
+- _Orphan pages_: reorder seeding to **sitemap-last** (we currently seed sitemap URLs _first_, the opposite of the SEOnaut pattern we cite, so sitemap pages eat the page budget and link discovery never exhausts), and **emit orphan findings only when the crawl completed** (queue exhausted below maxPages) — on truncated crawls "no observed inlinks" is true of nearly everything and would mass-false-positive, the exact Semrush health-score noise we mock. ~3–4 days including reordering and gating.
+- _Thin content + crawl depth_: explicit disposition — **build in P0** (word count already stored; BFS depth is one column). _Page size_: P1. ~1 day.
+
+**3. Issues UX.** Critical/Warning/Info grouping, one-paragraph plain-English explanation per type (adapt SEOnaut's YAML copy), affected-URL drill-down, LibreCrawl's ~100-glob default exclusions (wp-admin, checkout, feeds), and **CSV export** (previously unaccounted; it's one of the three core report capabilities from §1). ~4–5 days.
+
+**4. MCP audit tools — reframed.** Draft 1 claimed "a demo no competitor can match." **That was wrong: Ahrefs ships site-audit MCP tools today and Semrush has an MCP server.** Our wedge is not MCP existence; it's **credit-priced, zero-setup, agent-actionable audits**: audit any URL on demand for cents with no project setup or $129+/mo subscription, and return per-issue _fix instructions an agent can execute_ (the issue record carries remediation steps, not just labels). Tool shapes designed against Ahrefs' actual, inspectable MCP audit tools: `run_site_audit`, `get_audit_status`, `get_audit_issues(severity, type)`, `get_audit_pages(filter)`, each issue carrying `how_to_fix`. ~4 days.
+
+**5. Block detection — moved to weeks 1–2 (was week 5).** The panel is right that for our segment (indie founders on Vercel/Cloudflare/Netlify — bot protection there is a one-toggle setting and IP-reputation challenges hit datacenter egress even without it; the week-1 baseline will establish the real rate) blocking is a **first-run activation killer**, not an ops metric — and our own onboarding audit shows activation leaks are the existential problem. Classify 403s/challenge bodies/`cf-mitigated` headers as "we were blocked," say so honestly instead of recording error pages, and show owner-targeted allowlist instructions inline (GSC-verified flow, §4). This layer is response-classification, hence engine-independent, and instruments block-rate telemetry from day one. ~3 days.
+
+### P1 — weeks 8–14
+
+- **Scheduled audits + diffing**: extend the existing cron; store per-run issue counts, show deltas ("12 new broken links since last week"). The retention feature and the reason cloud beats desktop. **Data retention policy (new):** full link graphs kept for the latest 2 runs only; older runs keep issue counts/deltas (a 10k-page graph is ~100–150MB against D1's 10GB cap — weekly large audits would eat gigabytes/year otherwise). Large-audit edge lists optionally archived as compressed R2 blobs.
+- **AI-crawler accessibility audit — promoted from P2.** Which AI bots (GPTBot, ClaudeBot) can reach your content, robots/Cloudflare AI-block detection, is-key-content-in-static-HTML. The panel is right that our own words ("open greenfield, perfectly on-brand") contradicted P2 placement: it's mostly computable from data we already collect, and "is ChatGPT blocked from your site?" is the launch/marketing asset for the whole relaunch — nobody switches auditors for HSTS checks. (Skip llms.txt as a headline; no major LLM vendor consumes it.)
+- **JS rendering — deferred (consequence of the no-vendor decision, §4).** Plain fetch can't render SPAs. P1 ships the honest mitigation instead: detect client-rendered shells (near-empty body text with a large script payload / root-div-only HTML) and flag "this site renders client-side; HTML-level checks are incomplete" rather than reporting false missing-title/thin-content issues. If demand materializes, the revisit path is Cloudflare Browser Rendering as an opt-in, credit-priced add-on for GSC-verified own-site audits (where the owner can allowlist, neutralizing the fingerprint problem) — not a vendor API.
+- **Security/hygiene checks** (HTTPS, mixed content, HSTS/CSP, insecure forms) and **external broken links** (HEAD checks, per-crawl cache) — demoted below AI-crawler work.
+- **Cloudflare Verified Bots**: reframed as a **lottery ticket, not a mitigation**. Eligibility requires established, meaningful crawl volume; a new low-volume crawler routinely doesn't qualify, and Web Bot Auth is beta on Cloudflare's side. File opportunistically for our own UA (`OpenSEOBot`); plan zero dependency on it. The owner-targeted allowlist flow is the only mitigation we count on.
+
+### P2 — later
+
+- **`npx openseo-audit` local agent** (see §4 option D) — gated on block-rate telemetry, which now exists from week 1.
+- **Crawl snapshots in R2** ("what we saw" evidence; lightweight SEOnaut WACZ analog).
+- **Hreflang validation, structured-data validation, page-size checks, link-placement classification.**
+
+**Skip entirely (explicit dispositions):** 300-check parity, crawl visualizations, E-E-A-T scores, multi-language UI.
+
+---
+
+## 4. Architecture: where should the crawl run?
+
+| | **A. In-house Workers crawl + mitigations** | **B. DataForSEO OnPage offload** | **C. Desktop app** | **D. Hybrid: cloud + `npx` local agent** |
+| -------------------- | -------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------- | ----------------------------------------------------------- |
+| Blocking reliability | Low-medium: no stable IPs, no JS-challenge survival | Medium: still datacenter, but published IPs + named UA are _allowlistable_ | Highest (residential IP, real browser) | High when used; manual trigger only |
+| Scheduled audits | Yes (≤ a few k pages — step budget) | Yes | **No** | Yes (cloud path); **no for the blocked sites D exists for** |
+| JS rendering | No | Yes (priced) | Yes | Yes (local Playwright) |
+| Eng effort | Ongoing hardening forever | Low (vendor/SDK/billing integrated) | **Months** + signing/notarization/3-platform QA | ~3–5 weeks (see below) |
+| Cost per 1k pages | **Not $0**: D1 link writes ≈ $0.30/1k (with indexes) + Workers/Workflows | ~$0.125–0.50 base, ~$1.25 JS _(list prices from architecture research — re-verify if the §4 revisit triggers ever fire)_ | $0 marginal | $0 marginal + ingest |
+| Max crawl size | ~8.5k today, ~12–25k post-refactor (Lighthouse `all` binds first; instance chaining removes the ceiling) | Vendor-side, no Workflow ceiling | RAM-limited | Local machine-limited |
+| MCP / billing fit | Good / fine | Good / per-page cost maps to credits | **Bad** / awkward | Excellent for developers / fine |
+
+**Excluded options, explicitly (panel asked):** _Cloudflare Browser Rendering_ — solves JS but not blocking (egress is still Cloudflare datacenter IPs), adds per-browser-hour cost, and headless-Chrome fingerprints start their own anti-bot treadmill. With the vendor path now declined (see decision below), this becomes the only _cloud_ JS-rendering route (the P2 `npx` agent's local Playwright is the other) — scoped as a P2 opt-in for GSC-verified own-site audits where the owner can allowlist us first. _Residential/ISP proxy egress_ — rejected on principle and practice: it disguises crawler identity, which contradicts our documented-UA/allowlist posture and many proxy networks' sourcing is ethically murky; cost (~$1–8/GB) is also worst-in-class.
+
+**Decision: A — in-house Workers crawler (founder call: avoid DataForSEO vendor costs).** The original recommendation was B, contingent on a week-1 spike; the founder has decided against adding vendor spend, and "we were blocked" honest flagging is accepted as the blocking posture. That is a defensible position, but the proposal's job is to price it accurately. What the decision buys, and what it costs:
+
+**What going in-house buys:**
+
+- **No per-audit vendor bill on a free acquisition feature.** Every OnPage call is marginal cash out (~$0.006–0.025 per free 50-page audit, $1.25–5 per 10k-page crawl at list prices), and DataForSEO meters even failed tasks — a real footgun we've already hit on the SERP side. Crucially, the comparison must not double-count: D1 storage writes are common to **both** engines (vendor results would land in the same `audit_pages`/`audit_links` tables to run the same SQL checks), so the vendor fee is purely additive. In-house wins on marginal cost at every crawl size.
+- **No new vendor dependency or pricing-change exposure** on the product's centerpiece free feature; full control of crawl behavior, UA identity, and politeness.
+- **One less integration**: no task-queue/webhook polling model, no mapping a vendor's page schema into the normalized record.
+
+**What it costs — priced honestly:**
+
+1. **"Basically free" is true only until the free tier works.** D1 Paid includes 50M row-writes/month _shared with the entire product_. A 50-page free audit writes ~8–15k rows (pages + links + indexes), so **~3–5k free audits/month consumes the whole included allotment**; beyond it, $1/M rows — at 10k free audits/month, ~$50–100/month in-house vs ~$60–250/month vendor-side. A ~2x cost win, not zero, and it eats quota headroom from every other D1 feature. Required companion: a **TTL on free-tier audit data** (e.g., purge one-off free audits' link graphs after 30 days) — the P1 two-run retention policy only covers scheduled audits and doesn't protect the 10GB D1 cap from thousands of one-off free crawls.
+2. **JS rendering is off the roadmap** until/unless we build it on Cloudflare Browser Rendering. SPA-heavy sites (a real slice of indie-founder stacks: Next.js CSR pages, Vite SPAs) get degraded audits. Mitigation: detect-and-disclose (§3 P1), never silently mis-report — and count CSR-shell prevalence in the week-1 telemetry so the JS-rendering revisit trigger has a denominator too.
+3. **A step-budget ceiling — real, but it's Lighthouse that binds, not the crawl.** The ledger: today's 3 steps per 25-page batch supports ~8.5k pages in the 1,024-step budget; after the P0 #0 refactor (1–2 steps/batch), the crawl alone supports **~12–25k pages** — the 10k clamp can stand. The binding constraint is Lighthouse strategy `all` (2 steps per 10-URL batch ≈ 1,000 steps at 5k pages, over budget by itself): cap `all` at ~2k pages or switch to sampling, and document page caps _coupled to Lighthouse strategy_. Nor is the ceiling permanent: Workflow instances can chain (the P0 #0 refactor already puts frontier state in D1; a continuation instance resumes it) — roughly a week of cross-instance dedupe work, not an engine swap. The agency/large-site segment is deferred, not foreclosed.
+4. **Weaker blocking mitigation — and the allowlist fix is unverified on the exact tiers our ICP uses.** Workers egress comes from shared Cloudflare IP ranges — we cannot publish a dedicated allowlistable IP, so the GSC-verified fix flow must be **User-Agent-based** (`OpenSEOBot/1.0` via a WAF custom rule; spoofable, tolerable when the owner allowlists their _own_ site). But: on **Cloudflare Free, Bot Fight Mode supports no exceptions** — a skip-by-UA rule does not bypass it; the owner's only fix there is turning BFM off entirely, a much heavier ask than "add this one rule." Vercel's challenge bypass is plan-gated and may be unavailable on Hobby. **Week 1 therefore includes a 1–2 day validation**: crawl test zones on Cloudflare Free (BFM on), Cloudflare Pro (Super BFM), and Vercel Hobby/Pro (Attack Challenge Mode), and write per-platform fix copy that is honest where the real answer is "disable bot protection." Verified Bots filing (P1) stays a lottery ticket.
+5. **The ~1–2 weeks of crawler work inside P0 that the vendor path would have eliminated, plus permanent ownership.** Manual-redirect handling, per-host politeness inside steps, sitemap-last seeding, SSRF fix, step-budget restructuring, timeout/retry edges — already counted inside the 6–8-week P0 estimate (not additional to it), but under B most of it would have been deleted, and the hardening treadmill (new WAF behaviors, new edge cases) is ours indefinitely.
+6. **No fallback engine for blocked sites.** Under B, a site blocking us might still be crawlable via the vendor's allowlisted infrastructure. In-house, a blocked site stays blocked until the owner acts. The honest-flagging posture makes this acceptable for owned sites: "we were blocked, here's the fix" instead of pretending.
+7. **Third-party-URL audits have no remedy at all.** The MCP wedge includes "audit any URL on demand" (competitor audits, agent workflows) — but the allowlist flow only works for sites the user _owns_. For non-owned URLs, "we were blocked" is a terminal state, exactly where vendor infrastructure would have helped. Disposition: scope the claim — on-demand MCP audits are first-class for owned/verified sites, best-effort for everything else, and the tool response says which one it was.
+
+**Revisit triggers (set now, so the decision is cheap to reverse).** Define "audit blocked" first: start URL blocked, or ≥20% of fetched URLs classified as challenge/`cf-mitigated`. Then: (a) >15–20% of first-run audits blocked _and_ the fix funnel (block shown → instructions viewed → re-run unblocked, instrumented in week 1) shows owners not converting; (b) sustained demand for >10k-page crawls that chaining doesn't satisfy, or for JS rendering (CSR-shell prevalence from week-1 telemetry is the denominator); (c) DataForSEO/competitor pricing shifts. The reporter contract and schema are engine-agnostic by design, so adopting OnPage later — or the `npx` local agent (D) — is an adapter, not a rewrite.
+
+**Implementation notes for A (from §2's constraints):** politeness delays serialized _inside_ steps (wall-clock waits are fine; more steps are not), progress steps collapsed to stay under the 1,024-step budget, incremental per-batch D1 writes (P0 #0), block-rate baseline pulled from existing audit data (status 0/403 rates + PostHog) in week 1 so the revisit triggers have a denominator.
+
+One honest limitation, stated plainly: **for sites that block all datacenter crawlers, every recommended option loses scheduled audits** — D is a manual local run. The cloud-scheduling advantage evaporates for exactly that segment; our remedy there is the GSC-verified allowlist-fix flow, which restores cloud crawling rather than working around it.
+
+**Why not C (the desktop question, directly).** Desktop genuinely solves blocking — it's Screaming Frog's model. But scheduled audits die, server-side MCP can't wake a sleeping laptop, and it's a second product (signing, notarization, auto-update, months of work) whose own failure modes — RAM crashes, machine tie-up — are that model's loudest complaints. Draft 1's "~80% of the benefit for ~5% of the cost" for D was invented precision; the honest version: D recovers desktop's _crawl-origin_ benefit (residential IP, optional local Playwright) but not unattended operation, requires a **new** authenticated bulk-ingest endpoint feeding `audit_pages`/`audit_links` (no ingest path exists today; D depends on the P0 persistence refactor) plus a local crawler — the cheerio extraction logic is plain TS and ports, the Workflow orchestration doesn't. Estimate **~3–5 weeks**. **Panel pushback:** the panel suggested that if D balloons to 6–8 weeks the C-vs-D calculus changes. It doesn't: C strictly contains D's costs (local crawler, Playwright, sync) _plus_ installers/signing/3-platform QA, and still loses scheduling and MCP. Worst-case D remains cheaper than best-case C. Scope honesty: D serves the developer half of the ICP; for SEO freelancers (not terminal-native, and whose clients' sites are exactly the WAF'd ones) the realistic remedy is the GSC-verified allowlist flow — which is why that flow ranks above the CLI.
+
+**The GSC idea, honestly.** GSC verification **does not bypass any WAF** — Cloudflare neither knows nor cares about Google's ownership records. What it buys: (1) **policy cover** to raise crawl _rate_ for verified owners (industry-standard move; the Ahrefs precedent is directional, see §1) — but robots.txt respect stays the default with a per-project owner toggle, not auto-ignored: crawl traps and server load are real, and "verified" doesn't repeal politeness; (2) **the trustworthy block-fix loop** — on a detected block for a GSC-verified site, show the _owner_ the exact Cloudflare/Vercel allowlist rule for our UA (`OpenSEOBot` — UA-based since Workers egress IPs aren't publishable, see decision above). Highest-leverage mitigation available; OAuth already built; now surfaced in onboarding, not post-P0; (3) later, GSC-enriched audits (true orphans = GSC impressions, zero inlinks).
+
+---
+
+## 5. Recommended roadmap
+
+**Week 1:** `audit_pages`/`audit_links`/`audit_issues` schema + engine-agnostic reporter input contract; ship block detection + honest "we were blocked" messaging + telemetry — audit-level blocked definition, fix-funnel events (block shown → instructions viewed → re-run unblocked), CSR-shell prevalence counting, baseline from existing audit data (status 0/403 + PostHog; note raw 403s overcount — classify challenges, don't just count status codes); set the documented UA (`OpenSEOBot/1.0` with a /bot info page); **1–2 day allowlist validation** on Cloudflare Free (BFM), Cloudflare Pro (SBFM), Vercel Hobby/Pro (Attack Challenge) → per-platform fix copy.
+
+**Weeks 2–4:** incremental persistence refactor + SSRF fix; crawler hardening (manual redirects, per-host politeness inside steps, sitemap-last seeding, step collapse; page caps coupled to Lighthouse strategy — `all` capped ~2k or sampled, crawl clamp stays 10k); issue engine + per-page reporters on the normalized record; multipage SQL checks (duplicates, broken links, chains); orphans (sitemap-last + completeness gating).
+
+**Weeks 4–6:** Issues UX (severity, copy, drill-down, exclusions, CSV export); MCP audit tools with `how_to_fix` payloads; GSC-verified allowlist-fix flow in onboarding (UA-based WAF rule instructions).
+
+**Weeks 6–8:** client-side-rendering detection + disclosure; pricing posture shipped (free 50-page audits, credits beyond) with the free-tier TTL/retention policy from §4 cost #1; buffer for the hardening tail that always exists when you own the crawler.
+
+**Weeks 8–14 (P1):** scheduled audits + deltas with the 2-run link-graph retention policy; **AI-crawler accessibility report as the relaunch marketing asset**; thin-content/depth checks if not landed; security + external-link checks; opportunistic Verified Bots filing for `OpenSEOBot`.
+
+**Later (P2):** `npx` local agent (new ingest endpoint; gate on block telemetry vs the §4 triggers); Cloudflare Browser Rendering JS audits for GSC-verified sites if demand shows; R2 crawl snapshots; hreflang/structured-data/page-size.
+
+**The one-sentence version:** keep audits in the cloud on our own crawler (founder call — no vendor spend), give the small ones away to drive signups, spend engineering on a portable issue engine plus agent-actionable MCP output, answer bot-blocking with day-one honest "we were blocked" flagging and owner-targeted allowlist fixes validated per-platform in week 1, accept no JS rendering for now and Lighthouse-coupled page caps — with pre-set, measurable telemetry triggers that tell us if the no-vendor bet needs revisiting, a local CLI later, and a desktop app never.
diff --git a/drizzle-pg/0007_same_marvel_zombies.sql b/drizzle-pg/0007_same_marvel_zombies.sql
new file mode 100644
index 00000000..c6ed9bef
--- /dev/null
+++ b/drizzle-pg/0007_same_marvel_zombies.sql
@@ -0,0 +1,36 @@
+CREATE TABLE "audit_issues" (
+ "id" text PRIMARY KEY NOT NULL,
+ "audit_id" text NOT NULL,
+ "page_id" text,
+ "page_url" text NOT NULL,
+ "issue_type" text NOT NULL,
+ "severity" text DEFAULT 'info' NOT NULL,
+ "details_json" text
+);
+--> statement-breakpoint
+CREATE TABLE "audit_links" (
+ "id" text PRIMARY KEY NOT NULL,
+ "audit_id" text NOT NULL,
+ "source_page_id" text NOT NULL,
+ "source_url" text NOT NULL,
+ "target_url" text NOT NULL,
+ "anchor" text,
+ "is_internal" boolean DEFAULT true NOT NULL,
+ "is_nofollow" boolean DEFAULT false NOT NULL
+);
+--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "x_robots_tag" text;--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "header_canonical_url" text;--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "crawl_depth" integer;--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "in_sitemap" boolean DEFAULT false NOT NULL;--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "content_hash" text;--> statement-breakpoint
+ALTER TABLE "audit_pages" ADD COLUMN "fetch_class" text DEFAULT 'ok' NOT NULL;--> statement-breakpoint
+ALTER TABLE "audit_issues" ADD CONSTRAINT "audit_issues_audit_id_audits_id_fk" FOREIGN KEY ("audit_id") REFERENCES "public"."audits"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
+ALTER TABLE "audit_issues" ADD CONSTRAINT "audit_issues_page_id_audit_pages_id_fk" FOREIGN KEY ("page_id") REFERENCES "public"."audit_pages"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
+ALTER TABLE "audit_links" ADD CONSTRAINT "audit_links_audit_id_audits_id_fk" FOREIGN KEY ("audit_id") REFERENCES "public"."audits"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
+ALTER TABLE "audit_links" ADD CONSTRAINT "audit_links_source_page_id_audit_pages_id_fk" FOREIGN KEY ("source_page_id") REFERENCES "public"."audit_pages"("id") ON DELETE cascade ON UPDATE no action;--> statement-breakpoint
+CREATE INDEX "audit_issues_audit_id_idx" ON "audit_issues" USING btree ("audit_id");--> statement-breakpoint
+CREATE INDEX "audit_issues_audit_type_idx" ON "audit_issues" USING btree ("audit_id","issue_type");--> statement-breakpoint
+CREATE INDEX "audit_links_audit_id_idx" ON "audit_links" USING btree ("audit_id");--> statement-breakpoint
+CREATE INDEX "audit_links_audit_target_idx" ON "audit_links" USING btree ("audit_id","target_url");--> statement-breakpoint
+CREATE INDEX "audit_pages_audit_url_idx" ON "audit_pages" USING btree ("audit_id","url");
\ No newline at end of file
diff --git a/drizzle-pg/meta/0007_snapshot.json b/drizzle-pg/meta/0007_snapshot.json
new file mode 100644
index 00000000..b608cf7b
--- /dev/null
+++ b/drizzle-pg/meta/0007_snapshot.json
@@ -0,0 +1,3404 @@
+{
+ "id": "7317f11e-f526-4ac5-be84-0b3c6fad6ab0",
+ "prevId": "d7a0c2f9-269d-44eb-bc3e-c13b4bc3cb47",
+ "version": "7",
+ "dialect": "postgresql",
+ "tables": {
+ "public.keyword_metrics": {
+ "name": "keyword_metrics",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "serial",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'en'"
+ },
+ "search_volume": {
+ "name": "search_volume",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "cpc": {
+ "name": "cpc",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "competition": {
+ "name": "competition",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "keyword_difficulty": {
+ "name": "keyword_difficulty",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "intent": {
+ "name": "intent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "monthly_searches": {
+ "name": "monthly_searches",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "fetched_at": {
+ "name": "fetched_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "keyword_metrics_unique_project_keyword_location_language": {
+ "name": "keyword_metrics_unique_project_keyword_location_language",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "keyword",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "language_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "keyword_metrics_lookup_idx": {
+ "name": "keyword_metrics_lookup_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "keyword",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "language_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "fetched_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "keyword_metrics_project_id_projects_id_fk": {
+ "name": "keyword_metrics_project_id_projects_id_fk",
+ "tableFrom": "keyword_metrics",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.projects": {
+ "name": "projects",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "domain": {
+ "name": "domain",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'en'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "archived_at": {
+ "name": "archived_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "projects_one_default_per_organization_idx": {
+ "name": "projects_one_default_per_organization_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "where": "\"projects\".\"name\" = 'Default' AND \"projects\".\"domain\" IS NULL AND \"projects\".\"archived_at\" IS NULL",
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "projects_organization_id_idx": {
+ "name": "projects_organization_id_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "projects_organization_id_organization_id_fk": {
+ "name": "projects_organization_id_organization_id_fk",
+ "tableFrom": "projects",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.rank_check_runs": {
+ "name": "rank_check_runs",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "config_id": {
+ "name": "config_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'pending'"
+ },
+ "keywords_total": {
+ "name": "keywords_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "keywords_checked": {
+ "name": "keywords_checked",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "is_subset_run": {
+ "name": "is_subset_run",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ },
+ "error_message": {
+ "name": "error_message",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "started_at": {
+ "name": "started_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "rank_check_runs_config_idx": {
+ "name": "rank_check_runs_config_idx",
+ "columns": [
+ {
+ "expression": "config_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "started_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "rank_check_runs_project_idx": {
+ "name": "rank_check_runs_project_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "started_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "rank_check_runs_one_active_per_config_idx": {
+ "name": "rank_check_runs_one_active_per_config_idx",
+ "columns": [
+ {
+ "expression": "config_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "where": "\"rank_check_runs\".\"status\" IN ('pending', 'running')",
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "rank_check_runs_config_id_rank_tracking_configs_id_fk": {
+ "name": "rank_check_runs_config_id_rank_tracking_configs_id_fk",
+ "tableFrom": "rank_check_runs",
+ "tableTo": "rank_tracking_configs",
+ "columnsFrom": [
+ "config_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "rank_check_runs_project_id_projects_id_fk": {
+ "name": "rank_check_runs_project_id_projects_id_fk",
+ "tableFrom": "rank_check_runs",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.rank_snapshots": {
+ "name": "rank_snapshots",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "serial",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "run_id": {
+ "name": "run_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "tracking_keyword_id": {
+ "name": "tracking_keyword_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "device": {
+ "name": "device",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "position": {
+ "name": "position",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "url": {
+ "name": "url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "serp_features": {
+ "name": "serp_features",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "checked_at": {
+ "name": "checked_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "rank_snapshots_keyword_device_idx": {
+ "name": "rank_snapshots_keyword_device_idx",
+ "columns": [
+ {
+ "expression": "tracking_keyword_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "device",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "checked_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "rank_snapshots_run_keyword_device_idx": {
+ "name": "rank_snapshots_run_keyword_device_idx",
+ "columns": [
+ {
+ "expression": "run_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "tracking_keyword_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "device",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "rank_snapshots_run_id_rank_check_runs_id_fk": {
+ "name": "rank_snapshots_run_id_rank_check_runs_id_fk",
+ "tableFrom": "rank_snapshots",
+ "tableTo": "rank_check_runs",
+ "columnsFrom": [
+ "run_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.rank_tracking_configs": {
+ "name": "rank_tracking_configs",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "domain": {
+ "name": "domain",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'en'"
+ },
+ "devices": {
+ "name": "devices",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'both'"
+ },
+ "serp_depth": {
+ "name": "serp_depth",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "schedule_interval": {
+ "name": "schedule_interval",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'weekly'"
+ },
+ "location_name": {
+ "name": "location_name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "is_active": {
+ "name": "is_active",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": true
+ },
+ "last_checked_at": {
+ "name": "last_checked_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "next_check_at": {
+ "name": "next_check_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "last_skip_reason": {
+ "name": "last_skip_reason",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "rank_tracking_configs_national_idx": {
+ "name": "rank_tracking_configs_national_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "domain",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "where": "\"rank_tracking_configs\".\"location_name\" IS NULL",
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "rank_tracking_configs_local_idx": {
+ "name": "rank_tracking_configs_local_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "domain",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_name",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "where": "\"rank_tracking_configs\".\"location_name\" IS NOT NULL",
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "rank_tracking_configs_project_id_projects_id_fk": {
+ "name": "rank_tracking_configs_project_id_projects_id_fk",
+ "tableFrom": "rank_tracking_configs",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.rank_tracking_keywords": {
+ "name": "rank_tracking_keywords",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "config_id": {
+ "name": "config_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "search_volume": {
+ "name": "search_volume",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "keyword_difficulty": {
+ "name": "keyword_difficulty",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "cpc": {
+ "name": "cpc",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "metrics_fetched_at": {
+ "name": "metrics_fetched_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "rank_tracking_keywords_config_keyword_idx": {
+ "name": "rank_tracking_keywords_config_keyword_idx",
+ "columns": [
+ {
+ "expression": "config_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "keyword",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "rank_tracking_keywords_config_id_rank_tracking_configs_id_fk": {
+ "name": "rank_tracking_keywords_config_id_rank_tracking_configs_id_fk",
+ "tableFrom": "rank_tracking_keywords",
+ "tableTo": "rank_tracking_configs",
+ "columnsFrom": [
+ "config_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.saved_keyword_tag_assignments": {
+ "name": "saved_keyword_tag_assignments",
+ "schema": "",
+ "columns": {
+ "saved_keyword_id": {
+ "name": "saved_keyword_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "tag_id": {
+ "name": "tag_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "saved_keyword_tag_assignments_unique_idx": {
+ "name": "saved_keyword_tag_assignments_unique_idx",
+ "columns": [
+ {
+ "expression": "saved_keyword_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "tag_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "saved_keyword_tag_assignments_tag_idx": {
+ "name": "saved_keyword_tag_assignments_tag_idx",
+ "columns": [
+ {
+ "expression": "tag_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "saved_keyword_tag_assignments_saved_keyword_id_saved_keywords_id_fk": {
+ "name": "saved_keyword_tag_assignments_saved_keyword_id_saved_keywords_id_fk",
+ "tableFrom": "saved_keyword_tag_assignments",
+ "tableTo": "saved_keywords",
+ "columnsFrom": [
+ "saved_keyword_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "saved_keyword_tag_assignments_tag_id_saved_keyword_tags_id_fk": {
+ "name": "saved_keyword_tag_assignments_tag_id_saved_keyword_tags_id_fk",
+ "tableFrom": "saved_keyword_tag_assignments",
+ "tableTo": "saved_keyword_tags",
+ "columnsFrom": [
+ "tag_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.saved_keyword_tags": {
+ "name": "saved_keyword_tags",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "normalized_name": {
+ "name": "normalized_name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "color": {
+ "name": "color",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "saved_keyword_tags_project_normalized_name_idx": {
+ "name": "saved_keyword_tags_project_normalized_name_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "normalized_name",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "saved_keyword_tags_project_name_idx": {
+ "name": "saved_keyword_tags_project_name_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "name",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "saved_keyword_tags_project_id_projects_id_fk": {
+ "name": "saved_keyword_tags_project_id_projects_id_fk",
+ "tableFrom": "saved_keyword_tags",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.saved_keywords": {
+ "name": "saved_keywords",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'en'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "saved_keywords_unique_project_keyword_location_language": {
+ "name": "saved_keywords_unique_project_keyword_location_language",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "keyword",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "location_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "language_code",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "saved_keywords_project_created_idx": {
+ "name": "saved_keywords_project_created_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "created_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "saved_keywords_project_id_projects_id_fk": {
+ "name": "saved_keywords_project_id_projects_id_fk",
+ "tableFrom": "saved_keywords",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.user_onboarding_answers": {
+ "name": "user_onboarding_answers",
+ "schema": "",
+ "columns": {
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "interested_features": {
+ "name": "interested_features",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'[]'"
+ },
+ "work_for": {
+ "name": "work_for",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "client_website_count": {
+ "name": "client_website_count",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "found_via": {
+ "name": "found_via",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "mcp_setup_intent": {
+ "name": "mcp_setup_intent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "gsc_nudge_dismissed_at": {
+ "name": "gsc_nudge_dismissed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "user_onboarding_answers_organization_idx": {
+ "name": "user_onboarding_answers_organization_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "user_onboarding_answers_user_id_user_id_fk": {
+ "name": "user_onboarding_answers_user_id_user_id_fk",
+ "tableFrom": "user_onboarding_answers",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "user_onboarding_answers_organization_id_organization_id_fk": {
+ "name": "user_onboarding_answers_organization_id_organization_id_fk",
+ "tableFrom": "user_onboarding_answers",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.audit_issues": {
+ "name": "audit_issues",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "page_id": {
+ "name": "page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "page_url": {
+ "name": "page_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "issue_type": {
+ "name": "issue_type",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "severity": {
+ "name": "severity",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'info'"
+ },
+ "details_json": {
+ "name": "details_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "audit_issues_audit_id_idx": {
+ "name": "audit_issues_audit_id_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "audit_issues_audit_type_idx": {
+ "name": "audit_issues_audit_type_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "issue_type",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "audit_issues_audit_id_audits_id_fk": {
+ "name": "audit_issues_audit_id_audits_id_fk",
+ "tableFrom": "audit_issues",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_issues_page_id_audit_pages_id_fk": {
+ "name": "audit_issues_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_issues",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.audit_lighthouse_results": {
+ "name": "audit_lighthouse_results",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "page_id": {
+ "name": "page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "strategy": {
+ "name": "strategy",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "performance_score": {
+ "name": "performance_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "accessibility_score": {
+ "name": "accessibility_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "best_practices_score": {
+ "name": "best_practices_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "seo_score": {
+ "name": "seo_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "lcp_ms": {
+ "name": "lcp_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "cls": {
+ "name": "cls",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "inp_ms": {
+ "name": "inp_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "ttfb_ms": {
+ "name": "ttfb_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "error_message": {
+ "name": "error_message",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "r2_key": {
+ "name": "r2_key",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "payload_size_bytes": {
+ "name": "payload_size_bytes",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "audit_lighthouse_results_audit_id_idx": {
+ "name": "audit_lighthouse_results_audit_id_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "audit_lighthouse_results_audit_id_audits_id_fk": {
+ "name": "audit_lighthouse_results_audit_id_audits_id_fk",
+ "tableFrom": "audit_lighthouse_results",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_lighthouse_results_page_id_audit_pages_id_fk": {
+ "name": "audit_lighthouse_results_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_lighthouse_results",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.audit_links": {
+ "name": "audit_links",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "source_page_id": {
+ "name": "source_page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "source_url": {
+ "name": "source_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "target_url": {
+ "name": "target_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "anchor": {
+ "name": "anchor",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "is_internal": {
+ "name": "is_internal",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": true
+ },
+ "is_nofollow": {
+ "name": "is_nofollow",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ }
+ },
+ "indexes": {
+ "audit_links_audit_id_idx": {
+ "name": "audit_links_audit_id_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "audit_links_audit_target_idx": {
+ "name": "audit_links_audit_target_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "target_url",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "audit_links_audit_id_audits_id_fk": {
+ "name": "audit_links_audit_id_audits_id_fk",
+ "tableFrom": "audit_links",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_links_source_page_id_audit_pages_id_fk": {
+ "name": "audit_links_source_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_links",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "source_page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.audit_pages": {
+ "name": "audit_pages",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "url": {
+ "name": "url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "status_code": {
+ "name": "status_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "redirect_url": {
+ "name": "redirect_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "title": {
+ "name": "title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "meta_description": {
+ "name": "meta_description",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "canonical_url": {
+ "name": "canonical_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "robots_meta": {
+ "name": "robots_meta",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "og_title": {
+ "name": "og_title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "og_description": {
+ "name": "og_description",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "og_image": {
+ "name": "og_image",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "h1_count": {
+ "name": "h1_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "h2_count": {
+ "name": "h2_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "h3_count": {
+ "name": "h3_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "h4_count": {
+ "name": "h4_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "h5_count": {
+ "name": "h5_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "h6_count": {
+ "name": "h6_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "heading_order_json": {
+ "name": "heading_order_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "word_count": {
+ "name": "word_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "images_total": {
+ "name": "images_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "images_missing_alt": {
+ "name": "images_missing_alt",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "images_json": {
+ "name": "images_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "internal_link_count": {
+ "name": "internal_link_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "external_link_count": {
+ "name": "external_link_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "has_structured_data": {
+ "name": "has_structured_data",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ },
+ "hreflang_tags_json": {
+ "name": "hreflang_tags_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "is_indexable": {
+ "name": "is_indexable",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": true
+ },
+ "x_robots_tag": {
+ "name": "x_robots_tag",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "header_canonical_url": {
+ "name": "header_canonical_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "crawl_depth": {
+ "name": "crawl_depth",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "in_sitemap": {
+ "name": "in_sitemap",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ },
+ "content_hash": {
+ "name": "content_hash",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "fetch_class": {
+ "name": "fetch_class",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'ok'"
+ },
+ "response_time_ms": {
+ "name": "response_time_ms",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "audit_pages_audit_id_idx": {
+ "name": "audit_pages_audit_id_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "audit_pages_audit_url_idx": {
+ "name": "audit_pages_audit_url_idx",
+ "columns": [
+ {
+ "expression": "audit_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "url",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "audit_pages_audit_id_audits_id_fk": {
+ "name": "audit_pages_audit_id_audits_id_fk",
+ "tableFrom": "audit_pages",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.audits": {
+ "name": "audits",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "started_by_user_id": {
+ "name": "started_by_user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "start_url": {
+ "name": "start_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'running'"
+ },
+ "workflow_instance_id": {
+ "name": "workflow_instance_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "config": {
+ "name": "config",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'{}'"
+ },
+ "pages_crawled": {
+ "name": "pages_crawled",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "pages_total": {
+ "name": "pages_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "lighthouse_total": {
+ "name": "lighthouse_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "lighthouse_completed": {
+ "name": "lighthouse_completed",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "lighthouse_failed": {
+ "name": "lighthouse_failed",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "default": 0
+ },
+ "current_phase": {
+ "name": "current_phase",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "default": "'discovery'"
+ },
+ "started_at": {
+ "name": "started_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "audits_project_id_idx": {
+ "name": "audits_project_id_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "audits_started_by_user_id_idx": {
+ "name": "audits_started_by_user_id_idx",
+ "columns": [
+ {
+ "expression": "started_by_user_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "audits_project_id_projects_id_fk": {
+ "name": "audits_project_id_projects_id_fk",
+ "tableFrom": "audits",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.sam_project_memory": {
+ "name": "sam_project_memory",
+ "schema": "",
+ "columns": {
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "label": {
+ "name": "label",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "content": {
+ "name": "content",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {},
+ "foreignKeys": {
+ "sam_project_memory_project_id_projects_id_fk": {
+ "name": "sam_project_memory_project_id_projects_id_fk",
+ "tableFrom": "sam_project_memory",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {
+ "sam_project_memory_project_id_label_pk": {
+ "name": "sam_project_memory_project_id_label_pk",
+ "columns": [
+ "project_id",
+ "label"
+ ]
+ }
+ },
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.sam_sessions": {
+ "name": "sam_sessions",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "title": {
+ "name": "title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'New chat'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "archived_at": {
+ "name": "archived_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "sam_sessions_project_updated_idx": {
+ "name": "sam_sessions_project_updated_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "updated_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "sam_sessions_project_id_projects_id_fk": {
+ "name": "sam_sessions_project_id_projects_id_fk",
+ "tableFrom": "sam_sessions",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "sam_sessions_user_id_user_id_fk": {
+ "name": "sam_sessions_user_id_user_id_fk",
+ "tableFrom": "sam_sessions",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.account": {
+ "name": "account",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "account_id": {
+ "name": "account_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "provider_id": {
+ "name": "provider_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "access_token": {
+ "name": "access_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "refresh_token": {
+ "name": "refresh_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "id_token": {
+ "name": "id_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "access_token_expires_at": {
+ "name": "access_token_expires_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "refresh_token_expires_at": {
+ "name": "refresh_token_expires_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "scope": {
+ "name": "scope",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "password": {
+ "name": "password",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ }
+ },
+ "indexes": {
+ "account_userId_idx": {
+ "name": "account_userId_idx",
+ "columns": [
+ {
+ "expression": "user_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "account_accountId_providerId_idx": {
+ "name": "account_accountId_providerId_idx",
+ "columns": [
+ {
+ "expression": "account_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ },
+ {
+ "expression": "provider_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "account_user_id_user_id_fk": {
+ "name": "account_user_id_user_id_fk",
+ "tableFrom": "account",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.invitation": {
+ "name": "invitation",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "email": {
+ "name": "email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "role": {
+ "name": "role",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'pending'"
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "inviter_id": {
+ "name": "inviter_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ }
+ },
+ "indexes": {
+ "invitation_organizationId_idx": {
+ "name": "invitation_organizationId_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "invitation_email_idx": {
+ "name": "invitation_email_idx",
+ "columns": [
+ {
+ "expression": "email",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "invitation_organization_id_organization_id_fk": {
+ "name": "invitation_organization_id_organization_id_fk",
+ "tableFrom": "invitation",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "invitation_inviter_id_user_id_fk": {
+ "name": "invitation_inviter_id_user_id_fk",
+ "tableFrom": "invitation",
+ "tableTo": "user",
+ "columnsFrom": [
+ "inviter_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.member": {
+ "name": "member",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "role": {
+ "name": "role",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "'member'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ }
+ },
+ "indexes": {
+ "member_organizationId_idx": {
+ "name": "member_organizationId_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "member_userId_idx": {
+ "name": "member_userId_idx",
+ "columns": [
+ {
+ "expression": "user_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "member_organization_id_organization_id_fk": {
+ "name": "member_organization_id_organization_id_fk",
+ "tableFrom": "member",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "member_user_id_user_id_fk": {
+ "name": "member_user_id_user_id_fk",
+ "tableFrom": "member",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.organization": {
+ "name": "organization",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "slug": {
+ "name": "slug",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "logo": {
+ "name": "logo",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "metadata": {
+ "name": "metadata",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "organization_slug_uidx": {
+ "name": "organization_slug_uidx",
+ "columns": [
+ {
+ "expression": "slug",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {
+ "organization_slug_unique": {
+ "name": "organization_slug_unique",
+ "nullsNotDistinct": false,
+ "columns": [
+ "slug"
+ ]
+ }
+ },
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.session": {
+ "name": "session",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "token": {
+ "name": "token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "ip_address": {
+ "name": "ip_address",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "user_agent": {
+ "name": "user_agent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "active_organization_id": {
+ "name": "active_organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {
+ "session_userId_idx": {
+ "name": "session_userId_idx",
+ "columns": [
+ {
+ "expression": "user_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "session_user_id_user_id_fk": {
+ "name": "session_user_id_user_id_fk",
+ "tableFrom": "session",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {
+ "session_token_unique": {
+ "name": "session_token_unique",
+ "nullsNotDistinct": false,
+ "columns": [
+ "token"
+ ]
+ }
+ },
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.user": {
+ "name": "user",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "email": {
+ "name": "email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "email_verified": {
+ "name": "email_verified",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ },
+ "image": {
+ "name": "image",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "analytics_opted_out": {
+ "name": "analytics_opted_out",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": false
+ }
+ },
+ "indexes": {},
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {
+ "user_email_unique": {
+ "name": "user_email_unique",
+ "nullsNotDistinct": false,
+ "columns": [
+ "email"
+ ]
+ }
+ },
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.verification": {
+ "name": "verification",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "identifier": {
+ "name": "identifier",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "value": {
+ "name": "value",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "timestamp with time zone",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "now()"
+ }
+ },
+ "indexes": {
+ "verification_identifier_idx": {
+ "name": "verification_identifier_idx",
+ "columns": [
+ {
+ "expression": "identifier",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "verification_expiresAt_idx": {
+ "name": "verification_expiresAt_idx",
+ "columns": [
+ {
+ "expression": "expires_at",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.billing_customer_status": {
+ "name": "billing_customer_status",
+ "schema": "",
+ "columns": {
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "is_paying": {
+ "name": "is_paying",
+ "type": "boolean",
+ "primaryKey": false,
+ "notNull": true,
+ "default": false
+ },
+ "paid_plan_id": {
+ "name": "paid_plan_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "paid_plan_status": {
+ "name": "paid_plan_status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "customer_json": {
+ "name": "customer_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "synced_at": {
+ "name": "synced_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {},
+ "foreignKeys": {
+ "billing_customer_status_organization_id_organization_id_fk": {
+ "name": "billing_customer_status_organization_id_organization_id_fk",
+ "tableFrom": "billing_customer_status",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.gsc_connections": {
+ "name": "gsc_connections",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "site_url": {
+ "name": "site_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "connected_by_user_id": {
+ "name": "connected_by_user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "connected_account_email": {
+ "name": "connected_account_email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "gsc_connections_project_idx": {
+ "name": "gsc_connections_project_idx",
+ "columns": [
+ {
+ "expression": "project_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "gsc_connections_organization_idx": {
+ "name": "gsc_connections_organization_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "gsc_connections_project_id_projects_id_fk": {
+ "name": "gsc_connections_project_id_projects_id_fk",
+ "tableFrom": "gsc_connections",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "gsc_connections_organization_id_organization_id_fk": {
+ "name": "gsc_connections_organization_id_organization_id_fk",
+ "tableFrom": "gsc_connections",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ },
+ "public.reddit_attributions": {
+ "name": "reddit_attributions",
+ "schema": "",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true
+ },
+ "click_id": {
+ "name": "click_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "uuid": {
+ "name": "uuid",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "landing_page": {
+ "name": "landing_page",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "referrer": {
+ "name": "referrer",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "utm_source": {
+ "name": "utm_source",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "utm_medium": {
+ "name": "utm_medium",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "utm_campaign": {
+ "name": "utm_campaign",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "utm_term": {
+ "name": "utm_term",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "utm_content": {
+ "name": "utm_content",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "signup_sent_at": {
+ "name": "signup_sent_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "purchase_sent_at": {
+ "name": "purchase_sent_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "default": "to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD\"T\"HH24:MI:SS.MS\"Z\"')"
+ }
+ },
+ "indexes": {
+ "reddit_attributions_user_idx": {
+ "name": "reddit_attributions_user_idx",
+ "columns": [
+ {
+ "expression": "user_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": true,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ },
+ "reddit_attributions_organization_idx": {
+ "name": "reddit_attributions_organization_idx",
+ "columns": [
+ {
+ "expression": "organization_id",
+ "isExpression": false,
+ "asc": true,
+ "nulls": "last"
+ }
+ ],
+ "isUnique": false,
+ "concurrently": false,
+ "method": "btree",
+ "with": {}
+ }
+ },
+ "foreignKeys": {
+ "reddit_attributions_user_id_user_id_fk": {
+ "name": "reddit_attributions_user_id_user_id_fk",
+ "tableFrom": "reddit_attributions",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "reddit_attributions_organization_id_organization_id_fk": {
+ "name": "reddit_attributions_organization_id_organization_id_fk",
+ "tableFrom": "reddit_attributions",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "policies": {},
+ "checkConstraints": {},
+ "isRLSEnabled": false
+ }
+ },
+ "enums": {},
+ "schemas": {},
+ "sequences": {},
+ "roles": {},
+ "policies": {},
+ "views": {},
+ "_meta": {
+ "columns": {},
+ "schemas": {},
+ "tables": {}
+ }
+}
\ No newline at end of file
diff --git a/drizzle-pg/meta/_journal.json b/drizzle-pg/meta/_journal.json
index d50969bf..5c7196da 100644
--- a/drizzle-pg/meta/_journal.json
+++ b/drizzle-pg/meta/_journal.json
@@ -50,6 +50,13 @@
"when": 1783438455708,
"tag": "0006_location_name",
"breakpoints": true
+ },
+ {
+ "idx": 7,
+ "version": "7",
+ "when": 1783306049167,
+ "tag": "0007_same_marvel_zombies",
+ "breakpoints": true
}
]
}
\ No newline at end of file
diff --git a/drizzle/0030_legal_reaper.sql b/drizzle/0030_legal_reaper.sql
new file mode 100644
index 00000000..3bd9775d
--- /dev/null
+++ b/drizzle/0030_legal_reaper.sql
@@ -0,0 +1,36 @@
+CREATE TABLE `audit_issues` (
+ `id` text PRIMARY KEY NOT NULL,
+ `audit_id` text NOT NULL,
+ `page_id` text,
+ `page_url` text NOT NULL,
+ `issue_type` text NOT NULL,
+ `severity` text DEFAULT 'info' NOT NULL,
+ `details_json` text,
+ FOREIGN KEY (`audit_id`) REFERENCES `audits`(`id`) ON UPDATE no action ON DELETE cascade,
+ FOREIGN KEY (`page_id`) REFERENCES `audit_pages`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE INDEX `audit_issues_audit_id_idx` ON `audit_issues` (`audit_id`);--> statement-breakpoint
+CREATE INDEX `audit_issues_audit_type_idx` ON `audit_issues` (`audit_id`,`issue_type`);--> statement-breakpoint
+CREATE TABLE `audit_links` (
+ `id` text PRIMARY KEY NOT NULL,
+ `audit_id` text NOT NULL,
+ `source_page_id` text NOT NULL,
+ `source_url` text NOT NULL,
+ `target_url` text NOT NULL,
+ `anchor` text,
+ `is_internal` integer DEFAULT true NOT NULL,
+ `is_nofollow` integer DEFAULT false NOT NULL,
+ FOREIGN KEY (`audit_id`) REFERENCES `audits`(`id`) ON UPDATE no action ON DELETE cascade,
+ FOREIGN KEY (`source_page_id`) REFERENCES `audit_pages`(`id`) ON UPDATE no action ON DELETE cascade
+);
+--> statement-breakpoint
+CREATE INDEX `audit_links_audit_id_idx` ON `audit_links` (`audit_id`);--> statement-breakpoint
+CREATE INDEX `audit_links_audit_target_idx` ON `audit_links` (`audit_id`,`target_url`);--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `x_robots_tag` text;--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `header_canonical_url` text;--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `crawl_depth` integer;--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `in_sitemap` integer DEFAULT false NOT NULL;--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `content_hash` text;--> statement-breakpoint
+ALTER TABLE `audit_pages` ADD `fetch_class` text DEFAULT 'ok' NOT NULL;--> statement-breakpoint
+CREATE INDEX `audit_pages_audit_url_idx` ON `audit_pages` (`audit_id`,`url`);
\ No newline at end of file
diff --git a/drizzle/meta/0030_snapshot.json b/drizzle/meta/0030_snapshot.json
new file mode 100644
index 00000000..d5f4d230
--- /dev/null
+++ b/drizzle/meta/0030_snapshot.json
@@ -0,0 +1,3086 @@
+{
+ "version": "6",
+ "dialect": "sqlite",
+ "id": "57a79baa-091a-4d26-9324-dd8623e530a9",
+ "prevId": "04f10078-feca-42c4-9158-f1ba8df6091f",
+ "tables": {
+ "keyword_metrics": {
+ "name": "keyword_metrics",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "integer",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": true
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'en'"
+ },
+ "search_volume": {
+ "name": "search_volume",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "cpc": {
+ "name": "cpc",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "competition": {
+ "name": "competition",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "keyword_difficulty": {
+ "name": "keyword_difficulty",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "intent": {
+ "name": "intent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "monthly_searches": {
+ "name": "monthly_searches",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "fetched_at": {
+ "name": "fetched_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "keyword_metrics_unique_project_keyword_location_language": {
+ "name": "keyword_metrics_unique_project_keyword_location_language",
+ "columns": [
+ "project_id",
+ "keyword",
+ "location_code",
+ "language_code"
+ ],
+ "isUnique": true
+ },
+ "keyword_metrics_lookup_idx": {
+ "name": "keyword_metrics_lookup_idx",
+ "columns": [
+ "project_id",
+ "keyword",
+ "location_code",
+ "language_code",
+ "fetched_at"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "keyword_metrics_project_id_projects_id_fk": {
+ "name": "keyword_metrics_project_id_projects_id_fk",
+ "tableFrom": "keyword_metrics",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "projects": {
+ "name": "projects",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "domain": {
+ "name": "domain",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'en'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "archived_at": {
+ "name": "archived_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "projects_one_default_per_organization_idx": {
+ "name": "projects_one_default_per_organization_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": true,
+ "where": "\"projects\".\"name\" = 'Default' AND \"projects\".\"domain\" IS NULL AND \"projects\".\"archived_at\" IS NULL"
+ },
+ "projects_organization_id_idx": {
+ "name": "projects_organization_id_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "projects_organization_id_organization_id_fk": {
+ "name": "projects_organization_id_organization_id_fk",
+ "tableFrom": "projects",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "rank_check_runs": {
+ "name": "rank_check_runs",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "config_id": {
+ "name": "config_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'pending'"
+ },
+ "keywords_total": {
+ "name": "keywords_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "keywords_checked": {
+ "name": "keywords_checked",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "is_subset_run": {
+ "name": "is_subset_run",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ },
+ "error_message": {
+ "name": "error_message",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "started_at": {
+ "name": "started_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "rank_check_runs_config_idx": {
+ "name": "rank_check_runs_config_idx",
+ "columns": [
+ "config_id",
+ "started_at"
+ ],
+ "isUnique": false
+ },
+ "rank_check_runs_project_idx": {
+ "name": "rank_check_runs_project_idx",
+ "columns": [
+ "project_id",
+ "started_at"
+ ],
+ "isUnique": false
+ },
+ "rank_check_runs_one_active_per_config_idx": {
+ "name": "rank_check_runs_one_active_per_config_idx",
+ "columns": [
+ "config_id"
+ ],
+ "isUnique": true,
+ "where": "\"rank_check_runs\".\"status\" IN ('pending', 'running')"
+ }
+ },
+ "foreignKeys": {
+ "rank_check_runs_config_id_rank_tracking_configs_id_fk": {
+ "name": "rank_check_runs_config_id_rank_tracking_configs_id_fk",
+ "tableFrom": "rank_check_runs",
+ "tableTo": "rank_tracking_configs",
+ "columnsFrom": [
+ "config_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "rank_check_runs_project_id_projects_id_fk": {
+ "name": "rank_check_runs_project_id_projects_id_fk",
+ "tableFrom": "rank_check_runs",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "rank_snapshots": {
+ "name": "rank_snapshots",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "integer",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": true
+ },
+ "run_id": {
+ "name": "run_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "tracking_keyword_id": {
+ "name": "tracking_keyword_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "device": {
+ "name": "device",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "position": {
+ "name": "position",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "url": {
+ "name": "url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "serp_features": {
+ "name": "serp_features",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "checked_at": {
+ "name": "checked_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "rank_snapshots_keyword_device_idx": {
+ "name": "rank_snapshots_keyword_device_idx",
+ "columns": [
+ "tracking_keyword_id",
+ "device",
+ "checked_at"
+ ],
+ "isUnique": false
+ },
+ "rank_snapshots_run_keyword_device_idx": {
+ "name": "rank_snapshots_run_keyword_device_idx",
+ "columns": [
+ "run_id",
+ "tracking_keyword_id",
+ "device"
+ ],
+ "isUnique": true
+ }
+ },
+ "foreignKeys": {
+ "rank_snapshots_run_id_rank_check_runs_id_fk": {
+ "name": "rank_snapshots_run_id_rank_check_runs_id_fk",
+ "tableFrom": "rank_snapshots",
+ "tableTo": "rank_check_runs",
+ "columnsFrom": [
+ "run_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "rank_tracking_configs": {
+ "name": "rank_tracking_configs",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "domain": {
+ "name": "domain",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'en'"
+ },
+ "devices": {
+ "name": "devices",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'both'"
+ },
+ "serp_depth": {
+ "name": "serp_depth",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "schedule_interval": {
+ "name": "schedule_interval",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'weekly'"
+ },
+ "location_name": {
+ "name": "location_name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "is_active": {
+ "name": "is_active",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": true
+ },
+ "last_checked_at": {
+ "name": "last_checked_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "next_check_at": {
+ "name": "next_check_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "last_skip_reason": {
+ "name": "last_skip_reason",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "rank_tracking_configs_national_idx": {
+ "name": "rank_tracking_configs_national_idx",
+ "columns": [
+ "project_id",
+ "domain",
+ "location_code"
+ ],
+ "isUnique": true,
+ "where": "\"rank_tracking_configs\".\"location_name\" IS NULL"
+ },
+ "rank_tracking_configs_local_idx": {
+ "name": "rank_tracking_configs_local_idx",
+ "columns": [
+ "project_id",
+ "domain",
+ "location_code",
+ "location_name"
+ ],
+ "isUnique": true,
+ "where": "\"rank_tracking_configs\".\"location_name\" IS NOT NULL"
+ }
+ },
+ "foreignKeys": {
+ "rank_tracking_configs_project_id_projects_id_fk": {
+ "name": "rank_tracking_configs_project_id_projects_id_fk",
+ "tableFrom": "rank_tracking_configs",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "rank_tracking_keywords": {
+ "name": "rank_tracking_keywords",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "config_id": {
+ "name": "config_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "search_volume": {
+ "name": "search_volume",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "keyword_difficulty": {
+ "name": "keyword_difficulty",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "cpc": {
+ "name": "cpc",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "metrics_fetched_at": {
+ "name": "metrics_fetched_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "rank_tracking_keywords_config_keyword_idx": {
+ "name": "rank_tracking_keywords_config_keyword_idx",
+ "columns": [
+ "config_id",
+ "keyword"
+ ],
+ "isUnique": true
+ }
+ },
+ "foreignKeys": {
+ "rank_tracking_keywords_config_id_rank_tracking_configs_id_fk": {
+ "name": "rank_tracking_keywords_config_id_rank_tracking_configs_id_fk",
+ "tableFrom": "rank_tracking_keywords",
+ "tableTo": "rank_tracking_configs",
+ "columnsFrom": [
+ "config_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "saved_keyword_tag_assignments": {
+ "name": "saved_keyword_tag_assignments",
+ "columns": {
+ "saved_keyword_id": {
+ "name": "saved_keyword_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "tag_id": {
+ "name": "tag_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "saved_keyword_tag_assignments_unique_idx": {
+ "name": "saved_keyword_tag_assignments_unique_idx",
+ "columns": [
+ "saved_keyword_id",
+ "tag_id"
+ ],
+ "isUnique": true
+ },
+ "saved_keyword_tag_assignments_tag_idx": {
+ "name": "saved_keyword_tag_assignments_tag_idx",
+ "columns": [
+ "tag_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "saved_keyword_tag_assignments_saved_keyword_id_saved_keywords_id_fk": {
+ "name": "saved_keyword_tag_assignments_saved_keyword_id_saved_keywords_id_fk",
+ "tableFrom": "saved_keyword_tag_assignments",
+ "tableTo": "saved_keywords",
+ "columnsFrom": [
+ "saved_keyword_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "saved_keyword_tag_assignments_tag_id_saved_keyword_tags_id_fk": {
+ "name": "saved_keyword_tag_assignments_tag_id_saved_keyword_tags_id_fk",
+ "tableFrom": "saved_keyword_tag_assignments",
+ "tableTo": "saved_keyword_tags",
+ "columnsFrom": [
+ "tag_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "saved_keyword_tags": {
+ "name": "saved_keyword_tags",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "normalized_name": {
+ "name": "normalized_name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "color": {
+ "name": "color",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "saved_keyword_tags_project_normalized_name_idx": {
+ "name": "saved_keyword_tags_project_normalized_name_idx",
+ "columns": [
+ "project_id",
+ "normalized_name"
+ ],
+ "isUnique": true
+ },
+ "saved_keyword_tags_project_name_idx": {
+ "name": "saved_keyword_tags_project_name_idx",
+ "columns": [
+ "project_id",
+ "name"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "saved_keyword_tags_project_id_projects_id_fk": {
+ "name": "saved_keyword_tags_project_id_projects_id_fk",
+ "tableFrom": "saved_keyword_tags",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "saved_keywords": {
+ "name": "saved_keywords",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "keyword": {
+ "name": "keyword",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "location_code": {
+ "name": "location_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 2840
+ },
+ "language_code": {
+ "name": "language_code",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'en'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "saved_keywords_unique_project_keyword_location_language": {
+ "name": "saved_keywords_unique_project_keyword_location_language",
+ "columns": [
+ "project_id",
+ "keyword",
+ "location_code",
+ "language_code"
+ ],
+ "isUnique": true
+ },
+ "saved_keywords_project_created_idx": {
+ "name": "saved_keywords_project_created_idx",
+ "columns": [
+ "project_id",
+ "created_at"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "saved_keywords_project_id_projects_id_fk": {
+ "name": "saved_keywords_project_id_projects_id_fk",
+ "tableFrom": "saved_keywords",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "user_onboarding_answers": {
+ "name": "user_onboarding_answers",
+ "columns": {
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "interested_features": {
+ "name": "interested_features",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'[]'"
+ },
+ "work_for": {
+ "name": "work_for",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "client_website_count": {
+ "name": "client_website_count",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "found_via": {
+ "name": "found_via",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "mcp_setup_intent": {
+ "name": "mcp_setup_intent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "gsc_nudge_dismissed_at": {
+ "name": "gsc_nudge_dismissed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "user_onboarding_answers_organization_idx": {
+ "name": "user_onboarding_answers_organization_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "user_onboarding_answers_user_id_user_id_fk": {
+ "name": "user_onboarding_answers_user_id_user_id_fk",
+ "tableFrom": "user_onboarding_answers",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "user_onboarding_answers_organization_id_organization_id_fk": {
+ "name": "user_onboarding_answers_organization_id_organization_id_fk",
+ "tableFrom": "user_onboarding_answers",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "audit_issues": {
+ "name": "audit_issues",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "page_id": {
+ "name": "page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "page_url": {
+ "name": "page_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "issue_type": {
+ "name": "issue_type",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "severity": {
+ "name": "severity",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'info'"
+ },
+ "details_json": {
+ "name": "details_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "audit_issues_audit_id_idx": {
+ "name": "audit_issues_audit_id_idx",
+ "columns": [
+ "audit_id"
+ ],
+ "isUnique": false
+ },
+ "audit_issues_audit_type_idx": {
+ "name": "audit_issues_audit_type_idx",
+ "columns": [
+ "audit_id",
+ "issue_type"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "audit_issues_audit_id_audits_id_fk": {
+ "name": "audit_issues_audit_id_audits_id_fk",
+ "tableFrom": "audit_issues",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_issues_page_id_audit_pages_id_fk": {
+ "name": "audit_issues_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_issues",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "audit_lighthouse_results": {
+ "name": "audit_lighthouse_results",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "page_id": {
+ "name": "page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "strategy": {
+ "name": "strategy",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "performance_score": {
+ "name": "performance_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "accessibility_score": {
+ "name": "accessibility_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "best_practices_score": {
+ "name": "best_practices_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "seo_score": {
+ "name": "seo_score",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "lcp_ms": {
+ "name": "lcp_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "cls": {
+ "name": "cls",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "inp_ms": {
+ "name": "inp_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "ttfb_ms": {
+ "name": "ttfb_ms",
+ "type": "real",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "error_message": {
+ "name": "error_message",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "r2_key": {
+ "name": "r2_key",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "payload_size_bytes": {
+ "name": "payload_size_bytes",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "audit_lighthouse_results_audit_id_idx": {
+ "name": "audit_lighthouse_results_audit_id_idx",
+ "columns": [
+ "audit_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "audit_lighthouse_results_audit_id_audits_id_fk": {
+ "name": "audit_lighthouse_results_audit_id_audits_id_fk",
+ "tableFrom": "audit_lighthouse_results",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_lighthouse_results_page_id_audit_pages_id_fk": {
+ "name": "audit_lighthouse_results_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_lighthouse_results",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "audit_links": {
+ "name": "audit_links",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "source_page_id": {
+ "name": "source_page_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "source_url": {
+ "name": "source_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "target_url": {
+ "name": "target_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "anchor": {
+ "name": "anchor",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "is_internal": {
+ "name": "is_internal",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": true
+ },
+ "is_nofollow": {
+ "name": "is_nofollow",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ }
+ },
+ "indexes": {
+ "audit_links_audit_id_idx": {
+ "name": "audit_links_audit_id_idx",
+ "columns": [
+ "audit_id"
+ ],
+ "isUnique": false
+ },
+ "audit_links_audit_target_idx": {
+ "name": "audit_links_audit_target_idx",
+ "columns": [
+ "audit_id",
+ "target_url"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "audit_links_audit_id_audits_id_fk": {
+ "name": "audit_links_audit_id_audits_id_fk",
+ "tableFrom": "audit_links",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "audit_links_source_page_id_audit_pages_id_fk": {
+ "name": "audit_links_source_page_id_audit_pages_id_fk",
+ "tableFrom": "audit_links",
+ "tableTo": "audit_pages",
+ "columnsFrom": [
+ "source_page_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "audit_pages": {
+ "name": "audit_pages",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "audit_id": {
+ "name": "audit_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "url": {
+ "name": "url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "status_code": {
+ "name": "status_code",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "redirect_url": {
+ "name": "redirect_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "title": {
+ "name": "title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "meta_description": {
+ "name": "meta_description",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "canonical_url": {
+ "name": "canonical_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "robots_meta": {
+ "name": "robots_meta",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "og_title": {
+ "name": "og_title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "og_description": {
+ "name": "og_description",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "og_image": {
+ "name": "og_image",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "h1_count": {
+ "name": "h1_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "h2_count": {
+ "name": "h2_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "h3_count": {
+ "name": "h3_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "h4_count": {
+ "name": "h4_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "h5_count": {
+ "name": "h5_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "h6_count": {
+ "name": "h6_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "heading_order_json": {
+ "name": "heading_order_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "word_count": {
+ "name": "word_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "images_total": {
+ "name": "images_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "images_missing_alt": {
+ "name": "images_missing_alt",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "images_json": {
+ "name": "images_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "internal_link_count": {
+ "name": "internal_link_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "external_link_count": {
+ "name": "external_link_count",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "has_structured_data": {
+ "name": "has_structured_data",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ },
+ "hreflang_tags_json": {
+ "name": "hreflang_tags_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "is_indexable": {
+ "name": "is_indexable",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": true
+ },
+ "x_robots_tag": {
+ "name": "x_robots_tag",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "header_canonical_url": {
+ "name": "header_canonical_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "crawl_depth": {
+ "name": "crawl_depth",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "in_sitemap": {
+ "name": "in_sitemap",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ },
+ "content_hash": {
+ "name": "content_hash",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "fetch_class": {
+ "name": "fetch_class",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'ok'"
+ },
+ "response_time_ms": {
+ "name": "response_time_ms",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "audit_pages_audit_id_idx": {
+ "name": "audit_pages_audit_id_idx",
+ "columns": [
+ "audit_id"
+ ],
+ "isUnique": false
+ },
+ "audit_pages_audit_url_idx": {
+ "name": "audit_pages_audit_url_idx",
+ "columns": [
+ "audit_id",
+ "url"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "audit_pages_audit_id_audits_id_fk": {
+ "name": "audit_pages_audit_id_audits_id_fk",
+ "tableFrom": "audit_pages",
+ "tableTo": "audits",
+ "columnsFrom": [
+ "audit_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "audits": {
+ "name": "audits",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "started_by_user_id": {
+ "name": "started_by_user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "start_url": {
+ "name": "start_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'running'"
+ },
+ "workflow_instance_id": {
+ "name": "workflow_instance_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "config": {
+ "name": "config",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'{}'"
+ },
+ "pages_crawled": {
+ "name": "pages_crawled",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "pages_total": {
+ "name": "pages_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "lighthouse_total": {
+ "name": "lighthouse_total",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "lighthouse_completed": {
+ "name": "lighthouse_completed",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "lighthouse_failed": {
+ "name": "lighthouse_failed",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": 0
+ },
+ "current_phase": {
+ "name": "current_phase",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false,
+ "default": "'discovery'"
+ },
+ "started_at": {
+ "name": "started_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "completed_at": {
+ "name": "completed_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "audits_project_id_idx": {
+ "name": "audits_project_id_idx",
+ "columns": [
+ "project_id"
+ ],
+ "isUnique": false
+ },
+ "audits_started_by_user_id_idx": {
+ "name": "audits_started_by_user_id_idx",
+ "columns": [
+ "started_by_user_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "audits_project_id_projects_id_fk": {
+ "name": "audits_project_id_projects_id_fk",
+ "tableFrom": "audits",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "sam_project_memory": {
+ "name": "sam_project_memory",
+ "columns": {
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "label": {
+ "name": "label",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "content": {
+ "name": "content",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {},
+ "foreignKeys": {
+ "sam_project_memory_project_id_projects_id_fk": {
+ "name": "sam_project_memory_project_id_projects_id_fk",
+ "tableFrom": "sam_project_memory",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {
+ "sam_project_memory_project_id_label_pk": {
+ "columns": [
+ "project_id",
+ "label"
+ ],
+ "name": "sam_project_memory_project_id_label_pk"
+ }
+ },
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "sam_sessions": {
+ "name": "sam_sessions",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "title": {
+ "name": "title",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'New chat'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "archived_at": {
+ "name": "archived_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "sam_sessions_project_updated_idx": {
+ "name": "sam_sessions_project_updated_idx",
+ "columns": [
+ "project_id",
+ "updated_at"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "sam_sessions_project_id_projects_id_fk": {
+ "name": "sam_sessions_project_id_projects_id_fk",
+ "tableFrom": "sam_sessions",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "sam_sessions_user_id_user_id_fk": {
+ "name": "sam_sessions_user_id_user_id_fk",
+ "tableFrom": "sam_sessions",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "account": {
+ "name": "account",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "account_id": {
+ "name": "account_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "provider_id": {
+ "name": "provider_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "access_token": {
+ "name": "access_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "refresh_token": {
+ "name": "refresh_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "id_token": {
+ "name": "id_token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "access_token_expires_at": {
+ "name": "access_token_expires_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "refresh_token_expires_at": {
+ "name": "refresh_token_expires_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "scope": {
+ "name": "scope",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "password": {
+ "name": "password",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "account_userId_idx": {
+ "name": "account_userId_idx",
+ "columns": [
+ "user_id"
+ ],
+ "isUnique": false
+ },
+ "account_accountId_providerId_idx": {
+ "name": "account_accountId_providerId_idx",
+ "columns": [
+ "account_id",
+ "provider_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "account_user_id_user_id_fk": {
+ "name": "account_user_id_user_id_fk",
+ "tableFrom": "account",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "invitation": {
+ "name": "invitation",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "email": {
+ "name": "email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "role": {
+ "name": "role",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "status": {
+ "name": "status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'pending'"
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "inviter_id": {
+ "name": "inviter_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "invitation_organizationId_idx": {
+ "name": "invitation_organizationId_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ },
+ "invitation_email_idx": {
+ "name": "invitation_email_idx",
+ "columns": [
+ "email"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "invitation_organization_id_organization_id_fk": {
+ "name": "invitation_organization_id_organization_id_fk",
+ "tableFrom": "invitation",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "invitation_inviter_id_user_id_fk": {
+ "name": "invitation_inviter_id_user_id_fk",
+ "tableFrom": "invitation",
+ "tableTo": "user",
+ "columnsFrom": [
+ "inviter_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "member": {
+ "name": "member",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "role": {
+ "name": "role",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "'member'"
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "member_organizationId_idx": {
+ "name": "member_organizationId_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ },
+ "member_userId_idx": {
+ "name": "member_userId_idx",
+ "columns": [
+ "user_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "member_organization_id_organization_id_fk": {
+ "name": "member_organization_id_organization_id_fk",
+ "tableFrom": "member",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "member_user_id_user_id_fk": {
+ "name": "member_user_id_user_id_fk",
+ "tableFrom": "member",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "organization": {
+ "name": "organization",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "slug": {
+ "name": "slug",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "logo": {
+ "name": "logo",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "metadata": {
+ "name": "metadata",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "organization_slug_unique": {
+ "name": "organization_slug_unique",
+ "columns": [
+ "slug"
+ ],
+ "isUnique": true
+ },
+ "organization_slug_uidx": {
+ "name": "organization_slug_uidx",
+ "columns": [
+ "slug"
+ ],
+ "isUnique": true
+ }
+ },
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "session": {
+ "name": "session",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "token": {
+ "name": "token",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "ip_address": {
+ "name": "ip_address",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "user_agent": {
+ "name": "user_agent",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "active_organization_id": {
+ "name": "active_organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "session_token_unique": {
+ "name": "session_token_unique",
+ "columns": [
+ "token"
+ ],
+ "isUnique": true
+ },
+ "session_userId_idx": {
+ "name": "session_userId_idx",
+ "columns": [
+ "user_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "session_user_id_user_id_fk": {
+ "name": "session_user_id_user_id_fk",
+ "tableFrom": "session",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "user": {
+ "name": "user",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "name": {
+ "name": "name",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "email": {
+ "name": "email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "email_verified": {
+ "name": "email_verified",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ },
+ "image": {
+ "name": "image",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "analytics_opted_out": {
+ "name": "analytics_opted_out",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ }
+ },
+ "indexes": {
+ "user_email_unique": {
+ "name": "user_email_unique",
+ "columns": [
+ "email"
+ ],
+ "isUnique": true
+ }
+ },
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "verification": {
+ "name": "verification",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "identifier": {
+ "name": "identifier",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "value": {
+ "name": "value",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "expires_at": {
+ "name": "expires_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(cast(unixepoch('subsecond') * 1000 as integer))"
+ }
+ },
+ "indexes": {
+ "verification_identifier_idx": {
+ "name": "verification_identifier_idx",
+ "columns": [
+ "identifier"
+ ],
+ "isUnique": false
+ },
+ "verification_expiresAt_idx": {
+ "name": "verification_expiresAt_idx",
+ "columns": [
+ "expires_at"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {},
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "billing_customer_status": {
+ "name": "billing_customer_status",
+ "columns": {
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "is_paying": {
+ "name": "is_paying",
+ "type": "integer",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": false
+ },
+ "paid_plan_id": {
+ "name": "paid_plan_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "paid_plan_status": {
+ "name": "paid_plan_status",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "customer_json": {
+ "name": "customer_json",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "synced_at": {
+ "name": "synced_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {},
+ "foreignKeys": {
+ "billing_customer_status_organization_id_organization_id_fk": {
+ "name": "billing_customer_status_organization_id_organization_id_fk",
+ "tableFrom": "billing_customer_status",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "gsc_connections": {
+ "name": "gsc_connections",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "project_id": {
+ "name": "project_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "site_url": {
+ "name": "site_url",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "connected_by_user_id": {
+ "name": "connected_by_user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "connected_account_email": {
+ "name": "connected_account_email",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "gsc_connections_project_idx": {
+ "name": "gsc_connections_project_idx",
+ "columns": [
+ "project_id"
+ ],
+ "isUnique": true
+ },
+ "gsc_connections_organization_idx": {
+ "name": "gsc_connections_organization_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "gsc_connections_project_id_projects_id_fk": {
+ "name": "gsc_connections_project_id_projects_id_fk",
+ "tableFrom": "gsc_connections",
+ "tableTo": "projects",
+ "columnsFrom": [
+ "project_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "gsc_connections_organization_id_organization_id_fk": {
+ "name": "gsc_connections_organization_id_organization_id_fk",
+ "tableFrom": "gsc_connections",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ },
+ "reddit_attributions": {
+ "name": "reddit_attributions",
+ "columns": {
+ "id": {
+ "name": "id",
+ "type": "text",
+ "primaryKey": true,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "user_id": {
+ "name": "user_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "organization_id": {
+ "name": "organization_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false
+ },
+ "click_id": {
+ "name": "click_id",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "uuid": {
+ "name": "uuid",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "landing_page": {
+ "name": "landing_page",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "referrer": {
+ "name": "referrer",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "utm_source": {
+ "name": "utm_source",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "utm_medium": {
+ "name": "utm_medium",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "utm_campaign": {
+ "name": "utm_campaign",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "utm_term": {
+ "name": "utm_term",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "utm_content": {
+ "name": "utm_content",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "signup_sent_at": {
+ "name": "signup_sent_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "purchase_sent_at": {
+ "name": "purchase_sent_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": false,
+ "autoincrement": false
+ },
+ "created_at": {
+ "name": "created_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ },
+ "updated_at": {
+ "name": "updated_at",
+ "type": "text",
+ "primaryKey": false,
+ "notNull": true,
+ "autoincrement": false,
+ "default": "(current_timestamp)"
+ }
+ },
+ "indexes": {
+ "reddit_attributions_user_idx": {
+ "name": "reddit_attributions_user_idx",
+ "columns": [
+ "user_id"
+ ],
+ "isUnique": true
+ },
+ "reddit_attributions_organization_idx": {
+ "name": "reddit_attributions_organization_idx",
+ "columns": [
+ "organization_id"
+ ],
+ "isUnique": false
+ }
+ },
+ "foreignKeys": {
+ "reddit_attributions_user_id_user_id_fk": {
+ "name": "reddit_attributions_user_id_user_id_fk",
+ "tableFrom": "reddit_attributions",
+ "tableTo": "user",
+ "columnsFrom": [
+ "user_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ },
+ "reddit_attributions_organization_id_organization_id_fk": {
+ "name": "reddit_attributions_organization_id_organization_id_fk",
+ "tableFrom": "reddit_attributions",
+ "tableTo": "organization",
+ "columnsFrom": [
+ "organization_id"
+ ],
+ "columnsTo": [
+ "id"
+ ],
+ "onDelete": "cascade",
+ "onUpdate": "no action"
+ }
+ },
+ "compositePrimaryKeys": {},
+ "uniqueConstraints": {},
+ "checkConstraints": {}
+ }
+ },
+ "views": {},
+ "enums": {},
+ "_meta": {
+ "schemas": {},
+ "tables": {},
+ "columns": {}
+ },
+ "internal": {
+ "indexes": {}
+ }
+}
\ No newline at end of file
diff --git a/drizzle/meta/_journal.json b/drizzle/meta/_journal.json
index 8c858fda..dd32744a 100644
--- a/drizzle/meta/_journal.json
+++ b/drizzle/meta/_journal.json
@@ -211,6 +211,13 @@
"when": 1783438453855,
"tag": "0029_location_name",
"breakpoints": true
+ },
+ {
+ "idx": 30,
+ "version": "6",
+ "when": 1783306047190,
+ "tag": "0030_legal_reaper",
+ "breakpoints": true
}
]
}
\ No newline at end of file
diff --git a/src/db/app.schema.ts b/src/db/app.schema.ts
index 382dd652..a292982c 100644
--- a/src/db/app.schema.ts
+++ b/src/db/app.schema.ts
@@ -345,123 +345,3 @@ export const rankSnapshots = sqliteTable(
),
],
);
-
-// ============================================================================
-// Site Audit tables
-// ============================================================================
-
-// One row per audit run
-export const audits = sqliteTable(
- "audits",
- {
- id: text("id").primaryKey(),
- projectId: text("project_id")
- .notNull()
- .references(() => projects.id, { onDelete: "cascade" }),
- startedByUserId: text("started_by_user_id").notNull(),
- startUrl: text("start_url").notNull(),
- status: text("status", {
- enum: ["running", "completed", "failed"],
- })
- .notNull()
- .default("running"),
- workflowInstanceId: text("workflow_instance_id"),
- // JSON config: { maxPages, lighthouseStrategy }
- config: text("config").notNull().default("{}"),
- // Progress & summary
- pagesCrawled: integer("pages_crawled").notNull().default(0),
- pagesTotal: integer("pages_total").notNull().default(0),
- lighthouseTotal: integer("lighthouse_total").notNull().default(0),
- lighthouseCompleted: integer("lighthouse_completed").notNull().default(0),
- lighthouseFailed: integer("lighthouse_failed").notNull().default(0),
- currentPhase: text("current_phase").default("discovery"),
- startedAt: text("started_at")
- .notNull()
- .default(sql`(current_timestamp)`),
- completedAt: text("completed_at"),
- },
- (table) => [
- index("audits_project_id_idx").on(table.projectId),
- index("audits_started_by_user_id_idx").on(table.startedByUserId),
- ],
-);
-
-// One row per crawled page
-export const auditPages = sqliteTable(
- "audit_pages",
- {
- id: text("id").primaryKey(),
- auditId: text("audit_id")
- .notNull()
- .references(() => audits.id, { onDelete: "cascade" }),
- url: text("url").notNull(),
- statusCode: integer("status_code"),
- redirectUrl: text("redirect_url"),
- // Metadata
- title: text("title"),
- metaDescription: text("meta_description"),
- canonicalUrl: text("canonical_url"),
- robotsMeta: text("robots_meta"),
- // Open Graph
- ogTitle: text("og_title"),
- ogDescription: text("og_description"),
- ogImage: text("og_image"),
- // Headings
- h1Count: integer("h1_count").notNull().default(0),
- h2Count: integer("h2_count").notNull().default(0),
- h3Count: integer("h3_count").notNull().default(0),
- h4Count: integer("h4_count").notNull().default(0),
- h5Count: integer("h5_count").notNull().default(0),
- h6Count: integer("h6_count").notNull().default(0),
- headingOrderJson: text("heading_order_json"),
- // Content
- wordCount: integer("word_count").notNull().default(0),
- // Images
- imagesTotal: integer("images_total").notNull().default(0),
- imagesMissingAlt: integer("images_missing_alt").notNull().default(0),
- imagesJson: text("images_json"),
- // Links
- internalLinkCount: integer("internal_link_count").notNull().default(0),
- externalLinkCount: integer("external_link_count").notNull().default(0),
- // Structured data
- hasStructuredData: integer("has_structured_data", { mode: "boolean" })
- .notNull()
- .default(false),
- // Hreflang
- hreflangTagsJson: text("hreflang_tags_json"),
- // Indexability
- isIndexable: integer("is_indexable", { mode: "boolean" })
- .notNull()
- .default(true),
- // Performance
- responseTimeMs: integer("response_time_ms"),
- },
- (table) => [index("audit_pages_audit_id_idx").on(table.auditId)],
-);
-
-// One row per Lighthouse test (mobile + desktop per page).
-export const auditLighthouseResults = sqliteTable(
- "audit_lighthouse_results",
- {
- id: text("id").primaryKey(),
- auditId: text("audit_id")
- .notNull()
- .references(() => audits.id, { onDelete: "cascade" }),
- pageId: text("page_id")
- .notNull()
- .references(() => auditPages.id, { onDelete: "cascade" }),
- strategy: text("strategy", { enum: ["mobile", "desktop"] }).notNull(),
- performanceScore: integer("performance_score"),
- accessibilityScore: integer("accessibility_score"),
- bestPracticesScore: integer("best_practices_score"),
- seoScore: integer("seo_score"),
- lcpMs: real("lcp_ms"),
- cls: real("cls"),
- inpMs: real("inp_ms"),
- ttfbMs: real("ttfb_ms"),
- errorMessage: text("error_message"),
- r2Key: text("r2_key"),
- payloadSizeBytes: integer("payload_size_bytes"),
- },
- (table) => [index("audit_lighthouse_results_audit_id_idx").on(table.auditId)],
-);
diff --git a/src/db/audit.schema.ts b/src/db/audit.schema.ts
new file mode 100644
index 00000000..55c3fa61
--- /dev/null
+++ b/src/db/audit.schema.ts
@@ -0,0 +1,201 @@
+import {
+ sqliteTable,
+ text,
+ integer,
+ real,
+ index,
+} from "drizzle-orm/sqlite-core";
+import { sql } from "drizzle-orm";
+import { projects } from "./app.schema";
+
+// ============================================================================
+// Site Audit tables
+// ============================================================================
+
+// One row per audit run
+export const audits = sqliteTable(
+ "audits",
+ {
+ id: text("id").primaryKey(),
+ projectId: text("project_id")
+ .notNull()
+ .references(() => projects.id, { onDelete: "cascade" }),
+ startedByUserId: text("started_by_user_id").notNull(),
+ startUrl: text("start_url").notNull(),
+ status: text("status", {
+ enum: ["running", "completed", "failed"],
+ })
+ .notNull()
+ .default("running"),
+ workflowInstanceId: text("workflow_instance_id"),
+ // JSON config: { maxPages, lighthouseStrategy }
+ config: text("config").notNull().default("{}"),
+ // Progress & summary
+ pagesCrawled: integer("pages_crawled").notNull().default(0),
+ pagesTotal: integer("pages_total").notNull().default(0),
+ lighthouseTotal: integer("lighthouse_total").notNull().default(0),
+ lighthouseCompleted: integer("lighthouse_completed").notNull().default(0),
+ lighthouseFailed: integer("lighthouse_failed").notNull().default(0),
+ currentPhase: text("current_phase").default("discovery"),
+ startedAt: text("started_at")
+ .notNull()
+ .default(sql`(current_timestamp)`),
+ completedAt: text("completed_at"),
+ },
+ (table) => [
+ index("audits_project_id_idx").on(table.projectId),
+ index("audits_started_by_user_id_idx").on(table.startedByUserId),
+ ],
+);
+
+// One row per crawled page
+export const auditPages = sqliteTable(
+ "audit_pages",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ url: text("url").notNull(),
+ statusCode: integer("status_code"),
+ redirectUrl: text("redirect_url"),
+ // Metadata
+ title: text("title"),
+ metaDescription: text("meta_description"),
+ canonicalUrl: text("canonical_url"),
+ robotsMeta: text("robots_meta"),
+ // Open Graph
+ ogTitle: text("og_title"),
+ ogDescription: text("og_description"),
+ ogImage: text("og_image"),
+ // Headings
+ h1Count: integer("h1_count").notNull().default(0),
+ h2Count: integer("h2_count").notNull().default(0),
+ h3Count: integer("h3_count").notNull().default(0),
+ h4Count: integer("h4_count").notNull().default(0),
+ h5Count: integer("h5_count").notNull().default(0),
+ h6Count: integer("h6_count").notNull().default(0),
+ headingOrderJson: text("heading_order_json"),
+ // Content
+ wordCount: integer("word_count").notNull().default(0),
+ // Images
+ imagesTotal: integer("images_total").notNull().default(0),
+ imagesMissingAlt: integer("images_missing_alt").notNull().default(0),
+ imagesJson: text("images_json"),
+ // Links
+ internalLinkCount: integer("internal_link_count").notNull().default(0),
+ externalLinkCount: integer("external_link_count").notNull().default(0),
+ // Structured data
+ hasStructuredData: integer("has_structured_data", { mode: "boolean" })
+ .notNull()
+ .default(false),
+ // Hreflang
+ hreflangTagsJson: text("hreflang_tags_json"),
+ // Indexability
+ isIndexable: integer("is_indexable", { mode: "boolean" })
+ .notNull()
+ .default(true),
+ // Indexability/canonical signals from response headers
+ xRobotsTag: text("x_robots_tag"),
+ headerCanonicalUrl: text("header_canonical_url"),
+ // Crawl metadata
+ // null depth = not reached via links (e.g. sitemap-seeded)
+ crawlDepth: integer("crawl_depth"),
+ inSitemap: integer("in_sitemap", { mode: "boolean" })
+ .notNull()
+ .default(false),
+ // SHA-256 of the visible body text, for duplicate-content grouping
+ contentHash: text("content_hash"),
+ // How the fetch resolved: ok | blocked (WAF/bot challenge) | error
+ fetchClass: text("fetch_class", { enum: ["ok", "blocked", "error"] })
+ .notNull()
+ .default("ok"),
+ // Performance
+ responseTimeMs: integer("response_time_ms"),
+ },
+ (table) => [
+ index("audit_pages_audit_id_idx").on(table.auditId),
+ index("audit_pages_audit_url_idx").on(table.auditId, table.url),
+ ],
+);
+
+// One row per unique (source page, target URL) link edge. Currently only
+// internal edges are stored (see AuditRepository); isInternal stays so
+// external-link checks can start writing rows without a migration.
+export const auditLinks = sqliteTable(
+ "audit_links",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ sourcePageId: text("source_page_id")
+ .notNull()
+ .references(() => auditPages.id, { onDelete: "cascade" }),
+ sourceUrl: text("source_url").notNull(),
+ targetUrl: text("target_url").notNull(),
+ anchor: text("anchor"),
+ isInternal: integer("is_internal", { mode: "boolean" })
+ .notNull()
+ .default(true),
+ isNofollow: integer("is_nofollow", { mode: "boolean" })
+ .notNull()
+ .default(false),
+ },
+ (table) => [
+ index("audit_links_audit_id_idx").on(table.auditId),
+ index("audit_links_audit_target_idx").on(table.auditId, table.targetUrl),
+ ],
+);
+
+// One row per (issue type, affected page)
+export const auditIssues = sqliteTable(
+ "audit_issues",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ pageId: text("page_id").references(() => auditPages.id, {
+ onDelete: "cascade",
+ }),
+ pageUrl: text("page_url").notNull(),
+ issueType: text("issue_type").notNull(),
+ severity: text("severity", { enum: ["critical", "warning", "info"] })
+ .notNull()
+ .default("info"),
+ // JSON details specific to the issue type (e.g. broken link target)
+ detailsJson: text("details_json"),
+ },
+ (table) => [
+ index("audit_issues_audit_id_idx").on(table.auditId),
+ index("audit_issues_audit_type_idx").on(table.auditId, table.issueType),
+ ],
+);
+
+// One row per Lighthouse test (mobile + desktop per page).
+export const auditLighthouseResults = sqliteTable(
+ "audit_lighthouse_results",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ pageId: text("page_id")
+ .notNull()
+ .references(() => auditPages.id, { onDelete: "cascade" }),
+ strategy: text("strategy", { enum: ["mobile", "desktop"] }).notNull(),
+ performanceScore: integer("performance_score"),
+ accessibilityScore: integer("accessibility_score"),
+ bestPracticesScore: integer("best_practices_score"),
+ seoScore: integer("seo_score"),
+ lcpMs: real("lcp_ms"),
+ cls: real("cls"),
+ inpMs: real("inp_ms"),
+ ttfbMs: real("ttfb_ms"),
+ errorMessage: text("error_message"),
+ r2Key: text("r2_key"),
+ payloadSizeBytes: integer("payload_size_bytes"),
+ },
+ (table) => [index("audit_lighthouse_results_audit_id_idx").on(table.auditId)],
+);
diff --git a/src/db/d1/schema.ts b/src/db/d1/schema.ts
index 74d00809..63ef60b0 100644
--- a/src/db/d1/schema.ts
+++ b/src/db/d1/schema.ts
@@ -2,6 +2,7 @@
// which is the provider-aware barrel) so the D1 client always binds to the
// SQLite tables regardless of DATABASE_PROVIDER.
export * from "../app.schema";
+export * from "../audit.schema";
export * from "../sam.schema";
export * from "../better-auth-schema";
export * from "../billing.schema";
diff --git a/src/db/pg/app.schema.ts b/src/db/pg/app.schema.ts
index ff427d48..b75fe5be 100644
--- a/src/db/pg/app.schema.ts
+++ b/src/db/pg/app.schema.ts
@@ -338,112 +338,3 @@ export const rankSnapshots = pgTable(
),
],
);
-
-// ============================================================================
-// Site Audit tables
-// ============================================================================
-
-// One row per audit run
-export const audits = pgTable(
- "audits",
- {
- id: text("id").primaryKey(),
- projectId: text("project_id")
- .notNull()
- .references(() => projects.id, { onDelete: "cascade" }),
- startedByUserId: text("started_by_user_id").notNull(),
- startUrl: text("start_url").notNull(),
- status: text("status", {
- enum: ["running", "completed", "failed"],
- })
- .notNull()
- .default("running"),
- workflowInstanceId: text("workflow_instance_id"),
- // JSON config: { maxPages, lighthouseStrategy }
- config: text("config").notNull().default("{}"),
- // Progress & summary
- pagesCrawled: integer("pages_crawled").notNull().default(0),
- pagesTotal: integer("pages_total").notNull().default(0),
- lighthouseTotal: integer("lighthouse_total").notNull().default(0),
- lighthouseCompleted: integer("lighthouse_completed").notNull().default(0),
- lighthouseFailed: integer("lighthouse_failed").notNull().default(0),
- currentPhase: text("current_phase").default("discovery"),
- startedAt: timestampColumn("started_at").notNull().default(isoNow),
- completedAt: timestampColumn("completed_at"),
- },
- (table) => [
- index("audits_project_id_idx").on(table.projectId),
- index("audits_started_by_user_id_idx").on(table.startedByUserId),
- ],
-);
-
-// One row per crawled page
-export const auditPages = pgTable(
- "audit_pages",
- {
- id: text("id").primaryKey(),
- auditId: text("audit_id")
- .notNull()
- .references(() => audits.id, { onDelete: "cascade" }),
- url: text("url").notNull(),
- statusCode: integer("status_code"),
- redirectUrl: text("redirect_url"),
- // Metadata
- title: text("title"),
- metaDescription: text("meta_description"),
- canonicalUrl: text("canonical_url"),
- robotsMeta: text("robots_meta"),
- // Open Graph
- ogTitle: text("og_title"),
- ogDescription: text("og_description"),
- ogImage: text("og_image"),
- // Headings
- h1Count: integer("h1_count").notNull().default(0),
- h2Count: integer("h2_count").notNull().default(0),
- h3Count: integer("h3_count").notNull().default(0),
- h4Count: integer("h4_count").notNull().default(0),
- h5Count: integer("h5_count").notNull().default(0),
- h6Count: integer("h6_count").notNull().default(0),
- headingOrderJson: text("heading_order_json"),
- // Content
- wordCount: integer("word_count").notNull().default(0),
- // Images
- imagesTotal: integer("images_total").notNull().default(0),
- imagesMissingAlt: integer("images_missing_alt").notNull().default(0),
- imagesJson: text("images_json"),
- internalLinkCount: integer("internal_link_count").notNull().default(0),
- externalLinkCount: integer("external_link_count").notNull().default(0),
- hasStructuredData: boolean("has_structured_data").notNull().default(false),
- hreflangTagsJson: text("hreflang_tags_json"),
- isIndexable: boolean("is_indexable").notNull().default(true),
- responseTimeMs: integer("response_time_ms"),
- },
- (table) => [index("audit_pages_audit_id_idx").on(table.auditId)],
-);
-
-// One row per Lighthouse test (mobile + desktop per page).
-export const auditLighthouseResults = pgTable(
- "audit_lighthouse_results",
- {
- id: text("id").primaryKey(),
- auditId: text("audit_id")
- .notNull()
- .references(() => audits.id, { onDelete: "cascade" }),
- pageId: text("page_id")
- .notNull()
- .references(() => auditPages.id, { onDelete: "cascade" }),
- strategy: text("strategy", { enum: ["mobile", "desktop"] }).notNull(),
- performanceScore: integer("performance_score"),
- accessibilityScore: integer("accessibility_score"),
- bestPracticesScore: integer("best_practices_score"),
- seoScore: integer("seo_score"),
- lcpMs: real("lcp_ms"),
- cls: real("cls"),
- inpMs: real("inp_ms"),
- ttfbMs: real("ttfb_ms"),
- errorMessage: text("error_message"),
- r2Key: text("r2_key"),
- payloadSizeBytes: integer("payload_size_bytes"),
- },
- (table) => [index("audit_lighthouse_results_audit_id_idx").on(table.auditId)],
-);
diff --git a/src/db/pg/audit.schema.ts b/src/db/pg/audit.schema.ts
new file mode 100644
index 00000000..5f57113c
--- /dev/null
+++ b/src/db/pg/audit.schema.ts
@@ -0,0 +1,196 @@
+import { sql } from "drizzle-orm";
+import {
+ boolean,
+ index,
+ integer,
+ pgTable,
+ real,
+ text,
+} from "drizzle-orm/pg-core";
+import { projects } from "./app.schema";
+
+// Timestamps are stored as *text* (same column shape as the SQLite schema); see
+// the note in pg/app.schema.ts. `isoNow` matches `new Date().toISOString()` so
+// DB-defaulted and app-written values sort together lexicographically.
+const isoNow = sql`to_char(now() AT TIME ZONE 'utc', 'YYYY-MM-DD"T"HH24:MI:SS.MS"Z"')`;
+const timestampColumn = (name: string) => text(name);
+
+// ============================================================================
+// Site Audit tables
+// ============================================================================
+
+// One row per audit run
+export const audits = pgTable(
+ "audits",
+ {
+ id: text("id").primaryKey(),
+ projectId: text("project_id")
+ .notNull()
+ .references(() => projects.id, { onDelete: "cascade" }),
+ startedByUserId: text("started_by_user_id").notNull(),
+ startUrl: text("start_url").notNull(),
+ status: text("status", {
+ enum: ["running", "completed", "failed"],
+ })
+ .notNull()
+ .default("running"),
+ workflowInstanceId: text("workflow_instance_id"),
+ // JSON config: { maxPages, lighthouseStrategy }
+ config: text("config").notNull().default("{}"),
+ // Progress & summary
+ pagesCrawled: integer("pages_crawled").notNull().default(0),
+ pagesTotal: integer("pages_total").notNull().default(0),
+ lighthouseTotal: integer("lighthouse_total").notNull().default(0),
+ lighthouseCompleted: integer("lighthouse_completed").notNull().default(0),
+ lighthouseFailed: integer("lighthouse_failed").notNull().default(0),
+ currentPhase: text("current_phase").default("discovery"),
+ startedAt: timestampColumn("started_at").notNull().default(isoNow),
+ completedAt: timestampColumn("completed_at"),
+ },
+ (table) => [
+ index("audits_project_id_idx").on(table.projectId),
+ index("audits_started_by_user_id_idx").on(table.startedByUserId),
+ ],
+);
+
+// One row per crawled page
+export const auditPages = pgTable(
+ "audit_pages",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ url: text("url").notNull(),
+ statusCode: integer("status_code"),
+ redirectUrl: text("redirect_url"),
+ // Metadata
+ title: text("title"),
+ metaDescription: text("meta_description"),
+ canonicalUrl: text("canonical_url"),
+ robotsMeta: text("robots_meta"),
+ // Open Graph
+ ogTitle: text("og_title"),
+ ogDescription: text("og_description"),
+ ogImage: text("og_image"),
+ // Headings
+ h1Count: integer("h1_count").notNull().default(0),
+ h2Count: integer("h2_count").notNull().default(0),
+ h3Count: integer("h3_count").notNull().default(0),
+ h4Count: integer("h4_count").notNull().default(0),
+ h5Count: integer("h5_count").notNull().default(0),
+ h6Count: integer("h6_count").notNull().default(0),
+ headingOrderJson: text("heading_order_json"),
+ // Content
+ wordCount: integer("word_count").notNull().default(0),
+ // Images
+ imagesTotal: integer("images_total").notNull().default(0),
+ imagesMissingAlt: integer("images_missing_alt").notNull().default(0),
+ imagesJson: text("images_json"),
+ // Links
+ internalLinkCount: integer("internal_link_count").notNull().default(0),
+ externalLinkCount: integer("external_link_count").notNull().default(0),
+ // Structured data
+ hasStructuredData: boolean("has_structured_data").notNull().default(false),
+ // Hreflang
+ hreflangTagsJson: text("hreflang_tags_json"),
+ // Indexability
+ isIndexable: boolean("is_indexable").notNull().default(true),
+ // Indexability/canonical signals from response headers
+ xRobotsTag: text("x_robots_tag"),
+ headerCanonicalUrl: text("header_canonical_url"),
+ // Crawl metadata
+ // null depth = not reached via links (e.g. sitemap-seeded)
+ crawlDepth: integer("crawl_depth"),
+ inSitemap: boolean("in_sitemap").notNull().default(false),
+ // SHA-256 of the visible body text, for duplicate-content grouping
+ contentHash: text("content_hash"),
+ // How the fetch resolved: ok | blocked (WAF/bot challenge) | error
+ fetchClass: text("fetch_class", { enum: ["ok", "blocked", "error"] })
+ .notNull()
+ .default("ok"),
+ // Performance
+ responseTimeMs: integer("response_time_ms"),
+ },
+ (table) => [
+ index("audit_pages_audit_id_idx").on(table.auditId),
+ index("audit_pages_audit_url_idx").on(table.auditId, table.url),
+ ],
+);
+
+// One row per unique (source page, target URL) link edge. Currently only
+// internal edges are stored (see AuditRepository); isInternal stays so
+// external-link checks can start writing rows without a migration.
+export const auditLinks = pgTable(
+ "audit_links",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ sourcePageId: text("source_page_id")
+ .notNull()
+ .references(() => auditPages.id, { onDelete: "cascade" }),
+ sourceUrl: text("source_url").notNull(),
+ targetUrl: text("target_url").notNull(),
+ anchor: text("anchor"),
+ isInternal: boolean("is_internal").notNull().default(true),
+ isNofollow: boolean("is_nofollow").notNull().default(false),
+ },
+ (table) => [
+ index("audit_links_audit_id_idx").on(table.auditId),
+ index("audit_links_audit_target_idx").on(table.auditId, table.targetUrl),
+ ],
+);
+
+// One row per (issue type, affected page)
+export const auditIssues = pgTable(
+ "audit_issues",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ pageId: text("page_id").references(() => auditPages.id, {
+ onDelete: "cascade",
+ }),
+ pageUrl: text("page_url").notNull(),
+ issueType: text("issue_type").notNull(),
+ severity: text("severity", { enum: ["critical", "warning", "info"] })
+ .notNull()
+ .default("info"),
+ // JSON details specific to the issue type (e.g. broken link target)
+ detailsJson: text("details_json"),
+ },
+ (table) => [
+ index("audit_issues_audit_id_idx").on(table.auditId),
+ index("audit_issues_audit_type_idx").on(table.auditId, table.issueType),
+ ],
+);
+
+// One row per Lighthouse test (mobile + desktop per page).
+export const auditLighthouseResults = pgTable(
+ "audit_lighthouse_results",
+ {
+ id: text("id").primaryKey(),
+ auditId: text("audit_id")
+ .notNull()
+ .references(() => audits.id, { onDelete: "cascade" }),
+ pageId: text("page_id")
+ .notNull()
+ .references(() => auditPages.id, { onDelete: "cascade" }),
+ strategy: text("strategy", { enum: ["mobile", "desktop"] }).notNull(),
+ performanceScore: integer("performance_score"),
+ accessibilityScore: integer("accessibility_score"),
+ bestPracticesScore: integer("best_practices_score"),
+ seoScore: integer("seo_score"),
+ lcpMs: real("lcp_ms"),
+ cls: real("cls"),
+ inpMs: real("inp_ms"),
+ ttfbMs: real("ttfb_ms"),
+ errorMessage: text("error_message"),
+ r2Key: text("r2_key"),
+ payloadSizeBytes: integer("payload_size_bytes"),
+ },
+ (table) => [index("audit_lighthouse_results_audit_id_idx").on(table.auditId)],
+);
diff --git a/src/db/pg/schema.ts b/src/db/pg/schema.ts
index 9d03f35f..db15b8c8 100644
--- a/src/db/pg/schema.ts
+++ b/src/db/pg/schema.ts
@@ -1,4 +1,5 @@
export * from "./app.schema";
+export * from "./audit.schema";
export * from "./sam.schema";
export * from "./better-auth-schema";
export * from "./billing.schema";
diff --git a/src/db/schema.ts b/src/db/schema.ts
index 75f40d33..2e21d2d4 100644
--- a/src/db/schema.ts
+++ b/src/db/schema.ts
@@ -1,11 +1,13 @@
import { getDatabaseProvider } from "./provider";
import * as sqliteApp from "./app.schema";
+import * as sqliteAudit from "./audit.schema";
import * as sqliteSam from "./sam.schema";
import * as sqliteAuth from "./better-auth-schema";
import * as sqliteBilling from "./billing.schema";
import * as sqliteGsc from "./gsc.schema";
import * as sqliteReddit from "./reddit-attribution.schema";
import * as pgApp from "./pg/app.schema";
+import * as pgAudit from "./pg/audit.schema";
import * as pgSam from "./pg/sam.schema";
import * as pgAuth from "./pg/better-auth-schema";
import * as pgBilling from "./pg/billing.schema";
@@ -23,6 +25,7 @@ import * as pgReddit from "./pg/reddit-attribution.schema";
// schema is the one structural artifact NOT regenerated by `db:generate`, so the
// parity test is its drift guard.
type AppSchema = typeof sqliteApp &
+ typeof sqliteAudit &
typeof sqliteSam &
typeof sqliteAuth &
typeof sqliteBilling &
@@ -31,9 +34,18 @@ type AppSchema = typeof sqliteApp &
const runtimeSchema =
getDatabaseProvider() === "postgres"
- ? { ...pgApp, ...pgSam, ...pgAuth, ...pgBilling, ...pgGsc, ...pgReddit }
+ ? {
+ ...pgApp,
+ ...pgAudit,
+ ...pgSam,
+ ...pgAuth,
+ ...pgBilling,
+ ...pgGsc,
+ ...pgReddit,
+ }
: {
...sqliteApp,
+ ...sqliteAudit,
...sqliteSam,
...sqliteAuth,
...sqliteBilling,
@@ -57,6 +69,8 @@ export const {
rankSnapshots,
audits,
auditPages,
+ auditLinks,
+ auditIssues,
auditLighthouseResults,
samSessions,
samProjectMemory,
diff --git a/src/server/features/audit/repositories/AuditRepository.ts b/src/server/features/audit/repositories/AuditRepository.ts
index a01a0751..9fa0ad0f 100644
--- a/src/server/features/audit/repositories/AuditRepository.ts
+++ b/src/server/features/audit/repositories/AuditRepository.ts
@@ -1,17 +1,34 @@
/**
* Data access layer for site audit tables.
- * Provider-aware (D1 or Postgres) via the `@/db` handle.
+ * Provider-aware (D1 or Postgres) via the `@/db` handle. Covers audits,
+ * audit_pages, audit_links, audit_issues, and stored Lighthouse results.
*/
import { and, desc, eq } from "drizzle-orm";
import { db } from "@/db";
-import { audits, auditLighthouseResults, auditPages } from "@/db/schema";
+import {
+ audits,
+ auditIssues,
+ auditLighthouseResults,
+ auditLinks,
+ auditPages,
+} from "@/db/schema";
import { executeInBatches } from "@/db/runBatch";
+import { AUDIT_ISSUE_TYPES } from "@/shared/audit-issues";
+import { deterministicAuditRowId } from "@/server/lib/audit/ids";
+import type { DetectedIssue } from "@/server/lib/audit/issues/page-reporters";
import type {
AuditConfig,
+ CrawledPageResult,
LighthouseResult,
- StepPageResult,
} from "@/server/lib/audit/types";
+// Only internal links are stored: both consumers (broken-internal-link and
+// orphan checks) filter on isInternal, and per-page external counts already
+// live on audit_pages. External rows come back when P1 adds external-link
+// checks. Mega-menu/footer-heavy sites can carry 1000+ links per page; cap
+// what we store so a 10k-page crawl can't write tens of millions of link rows.
+const MAX_STORED_LINKS_PER_PAGE = 500;
+
async function createAudit(data: {
id: string;
projectId: string;
@@ -84,6 +101,9 @@ async function completeAudit(
}
async function failAudit(auditId: string, workflowInstanceId: string) {
+ // Only a running audit can transition to failed: the getStatus reconciler
+ // races the workflow's own finalize, and without this guard it could flip
+ // a just-completed audit to failed.
await db
.update(audits)
.set({
@@ -95,6 +115,7 @@ async function failAudit(auditId: string, workflowInstanceId: string) {
and(
eq(audits.id, auditId),
eq(audits.workflowInstanceId, workflowInstanceId),
+ eq(audits.status, "running"),
),
);
}
@@ -111,25 +132,24 @@ async function getAuditForWorkflow(
});
}
-async function batchWriteResults(
+/**
+ * Persist one crawl batch (pages + link edges + per-page issues).
+ * Called inside the crawl-batch Workflow step so results land in D1
+ * incrementally instead of accumulating in memory until finalize.
+ *
+ * Idempotent on step retry: callers assign deterministic page ids
+ * (deterministicAuditRowId) and link/issue ids are derived from stable
+ * content. Page rows upsert (a retried fetch may legitimately differ — last
+ * attempt wins, matching what the step returns); links and issues are
+ * insert-or-ignore.
+ */
+async function insertCrawledBatch(
auditId: string,
- pages: StepPageResult[],
- lighthouseResults: LighthouseResult[],
+ pages: CrawledPageResult[],
+ issues: DetectedIssue[],
) {
- // The `finalize` workflow step can retry after a partial write (multi-chunk
- // inserts aren't atomic, and steps after the insert can throw). Clear any
- // rows from a prior attempt first so the re-run is idempotent — otherwise
- // stable page ids collide on the PK and lighthouse rows silently duplicate.
- // audit_lighthouse_results.page_id FKs audit_pages, so delete it first.
- await db
- .delete(auditLighthouseResults)
- .where(eq(auditLighthouseResults.auditId, auditId));
- await db.delete(auditPages).where(eq(auditPages.auditId, auditId));
-
- await executeInBatches(pages, (tx, page) =>
- tx.insert(auditPages).values({
- id: page.id,
- auditId,
+ await executeInBatches(pages, (tx, page) => {
+ const dataColumns = {
url: page.url,
statusCode: page.statusCode,
redirectUrl: page.redirectUrl,
@@ -137,6 +157,8 @@ async function batchWriteResults(
metaDescription: page.metaDescription,
canonicalUrl: page.canonicalUrl,
robotsMeta: page.robotsMeta,
+ xRobotsTag: page.xRobotsTag,
+ headerCanonicalUrl: page.headerCanonicalUrl,
ogTitle: page.ogTitle,
ogDescription: page.ogDescription,
ogImage: page.ogImage,
@@ -148,25 +170,83 @@ async function batchWriteResults(
h6Count: page.h6Count,
headingOrderJson: JSON.stringify(page.headingOrder),
wordCount: page.wordCount,
+ contentHash: page.contentHash,
imagesTotal: page.imagesTotal,
imagesMissingAlt: page.imagesMissingAlt,
imagesJson: JSON.stringify(page.images),
- internalLinkCount: page.internalLinks.length,
- externalLinkCount: page.externalLinks.length,
+ internalLinkCount: page.links.filter((l) => l.isInternal).length,
+ externalLinkCount: page.links.filter((l) => !l.isInternal).length,
hasStructuredData: page.hasStructuredData,
hreflangTagsJson: JSON.stringify(page.hreflangTags),
isIndexable: page.isIndexable,
+ fetchClass: page.fetchClass,
+ crawlDepth: page.crawlDepth,
+ inSitemap: page.inSitemap,
responseTimeMs: page.responseTimeMs,
- }),
+ };
+ return tx
+ .insert(auditPages)
+ .values({ id: page.id, auditId, ...dataColumns })
+ .onConflictDoUpdate({ target: auditPages.id, set: dataColumns });
+ });
+
+ const linkRows = await Promise.all(
+ pages.flatMap((page) =>
+ page.links
+ .filter((link) => link.isInternal)
+ .slice(0, MAX_STORED_LINKS_PER_PAGE)
+ .map(async (link) => ({
+ id: await deterministicAuditRowId(auditId, page.url, link.targetUrl),
+ auditId,
+ sourcePageId: page.id,
+ sourceUrl: page.url,
+ targetUrl: link.targetUrl,
+ anchor: link.anchor,
+ isInternal: link.isInternal,
+ isNofollow: link.isNofollow,
+ })),
+ ),
+ );
+ await executeInBatches(linkRows, (tx, row) =>
+ tx.insert(auditLinks).values(row).onConflictDoNothing(),
);
- if (lighthouseResults.length === 0) {
- return;
- }
+ await insertIssues(auditId, issues);
+}
- await executeInBatches(lighthouseResults, (tx, result) =>
- tx.insert(auditLighthouseResults).values({
- id: crypto.randomUUID(),
+async function insertIssues(auditId: string, issues: DetectedIssue[]) {
+ const issueRows = await Promise.all(
+ issues.map(async (issue) => ({
+ id: await deterministicAuditRowId(
+ auditId,
+ issue.pageUrl,
+ issue.issueType,
+ issue.dedupeKey ?? "",
+ ),
+ auditId,
+ pageId: issue.pageId,
+ pageUrl: issue.pageUrl,
+ issueType: issue.issueType,
+ severity: AUDIT_ISSUE_TYPES[issue.issueType].severity,
+ detailsJson: issue.details ? JSON.stringify(issue.details) : null,
+ })),
+ );
+ await executeInBatches(issueRows, (tx, row) =>
+ tx.insert(auditIssues).values(row).onConflictDoNothing(),
+ );
+}
+
+async function insertLighthouseResults(
+ auditId: string,
+ lighthouseResults: LighthouseResult[],
+) {
+ const rows = await Promise.all(
+ lighthouseResults.map(async (result) => ({
+ id: await deterministicAuditRowId(
+ auditId,
+ result.pageId,
+ result.strategy,
+ ),
auditId,
pageId: result.pageId,
strategy: result.strategy,
@@ -181,8 +261,17 @@ async function batchWriteResults(
errorMessage: result.errorMessage ?? null,
r2Key: result.r2Key ?? null,
payloadSizeBytes: result.payloadSizeBytes ?? null,
- }),
+ })),
);
+ // Upsert: a step retry can charge a second DataForSEO call whose result
+ // must not be silently dropped in favor of a failed first attempt.
+ await executeInBatches(rows, (tx, row) => {
+ const { id: _id, auditId: _auditId, ...dataColumns } = row;
+ return tx.insert(auditLighthouseResults).values(row).onConflictDoUpdate({
+ target: auditLighthouseResults.id,
+ set: dataColumns,
+ });
+ });
}
async function getAuditForProject(auditId: string, projectId: string) {
@@ -191,6 +280,58 @@ async function getAuditForProject(auditId: string, projectId: string) {
});
}
+async function getLatestAuditForProject(projectId: string) {
+ return db.query.audits.findFirst({
+ where: eq(audits.projectId, projectId),
+ orderBy: desc(audits.startedAt),
+ });
+}
+
+async function getIssuesForAudit(
+ auditId: string,
+ filters: { severity?: "critical" | "warning" | "info"; issueType?: string },
+) {
+ return db.query.auditIssues.findMany({
+ where: and(
+ eq(auditIssues.auditId, auditId),
+ filters.severity ? eq(auditIssues.severity, filters.severity) : undefined,
+ filters.issueType
+ ? eq(auditIssues.issueType, filters.issueType)
+ : undefined,
+ ),
+ });
+}
+
+async function getPagesForAudit(auditId: string) {
+ return db
+ .select({
+ id: auditPages.id,
+ url: auditPages.url,
+ statusCode: auditPages.statusCode,
+ fetchClass: auditPages.fetchClass,
+ redirectUrl: auditPages.redirectUrl,
+ title: auditPages.title,
+ metaDescription: auditPages.metaDescription,
+ wordCount: auditPages.wordCount,
+ isIndexable: auditPages.isIndexable,
+ crawlDepth: auditPages.crawlDepth,
+ inSitemap: auditPages.inSitemap,
+ internalLinkCount: auditPages.internalLinkCount,
+ responseTimeMs: auditPages.responseTimeMs,
+ })
+ .from(auditPages)
+ .where(eq(auditPages.auditId, auditId));
+}
+
+async function hasPagesForAudit(auditId: string): Promise<boolean> {
+ const rows = await db
+ .select({ id: auditPages.id })
+ .from(auditPages)
+ .where(eq(auditPages.auditId, auditId))
+ .limit(1);
+ return rows.length > 0;
+}
+
async function getAuditsByProject(projectId: string) {
const rows = await db
.select({ audit: audits })
@@ -223,19 +364,22 @@ async function getAuditUsageForUser(userId: string) {
async function getAuditResultsForProject(auditId: string, projectId: string) {
const audit = await getAuditForProject(auditId, projectId);
if (!audit) {
- return { audit: null, pages: [], lighthouse: [] };
+ return { audit: null, pages: [], lighthouse: [], issues: [] };
}
- const [pages, lighthouse] = await Promise.all([
+ const [pages, lighthouse, issues] = await Promise.all([
db.query.auditPages.findMany({
where: eq(auditPages.auditId, auditId),
}),
db.query.auditLighthouseResults.findMany({
where: eq(auditLighthouseResults.auditId, auditId),
}),
+ db.query.auditIssues.findMany({
+ where: eq(auditIssues.auditId, auditId),
+ }),
]);
- return { audit, pages, lighthouse };
+ return { audit, pages, lighthouse, issues };
}
async function getLighthouseResultById(input: {
@@ -285,8 +429,14 @@ export const AuditRepository = {
completeAudit,
failAudit,
getAuditForWorkflow,
- batchWriteResults,
+ insertCrawledBatch,
+ insertIssues,
+ insertLighthouseResults,
getAuditForProject,
+ getLatestAuditForProject,
+ getIssuesForAudit,
+ getPagesForAudit,
+ hasPagesForAudit,
getAuditsByProject,
getAuditUsageForUser,
getAuditResultsForProject,
diff --git a/src/server/features/audit/services/AuditService.ts b/src/server/features/audit/services/AuditService.ts
index d0ff340b..f36146c9 100644
--- a/src/server/features/audit/services/AuditService.ts
+++ b/src/server/features/audit/services/AuditService.ts
@@ -1,5 +1,9 @@
import { env } from "cloudflare:workers";
-import type { BillingCustomerContext } from "@/server/billing/subscription";
+import {
+ customerHasManagedAccess,
+ customerHasPaidPlan,
+ type BillingCustomerContext,
+} from "@/server/billing/subscription";
import { AuditRepository } from "@/server/features/audit/repositories/AuditRepository";
import {
AUDIT_LIMITS,
@@ -15,6 +19,24 @@ import {
type LighthouseStrategy,
} from "@/server/lib/audit/types";
import { normalizeAndValidateStartUrl } from "@/server/lib/audit/url-policy";
+import { isHostedServerAuthMode } from "@/server/lib/runtime-env";
+
+// Plan-tier limits are the abuse bound in hosted mode: free accounts get one
+// small audit at a time, paid keeps the full limits, and customers with no
+// Autumn product at all are turned away. Self-hosted isn't gated.
+async function resolveAuditLimitTier(
+ organizationId: string,
+): Promise<AuditLimitTier> {
+ if (!(await isHostedServerAuthMode())) return "paid";
+ const [hasManagedAccess, hasPaidPlan] = await Promise.all([
+ customerHasManagedAccess(organizationId),
+ customerHasPaidPlan(organizationId),
+ ]);
+ if (!hasManagedAccess) {
+ throw new AppError("PAYMENT_REQUIRED", "Subscribe to run site audits");
+ }
+ return hasPaidPlan ? "paid" : "free";
+}
async function startAudit(input: {
actorUserId: string;
@@ -98,8 +120,29 @@ async function startAudit(input: {
}
async function getStatus(auditId: string, projectId: string) {
- const audit = await AuditRepository.getAuditForProject(auditId, projectId);
- if (!audit) throw new AppError("NOT_FOUND");
+ let audit = await AuditRepository.getAuditForProject(auditId, projectId);
+ if (!audit)
+ throw new AppError("NOT_FOUND", "Audit not found in this project.");
+
+ // Self-heal audits whose workflow died without reaching the mark-failed
+ // step (instance terminated, mark-failed itself failed, deploys, ...).
+ // Without this they stay "running" forever and hold capacity.
+ if (audit.status === "running" && audit.workflowInstanceId) {
+ try {
+ const instance = await env.SITE_AUDIT_WORKFLOW.get(
+ audit.workflowInstanceId,
+ );
+ const { status } = await instance.status();
+ if (status === "errored" || status === "terminated") {
+ await AuditRepository.failAudit(audit.id, audit.workflowInstanceId);
+ audit =
+ (await AuditRepository.getAuditForProject(auditId, projectId)) ??
+ audit;
+ }
+ } catch {
+ // Instance not found or status unavailable — leave the audit as-is.
+ }
+ }
return {
id: audit.id,
@@ -117,7 +160,7 @@ async function getStatus(auditId: string, projectId: string) {
}
async function getResults(auditId: string, projectId: string) {
- const { audit, pages, lighthouse } =
+ const { audit, pages, lighthouse, issues } =
await AuditRepository.getAuditResultsForProject(auditId, projectId);
if (!audit) throw new AppError("NOT_FOUND");
@@ -140,6 +183,7 @@ async function getResults(auditId: string, projectId: string) {
},
pages,
lighthouse,
+ issues,
};
}
@@ -216,6 +260,7 @@ async function remove(auditId: string, projectId: string) {
}
export const AuditService = {
+ resolveAuditLimitTier,
startAudit,
getStatus,
getCrawlProgress,
diff --git a/src/server/features/audit/services/audit-capacity.test.ts b/src/server/features/audit/services/audit-capacity.test.ts
index adc64327..238991c7 100644
--- a/src/server/features/audit/services/audit-capacity.test.ts
+++ b/src/server/features/audit/services/audit-capacity.test.ts
@@ -21,16 +21,6 @@ describe("audit capacity helpers", () => {
lighthouseTotal: 0,
total: 100,
});
- expect(
- getEstimatedAuditCapacity({
- maxPages: 100,
- lighthouseStrategy: "manual",
- }),
- ).toEqual({
- pagesTotal: 100,
- lighthouseTotal: 0,
- total: 100,
- });
expect(
getEstimatedAuditCapacity({ maxPages: 100, lighthouseStrategy: "auto" }),
).toEqual({
diff --git a/src/server/lib/audit/discovery.ts b/src/server/lib/audit/discovery.ts
index f37b0a44..68bd387b 100644
--- a/src/server/lib/audit/discovery.ts
+++ b/src/server/lib/audit/discovery.ts
@@ -22,39 +22,39 @@ export interface RobotsResult {
}
/**
- * Fetch and parse robots.txt for a given origin.
- * Returns a helper to check if URLs are allowed + discovered sitemap URLs.
+ * Fetch the raw robots.txt body (null = missing/unreachable). Kept separate
+ * from parsing so Workflows can checkpoint the text as durable step state and
+ * re-derive the parsed result deterministically on replay.
*/
-export async function fetchRobotsTxt(origin: string): Promise<RobotsResult> {
- const robotsUrl = `${origin}/robots.txt`;
+async function fetchRobotsTxtText(origin: string): Promise<string | null> {
try {
- const response = await fetch(robotsUrl, {
+ const response = await fetch(`${origin}/robots.txt`, {
headers: { "User-Agent": "OpenSEO-Audit/1.0" },
signal: AbortSignal.timeout(10_000),
});
- if (!response.ok) {
- // No robots.txt = everything allowed
- return {
- isAllowed: () => true,
- sitemapUrls: [],
- };
- }
-
- const text = await response.text();
- const robots = robotsParser(robotsUrl, text);
-
- return {
- isAllowed: (url: string) => robots.isAllowed(url) ?? true,
- sitemapUrls: robots.getSitemaps(),
- };
+ if (!response.ok) return null;
+ return await response.text();
} catch (error) {
console.warn("Failed to fetch robots.txt:", error);
- return {
- isAllowed: () => true,
- sitemapUrls: [],
- };
+ return null;
+ }
+}
+
+/** Deterministic: same text in, same result out. Null = everything allowed. */
+export function parseRobotsTxt(
+ origin: string,
+ text: string | null,
+): RobotsResult {
+ if (text === null) {
+ return { isAllowed: () => true, sitemapUrls: [] };
}
+
+ const robots = robotsParser(`${origin}/robots.txt`, text);
+ return {
+ isAllowed: (url: string) => robots.isAllowed(url) ?? true,
+ sitemapUrls: robots.getSitemaps(),
+ };
}
/**
@@ -184,8 +184,9 @@ async function fetchSitemapDocumentWithRetry(sitemapUrl: string): Promise<{
export async function discoverUrls(
origin: string,
maxPages = 50,
-): Promise<{ urls: string[]; robots: RobotsResult; sitemapUrls: Set<string> }> {
- const robots = await fetchRobotsTxt(origin);
+): Promise<{ urls: string[]; robotsText: string | null }> {
+ const robotsText = await fetchRobotsTxtText(origin);
+ const robots = parseRobotsTxt(origin, robotsText);
// Collect sitemap URLs: from robots.txt + default location
const sitemapSources = new Set(robots.sitemapUrls);
@@ -262,9 +263,10 @@ export async function discoverUrls(
);
}
+ // Cap at the crawl's page budget: these are seeds, the crawl can never use
+ // more — and an uncapped list can blow the ~1MiB Workflow step-state limit.
return {
- urls: Array.from(allUrls),
- robots,
- sitemapUrls: allUrls,
+ urls: Array.from(allUrls).slice(0, maxPages),
+ robotsText,
};
}
diff --git a/src/server/lib/audit/ids.ts b/src/server/lib/audit/ids.ts
new file mode 100644
index 00000000..6dc30f27
--- /dev/null
+++ b/src/server/lib/audit/ids.ts
@@ -0,0 +1,24 @@
+/**
+ * Deterministic row ids for audit data.
+ *
+ * Crawl/lighthouse persistence happens inside Workflow steps, which retry on
+ * failure. Deriving ids from stable content (audit id + URL + ...) combined
+ * with `onConflictDoNothing` makes those writes idempotent across retries —
+ * a partially-written batch is simply completed on the next attempt instead
+ * of duplicated under fresh random ids.
+ */
+export async function sha256Hex(text: string): Promise<string> {
+ const digest = await crypto.subtle.digest(
+ "SHA-256",
+ new TextEncoder().encode(text),
+ );
+ return Array.from(new Uint8Array(digest))
+ .map((byte) => byte.toString(16).padStart(2, "0"))
+ .join("");
+}
+
+export async function deterministicAuditRowId(
+ ...parts: string[]
+): Promise<string> {
+ return (await sha256Hex(parts.join("|"))).slice(0, 36);
+}
diff --git a/src/server/lib/audit/issues/multipage-checks.ts b/src/server/lib/audit/issues/multipage-checks.ts
new file mode 100644
index 00000000..5858db5e
--- /dev/null
+++ b/src/server/lib/audit/issues/multipage-checks.ts
@@ -0,0 +1,180 @@
+/**
+ * Pure cross-page checks (no database access): duplicate grouping and
+ * redirect chain/loop detection. The D1-backed checks (broken links,
+ * orphans) live in multipage.ts.
+ */
+import type { DetectedIssue } from "@/server/lib/audit/issues/page-reporters";
+
+const DUPLICATE_GROUP_SAMPLE = 3;
+
+export interface SlimPage {
+ id: string;
+ url: string;
+ statusCode: number | null;
+ fetchClass: "ok" | "blocked" | "error";
+ title: string | null;
+ metaDescription: string | null;
+ contentHash: string | null;
+ redirectUrl: string | null;
+ wordCount: number;
+ isIndexable: boolean;
+ canonicalUrl: string | null;
+ headerCanonicalUrl: string | null;
+}
+
+function isOkHtmlPage(page: SlimPage): boolean {
+ return (
+ page.fetchClass === "ok" &&
+ page.statusCode !== null &&
+ page.statusCode >= 200 &&
+ page.statusCode < 300
+ );
+}
+
+/**
+ * Pages the owner already de-duplicated (noindex, or canonicalized to
+ * another URL) don't belong in duplicate groups — flagging them tells the
+ * user to fix something they already fixed.
+ */
+function isDuplicateCandidate(page: SlimPage): boolean {
+ if (!isOkHtmlPage(page) || !page.isIndexable) return false;
+ const effectiveCanonical = page.canonicalUrl ?? page.headerCanonicalUrl;
+ return !effectiveCanonical || effectiveCanonical === page.url;
+}
+
+export function findDuplicates(pages: SlimPage[]): DetectedIssue[] {
+ const okPages = pages.filter(isDuplicateCandidate);
+
+ const groupBy = (
+ keyOf: (page: SlimPage) => string | null,
+ ): Map<string, SlimPage[]> => {
+ const groups = new Map<string, SlimPage[]>();
+ for (const page of okPages) {
+ const key = keyOf(page);
+ if (!key) continue;
+ const group = groups.get(key);
+ if (group) group.push(page);
+ else groups.set(key, [page]);
+ }
+ return groups;
+ };
+
+ const issues: DetectedIssue[] = [];
+ const emitGroups = (
+ groups: Map<string, SlimPage[]>,
+ issueType: DetectedIssue["issueType"],
+ ) => {
+ for (const group of groups.values()) {
+ if (group.length < 2) continue;
+ for (const page of group) {
+ issues.push({
+ issueType,
+ pageId: page.id,
+ pageUrl: page.url,
+ details: {
+ groupSize: group.length,
+ otherUrls: group
+ .filter((other) => other.id !== page.id)
+ .slice(0, DUPLICATE_GROUP_SAMPLE)
+ .map((other) => other.url),
+ },
+ });
+ }
+ }
+ };
+
+ emitGroups(
+ groupBy((page) => page.title || null),
+ "duplicate-title",
+ );
+ emitGroups(
+ groupBy((page) => page.metaDescription || null),
+ "duplicate-meta-description",
+ );
+ emitGroups(
+ groupBy((page) => (page.wordCount > 0 ? page.contentHash : null)),
+ "duplicate-content",
+ );
+ return issues;
+}
+
+export function findRedirectChainsAndLoops(pages: SlimPage[]): DetectedIssue[] {
+ const redirects = new Map<string, SlimPage>();
+ for (const page of pages) {
+ const isRedirect =
+ page.statusCode !== null &&
+ page.statusCode >= 300 &&
+ page.statusCode < 400 &&
+ page.redirectUrl;
+ if (isRedirect) redirects.set(page.url, page);
+ }
+
+ const redirectTargets = new Set(
+ Array.from(redirects.values(), (page) => page.redirectUrl!),
+ );
+
+ const issues: DetectedIssue[] = [];
+ const walked = new Set<string>();
+
+ // Walk from chain heads (redirects nothing else redirects to), so a 5-hop
+ // chain yields one issue, not five.
+ for (const [url, head] of redirects) {
+ if (redirectTargets.has(url)) continue;
+
+ const hops: string[] = [url];
+ const seen = new Set(hops);
+ walked.add(url);
+ let current = head.redirectUrl;
+ let isLoop = false;
+ while (current) {
+ if (seen.has(current)) {
+ isLoop = true;
+ hops.push(current);
+ break;
+ }
+ hops.push(current);
+ seen.add(current);
+ if (redirects.has(current)) walked.add(current);
+ current = redirects.get(current)?.redirectUrl ?? null;
+ }
+
+ if (isLoop) {
+ issues.push({
+ issueType: "redirect-loop",
+ pageId: head.id,
+ pageUrl: url,
+ details: { hops },
+ });
+ } else if (hops.length > 2) {
+ // url -> a -> b: two redirects before content = a chain
+ issues.push({
+ issueType: "redirect-chain",
+ pageId: head.id,
+ pageUrl: url,
+ details: { hops, finalUrl: hops[hops.length - 1] },
+ });
+ }
+ }
+
+ // Headless cycles (every member is also a target — e.g. a↔b, or a→a) are
+ // never reached from a head; emit one loop issue per cycle.
+ for (const [url, page] of redirects) {
+ if (walked.has(url)) continue;
+
+ const cycle: string[] = [];
+ let current: string | null = url;
+ while (current && !walked.has(current)) {
+ walked.add(current);
+ cycle.push(current);
+ current = redirects.get(current)?.redirectUrl ?? null;
+ }
+ issues.push({
+ issueType: "redirect-loop",
+ pageId: page.id,
+ pageUrl: url,
+ details: { hops: [...cycle, url] },
+ });
+ }
+
+ return issues;
+}
diff --git a/src/server/lib/audit/issues/multipage.ts b/src/server/lib/audit/issues/multipage.ts
new file mode 100644
index 00000000..b089e50c
--- /dev/null
+++ b/src/server/lib/audit/issues/multipage.ts
@@ -0,0 +1,154 @@
+/**
+ * Cross-page (multipage) issue checks.
+ *
+ * These run once after the crawl, against the rows the crawl wrote to D1:
+ * duplicates, broken internal links, redirect chains/loops, orphan pages.
+ * Pure set-queries over crawl data — no fetching, no DOM.
+ */
+import { and, eq, gte, lt, ne, notExists, sql } from "drizzle-orm";
+import { alias } from "drizzle-orm/sqlite-core";
+import { db } from "@/db";
+import { auditLinks, auditPages } from "@/db/schema";
+import { normalizeUrl } from "@/server/lib/audit/url-utils";
+import {
+ findDuplicates,
+ findRedirectChainsAndLoops,
+ type SlimPage,
+} from "@/server/lib/audit/issues/multipage-checks";
+import type { DetectedIssue } from "@/server/lib/audit/issues/page-reporters";
+
+const BROKEN_LINK_ISSUE_CAP = 2_000;
+
+export async function runMultipageChecks(input: {
+ auditId: string;
+ startUrl: string;
+ /** Orphan detection only makes sense when the crawl wasn't truncated. */
+ crawlCompleted: boolean;
+}): Promise<DetectedIssue[]> {
+ const pages: SlimPage[] = await db
+ .select({
+ id: auditPages.id,
+ url: auditPages.url,
+ statusCode: auditPages.statusCode,
+ fetchClass: auditPages.fetchClass,
+ title: auditPages.title,
+ metaDescription: auditPages.metaDescription,
+ contentHash: auditPages.contentHash,
+ redirectUrl: auditPages.redirectUrl,
+ wordCount: auditPages.wordCount,
+ isIndexable: auditPages.isIndexable,
+ canonicalUrl: auditPages.canonicalUrl,
+ headerCanonicalUrl: auditPages.headerCanonicalUrl,
+ })
+ .from(auditPages)
+ .where(eq(auditPages.auditId, input.auditId));
+
+ const issues: DetectedIssue[] = [
+ ...findDuplicates(pages),
+ ...findRedirectChainsAndLoops(pages),
+ ...(await findBrokenInternalLinks(input.auditId)),
+ ];
+
+ if (input.crawlCompleted) {
+ // Page rows store normalized URLs; normalize the start URL the same way
+ // so the orphan exclusion matches.
+ const normalizedStart = normalizeUrl(input.startUrl) ?? input.startUrl;
+ issues.push(...(await findOrphanPages(input.auditId, normalizedStart)));
+ }
+
+ return issues;
+}
+
+async function findBrokenInternalLinks(
+ auditId: string,
+): Promise<DetectedIssue[]> {
+ // Only flag targets we actually crawled and saw fail — never inferred from
+ // absence. Blocked targets (WAF challenges) are excluded: a 403 from bot
+ // protection is not evidence of a broken link.
+ const rows = await db
+ .select({
+ sourcePageId: auditLinks.sourcePageId,
+ sourceUrl: auditLinks.sourceUrl,
+ targetUrl: auditLinks.targetUrl,
+ targetStatus: auditPages.statusCode,
+ })
+ .from(auditLinks)
+ .innerJoin(
+ auditPages,
+ and(
+ eq(auditPages.auditId, auditLinks.auditId),
+ eq(auditPages.url, auditLinks.targetUrl),
+ ),
+ )
+ .where(
+ and(
+ eq(auditLinks.auditId, auditId),
+ eq(auditLinks.isInternal, true),
+ gte(auditPages.statusCode, 400),
+ eq(auditPages.fetchClass, "ok"),
+ ),
+ )
+ .limit(BROKEN_LINK_ISSUE_CAP);
+
+ return rows.map((row) => ({
+ issueType: "broken-internal-link" as const,
+ pageId: row.sourcePageId,
+ pageUrl: row.sourceUrl,
+ dedupeKey: row.targetUrl,
+ details: { targetUrl: row.targetUrl, targetStatus: row.targetStatus },
+ }));
+}
+
+async function findOrphanPages(
+ auditId: string,
+ startUrl: string,
+): Promise<DetectedIssue[]> {
+ // A live 2xx page is an orphan when no OTHER crawled page links to it and
+ // nothing redirects to it. Only meaningful on a completed crawl: on a
+ // truncated one, "no observed inlinks" is true of nearly everything.
+ // Error/redirect rows aren't orphans — they already get their own issues.
+ const inlink = db
+ .select({ one: sql`1` })
+ .from(auditLinks)
+ .where(
+ and(
+ eq(auditLinks.auditId, auditId),
+ eq(auditLinks.isInternal, true),
+ eq(auditLinks.targetUrl, auditPages.url),
+ // Self-links (breadcrumbs, anchors) don't make a page reachable.
+ ne(auditLinks.sourcePageId, auditPages.id),
+ ),
+ );
+
+ const redirectSourcePages = alias(auditPages, "redirect_sources");
+ const redirectSources = db
+ .select({ one: sql`1` })
+ .from(redirectSourcePages)
+ .where(
+ and(
+ eq(redirectSourcePages.auditId, auditId),
+ eq(redirectSourcePages.redirectUrl, auditPages.url),
+ ),
+ );
+
+ const rows = await db
+ .select({ id: auditPages.id, url: auditPages.url })
+ .from(auditPages)
+ .where(
+ and(
+ eq(auditPages.auditId, auditId),
+ ne(auditPages.url, startUrl),
+ eq(auditPages.fetchClass, "ok"),
+ gte(auditPages.statusCode, 200),
+ lt(auditPages.statusCode, 300),
+ notExists(inlink),
+ notExists(redirectSources),
+ ),
+ );
+
+ return rows.map((row) => ({
+ issueType: "orphan-page" as const,
+ pageId: row.id,
+ pageUrl: row.url,
+ }));
+}
diff --git a/src/server/lib/audit/issues/page-reporters.test.ts b/src/server/lib/audit/issues/page-reporters.test.ts
new file mode 100644
index 00000000..576328f2
--- /dev/null
+++ b/src/server/lib/audit/issues/page-reporters.test.ts
@@ -0,0 +1,308 @@
+import { describe, expect, it } from "vitest";
+import { runPageReporters } from "@/server/lib/audit/issues/page-reporters";
+import {
+ findDuplicates,
+ findRedirectChainsAndLoops,
+ type SlimPage,
+} from "@/server/lib/audit/issues/multipage-checks";
+import type { CrawledPageResult } from "@/server/lib/audit/types";
+
+function makePage(overrides: Partial<CrawledPageResult>): CrawledPageResult {
+ return {
+ id: "page-1",
+ url: "https://example.com/a",
+ statusCode: 200,
+ fetchClass: "ok",
+ redirectUrl: null,
+ title: "A perfectly reasonable page title",
+ metaDescription:
+ "A reasonable meta description that says something useful about the page.",
+ canonicalUrl: null,
+ robotsMeta: null,
+ xRobotsTag: null,
+ headerCanonicalUrl: null,
+ ogTitle: null,
+ ogDescription: null,
+ ogImage: null,
+ h1Count: 1,
+ h2Count: 0,
+ h3Count: 0,
+ h4Count: 0,
+ h5Count: 0,
+ h6Count: 0,
+ headingOrder: [1, 2, 3],
+ wordCount: 500,
+ contentHash: "abc123",
+ isHtml: true,
+ imagesTotal: 0,
+ imagesMissingAlt: 0,
+ images: [],
+ links: [],
+ hasStructuredData: false,
+ hreflangTags: [],
+ isIndexable: true,
+ responseTimeMs: 200,
+ crawlDepth: 1,
+ inSitemap: true,
+ ...overrides,
+ };
+}
+
+function issueTypes(page: CrawledPageResult): string[] {
+ return runPageReporters(page).map((issue) => issue.issueType);
+}
+
+describe("runPageReporters", () => {
+ it("reports nothing for a healthy page", () => {
+ expect(issueTypes(makePage({}))).toEqual([]);
+ });
+
+ it("reports only blocked-page for a blocked fetch", () => {
+ expect(
+ issueTypes(makePage({ fetchClass: "blocked", statusCode: 403 })),
+ ).toEqual(["blocked-page"]);
+ });
+
+ it("reports nothing for a fetch error", () => {
+ expect(
+ issueTypes(makePage({ fetchClass: "error", statusCode: 0 })),
+ ).toEqual([]);
+ });
+
+ it("classifies error statuses by range", () => {
+ expect(issueTypes(makePage({ statusCode: 500 }))).toEqual(["server-error"]);
+ expect(issueTypes(makePage({ statusCode: 404 }))).toEqual(["broken-page"]);
+ expect(
+ issueTypes(
+ makePage({
+ statusCode: 301,
+ redirectUrl: "https://example.com/b",
+ }),
+ ),
+ ).toEqual([]);
+ });
+
+ it("checks titles and meta descriptions", () => {
+ expect(issueTypes(makePage({ title: "" }))).toContain("missing-title");
+ expect(issueTypes(makePage({ title: "x".repeat(70) }))).toContain(
+ "title-too-long",
+ );
+ expect(issueTypes(makePage({ title: "Tiny" }))).toContain(
+ "title-too-short",
+ );
+ expect(issueTypes(makePage({ metaDescription: "" }))).toContain(
+ "missing-meta-description",
+ );
+ expect(
+ issueTypes(makePage({ metaDescription: "x".repeat(200) })),
+ ).toContain("meta-description-too-long");
+ });
+
+ it("checks headings", () => {
+ expect(issueTypes(makePage({ h1Count: 0 }))).toContain("missing-h1");
+ expect(issueTypes(makePage({ h1Count: 3 }))).toContain("multiple-h1");
+ expect(issueTypes(makePage({ headingOrder: [1, 2, 4] }))).toContain(
+ "heading-order-skip",
+ );
+ });
+
+ it("skips content checks for non-HTML responses", () => {
+ const nonHtml = makePage({
+ isHtml: false,
+ title: "",
+ metaDescription: "",
+ h1Count: 0,
+ headingOrder: [],
+ wordCount: 0,
+ contentHash: null,
+ });
+ expect(issueTypes(nonHtml)).toEqual([]);
+ });
+
+ it("still checks empty-shell HTML pages", () => {
+ const shell = makePage({
+ isHtml: true,
+ title: "",
+ metaDescription: "",
+ h1Count: 0,
+ headingOrder: [],
+ wordCount: 0,
+ contentHash: null,
+ });
+ const types = issueTypes(shell);
+ expect(types).toContain("missing-title");
+ expect(types).toContain("missing-h1");
+ expect(types).toContain("thin-content");
+ });
+
+ it("flags indexability and canonical signals", () => {
+ expect(
+ issueTypes(makePage({ isIndexable: false, robotsMeta: "noindex" })),
+ ).toContain("noindex-page");
+
+ const conflicted = issueTypes(
+ makePage({
+ canonicalUrl: "https://example.com/canonical-a",
+ headerCanonicalUrl: "https://example.com/canonical-b",
+ }),
+ );
+ expect(conflicted).toContain("canonical-conflict");
+ expect(conflicted).toContain("canonicalized-page");
+
+ expect(
+ issueTypes(makePage({ canonicalUrl: "https://example.com/a" })),
+ ).not.toContain("canonicalized-page");
+ });
+
+ it("flags thin content only on indexable pages", () => {
+ expect(issueTypes(makePage({ wordCount: 50 }))).toContain("thin-content");
+ expect(
+ issueTypes(
+ makePage({ wordCount: 50, isIndexable: false, robotsMeta: "noindex" }),
+ ),
+ ).not.toContain("thin-content");
+ });
+
+ it("flags slow responses and deep pages", () => {
+ expect(issueTypes(makePage({ responseTimeMs: 3000 }))).toContain(
+ "slow-response",
+ );
+ expect(issueTypes(makePage({ crawlDepth: 6 }))).toContain("deep-page");
+ expect(issueTypes(makePage({ crawlDepth: null }))).not.toContain(
+ "deep-page",
+ );
+ });
+});
+
+function makeSlimPage(overrides: Partial<SlimPage>): SlimPage {
+ return {
+ id: overrides.url ?? "page",
+ url: "https://example.com/a",
+ statusCode: 200,
+ fetchClass: "ok",
+ title: null,
+ metaDescription: null,
+ contentHash: null,
+ redirectUrl: null,
+ wordCount: 100,
+ isIndexable: true,
+ canonicalUrl: null,
+ headerCanonicalUrl: null,
+ ...overrides,
+ };
+}
+
+describe("findDuplicates", () => {
+ it("flags duplicate titles across pages and includes the other URLs", () => {
+ const issues = findDuplicates([
+ makeSlimPage({ url: "https://example.com/a", title: "Same" }),
+ makeSlimPage({ url: "https://example.com/b", title: "Same" }),
+ makeSlimPage({ url: "https://example.com/c", title: "Different" }),
+ ]);
+ const duplicateTitles = issues.filter(
+ (issue) => issue.issueType === "duplicate-title",
+ );
+ expect(duplicateTitles).toHaveLength(2);
+ expect(duplicateTitles[0].details?.otherUrls).toEqual([
+ "https://example.com/b",
+ ]);
+ });
+
+ it("excludes noindexed and canonicalized pages from duplicate groups", () => {
+ const issues = findDuplicates([
+ makeSlimPage({ url: "https://example.com/a", title: "Same" }),
+ makeSlimPage({
+ url: "https://example.com/b",
+ title: "Same",
+ canonicalUrl: "https://example.com/a",
+ }),
+ makeSlimPage({
+ url: "https://example.com/c",
+ title: "Same",
+ isIndexable: false,
+ }),
+ ]);
+ expect(issues).toHaveLength(0);
+ });
+
+ it("ignores non-2xx and blocked pages", () => {
+ const issues = findDuplicates([
+ makeSlimPage({ url: "https://example.com/a", title: "Same" }),
+ makeSlimPage({
+ url: "https://example.com/b",
+ title: "Same",
+ fetchClass: "blocked",
+ statusCode: 403,
+ }),
+ ]);
+ expect(issues).toHaveLength(0);
+ });
+
+ it("groups duplicate content by hash only when there is text", () => {
+ const issues = findDuplicates([
+ makeSlimPage({ url: "https://example.com/a", contentHash: "h1" }),
+ makeSlimPage({ url: "https://example.com/b", contentHash: "h1" }),
+ makeSlimPage({
+ url: "https://example.com/empty-1",
+ contentHash: "h2",
+ wordCount: 0,
+ }),
+ makeSlimPage({
+ url: "https://example.com/empty-2",
+ contentHash: "h2",
+ wordCount: 0,
+ }),
+ ]);
+ expect(
+ issues.filter((issue) => issue.issueType === "duplicate-content"),
+ ).toHaveLength(2);
+ });
+});
+
+describe("findRedirectChainsAndLoops", () => {
+ const redirect = (url: string, target: string) =>
+ makeSlimPage({ url, statusCode: 301, redirectUrl: target });
+
+ it("ignores single redirects", () => {
+ expect(
+ findRedirectChainsAndLoops([
+ redirect("https://example.com/a", "https://example.com/b"),
+ makeSlimPage({ url: "https://example.com/b" }),
+ ]),
+ ).toHaveLength(0);
+ });
+
+ it("flags a chain once, on its head", () => {
+ const issues = findRedirectChainsAndLoops([
+ redirect("https://example.com/a", "https://example.com/b"),
+ redirect("https://example.com/b", "https://example.com/c"),
+ makeSlimPage({ url: "https://example.com/c" }),
+ ]);
+ expect(issues).toHaveLength(1);
+ expect(issues[0].issueType).toBe("redirect-chain");
+ expect(issues[0].pageUrl).toBe("https://example.com/a");
+ expect(issues[0].details?.hops).toEqual([
+ "https://example.com/a",
+ "https://example.com/b",
+ "https://example.com/c",
+ ]);
+ });
+
+ it("flags loops", () => {
+ const issues = findRedirectChainsAndLoops([
+ redirect("https://example.com/a", "https://example.com/b"),
+ redirect("https://example.com/b", "https://example.com/a"),
+ ]);
+ expect(
+ issues.filter((issue) => issue.issueType === "redirect-loop").length,
+ ).toBeGreaterThan(0);
+ });
+
+ it("flags self-loops", () => {
+ const issues = findRedirectChainsAndLoops([
+ redirect("https://example.com/a", "https://example.com/a"),
+ ]);
+ expect(issues).toHaveLength(1);
+ expect(issues[0].issueType).toBe("redirect-loop");
+ });
+});
diff --git a/src/server/lib/audit/issues/page-reporters.ts b/src/server/lib/audit/issues/page-reporters.ts
new file mode 100644
index 00000000..b9886784
--- /dev/null
+++ b/src/server/lib/audit/issues/page-reporters.ts
@@ -0,0 +1,146 @@
+/**
+ * Per-page issue reporters.
+ *
+ * Each reporter is a pure function over a single crawled page record —
+ * DOM-free by design (cheerio runs once in crawlPage), so the engine works
+ * over any crawl source that can produce a CrawledPageResult.
+ *
+ * Cross-page checks (duplicates, broken links, orphans, redirect chains)
+ * live in multipage.ts and run over D1 after the crawl.
+ */
+import type { AuditIssueType } from "@/shared/audit-issues";
+import type { CrawledPageResult } from "@/server/lib/audit/types";
+
+export interface DetectedIssue {
+ issueType: AuditIssueType;
+ pageId: string | null;
+ pageUrl: string;
+ details?: Record<string, unknown>;
+ /**
+ * Distinguishes multiple issues of the same type on the same page
+ * (e.g. one broken-internal-link issue per target). Part of the
+ * deterministic row id, so step retries don't duplicate issues.
+ */
+ dedupeKey?: string;
+}
+
+const TITLE_MAX_CHARS = 60;
+const TITLE_MIN_CHARS = 10;
+const META_DESCRIPTION_MAX_CHARS = 160;
+const THIN_CONTENT_WORDS = 150;
+const SLOW_RESPONSE_MS = 1500;
+const DEEP_PAGE_DEPTH = 5;
+
+function hasHeadingLevelSkip(headingOrder: number[]): boolean {
+ for (let i = 1; i < headingOrder.length; i++) {
+ if (headingOrder[i] > headingOrder[i - 1] + 1) return true;
+ }
+ return false;
+}
+
+export function runPageReporters(page: CrawledPageResult): DetectedIssue[] {
+ const issues: DetectedIssue[] = [];
+ const report = (
+ issueType: AuditIssueType,
+ details?: Record<string, unknown>,
+ ) => issues.push({ issueType, pageId: page.id, pageUrl: page.url, details });
+
+ if (page.fetchClass === "blocked") {
+ report("blocked-page", { statusCode: page.statusCode });
+ return issues;
+ }
+ if (page.fetchClass === "error") {
+ return issues;
+ }
+
+ if (page.statusCode >= 500) {
+ report("server-error", { statusCode: page.statusCode });
+ return issues;
+ }
+ if (page.statusCode >= 400) {
+ report("broken-page", { statusCode: page.statusCode });
+ return issues;
+ }
+ // Redirects are normal on their own; chains/loops are flagged in multipage.
+ if (page.statusCode >= 300) {
+ return issues;
+ }
+
+ if (page.responseTimeMs > SLOW_RESPONSE_MS) {
+ report("slow-response", { responseTimeMs: page.responseTimeMs });
+ }
+
+ // Content checks only make sense for analyzed HTML documents (a PDF has no
+ // title tag to miss; an empty-shell HTML page very much does).
+ if (!page.isHtml) {
+ return issues;
+ }
+
+ // Titles
+ if (!page.title) {
+ report("missing-title");
+ } else if (page.title.length > TITLE_MAX_CHARS) {
+ report("title-too-long", { length: page.title.length });
+ } else if (page.title.length < TITLE_MIN_CHARS) {
+ report("title-too-short", { length: page.title.length });
+ }
+
+ // Meta description
+ if (!page.metaDescription) {
+ report("missing-meta-description");
+ } else if (page.metaDescription.length > META_DESCRIPTION_MAX_CHARS) {
+ report("meta-description-too-long", {
+ length: page.metaDescription.length,
+ });
+ }
+
+ // Headings
+ if (page.h1Count === 0) {
+ report("missing-h1");
+ } else if (page.h1Count > 1) {
+ report("multiple-h1", { h1Count: page.h1Count });
+ }
+ if (hasHeadingLevelSkip(page.headingOrder)) {
+ report("heading-order-skip");
+ }
+
+ // Indexability + canonical signals
+ if (!page.isIndexable) {
+ report("noindex-page", {
+ robotsMeta: page.robotsMeta,
+ xRobotsTag: page.xRobotsTag,
+ });
+ }
+ if (
+ page.canonicalUrl &&
+ page.headerCanonicalUrl &&
+ page.canonicalUrl !== page.headerCanonicalUrl
+ ) {
+ report("canonical-conflict", {
+ htmlCanonical: page.canonicalUrl,
+ headerCanonical: page.headerCanonicalUrl,
+ });
+ }
+ const effectiveCanonical = page.canonicalUrl ?? page.headerCanonicalUrl;
+ if (effectiveCanonical && effectiveCanonical !== page.url) {
+ report("canonicalized-page", { canonicalUrl: effectiveCanonical });
+ }
+
+ // Content quality
+ if (page.isIndexable && page.wordCount < THIN_CONTENT_WORDS) {
+ report("thin-content", { wordCount: page.wordCount });
+ }
+ if (page.imagesMissingAlt > 0) {
+ report("images-missing-alt", {
+ imagesMissingAlt: page.imagesMissingAlt,
+ imagesTotal: page.imagesTotal,
+ });
+ }
+
+ // Structure
+ if (page.crawlDepth !== null && page.crawlDepth >= DEEP_PAGE_DEPTH) {
+ report("deep-page", { crawlDepth: page.crawlDepth });
+ }
+
+ return issues;
+}
diff --git a/src/server/lib/audit/lighthouse.ts b/src/server/lib/audit/lighthouse.ts
index 5194ed0c..50976f85 100644
--- a/src/server/lib/audit/lighthouse.ts
+++ b/src/server/lib/audit/lighthouse.ts
@@ -1,4 +1,4 @@
-import { detectUrlTemplate } from "./url-utils";
+import { detectUrlTemplate, normalizeUrl } from "./url-utils";
import type { BillingCustomerContext } from "@/server/billing/subscription";
import { createDataforseoClient } from "@/server/lib/dataforseo";
import type { LighthouseResult, LighthouseStrategy } from "./types";
@@ -127,16 +127,13 @@ export function selectLighthouseSample(
(p) => p.statusCode >= 200 && p.statusCode < 300,
);
- if (strategy === "manual") {
- // manual = user picks after crawl; for now return empty
- return [];
- }
-
// strategy === "auto": homepage + 1 per URL pattern, capped at 10
const selected = new Set<string>();
- // Always include the start URL / homepage
- const startPage = validPages.find((p) => p.url === startUrl);
+ // Always include the start URL / homepage. Page URLs are normalized;
+ // normalize the start URL the same way or the comparison silently misses.
+ const normalizedStart = normalizeUrl(startUrl) ?? startUrl;
+ const startPage = validPages.find((p) => p.url === normalizedStart);
if (startPage) selected.add(startPage.url);
// Group by URL template pattern
diff --git a/src/server/lib/audit/page-analyzer.ts b/src/server/lib/audit/page-analyzer.ts
index e853ae0c..8c424c88 100644
--- a/src/server/lib/audit/page-analyzer.ts
+++ b/src/server/lib/audit/page-analyzer.ts
@@ -7,7 +7,7 @@
*/
import * as cheerio from "cheerio";
import { normalizeUrl, isSameOrigin } from "./url-utils";
-import type { PageAnalysis } from "./types";
+import type { PageAnalysis, PageLink } from "./types";
/**
* Analyze an HTML string and extract all SEO-relevant data.
@@ -71,9 +71,8 @@ export function analyzeHtml(
});
});
- // --- Links ---
- const internalLinks: string[] = [];
- const externalLinks: string[] = [];
+ // --- Links (deduped by target URL; first anchor wins) ---
+ const linksByTarget = new Map<string, PageLink>();
$("a[href]").each((_, el) => {
const href = $(el).attr("href");
@@ -84,13 +83,18 @@ export function analyzeHtml(
const resolved = normalizeUrl(href, pageUrl);
if (!resolved) return;
-
- if (isSameOrigin(resolved, pageUrl)) {
- internalLinks.push(resolved);
- } else {
- externalLinks.push(resolved);
- }
+ if (linksByTarget.has(resolved)) return;
+
+ const anchor = $(el).text().replace(/\s+/g, " ").trim().slice(0, 200);
+ const rel = $(el).attr("rel")?.toLowerCase() ?? "";
+ linksByTarget.set(resolved, {
+ targetUrl: resolved,
+ anchor: anchor || null,
+ isInternal: isSameOrigin(resolved, pageUrl),
+ isNofollow: rel.split(/\s+/).includes("nofollow"),
+ });
});
+ const links = Array.from(linksByTarget.values());
// --- Structured data (JSON-LD) ---
let hasStructuredData = false;
@@ -119,9 +123,9 @@ export function analyzeHtml(
h1s,
headingOrder,
wordCount,
+ bodyText,
images,
- internalLinks,
- externalLinks,
+ links,
hasStructuredData,
hreflangTags,
};
diff --git a/src/server/lib/audit/types.ts b/src/server/lib/audit/types.ts
index bb2bf157..e7ca4028 100644
--- a/src/server/lib/audit/types.ts
+++ b/src/server/lib/audit/types.ts
@@ -6,7 +6,7 @@ import { z } from "zod";
import { MIN_AUDIT_PAGES, PAID_MAX_AUDIT_PAGES } from "@/shared/audit-limits";
import { jsonCodec } from "@/shared/json";
-export type LighthouseStrategy = "auto" | "manual" | "none";
+export type LighthouseStrategy = "auto" | "none";
export interface AuditConfig {
maxPages: number;
@@ -14,11 +14,21 @@ export interface AuditConfig {
}
// Read-side only (writes stringify a typed AuditConfig). Stored rows may hold
-// retired strategies (e.g. "all"); fall back to "auto" instead of failing the
-// whole config parse and making the audit's results unviewable.
+// retired strategies ("all", "manual") from older audits; map them onto the
+// closest surviving strategy — and fall back to "auto" on anything unknown —
+// instead of failing the whole config parse and making the audit's results
+// unviewable.
+const lighthouseStrategySchema = z
+ .enum(["auto", "all", "manual", "none"])
+ .transform(
+ (value): LighthouseStrategy =>
+ value === "all" ? "auto" : value === "manual" ? "none" : value,
+ )
+ .catch("auto");
+
const auditConfigSchema = z.object({
maxPages: z.number().int().min(MIN_AUDIT_PAGES).max(PAID_MAX_AUDIT_PAGES),
- lighthouseStrategy: z.enum(["auto", "manual", "none"]).catch("auto"),
+ lighthouseStrategy: lighthouseStrategySchema,
});
const auditConfigCodec = jsonCodec(auditConfigSchema);
@@ -29,6 +39,17 @@ export function parseAuditConfig(configRaw: string | null): AuditConfig | null {
return result.success ? result.data : null;
}
+/** How a page fetch resolved. "blocked" = WAF/bot challenge stood in the way. */
+export type PageFetchClass = "ok" | "blocked" | "error";
+
+/** One outgoing link edge, deduped by target URL within a page. */
+export interface PageLink {
+ targetUrl: string;
+ anchor: string | null;
+ isInternal: boolean;
+ isNofollow: boolean;
+}
+
/** Data extracted from a single page via cheerio. */
export interface PageAnalysis {
url: string;
@@ -51,13 +72,13 @@ export interface PageAnalysis {
// Content
wordCount: number;
+ bodyText: string;
// Images
images: Array<{ src: string | null; alt: string | null }>;
- // Links (raw href values from the HTML)
- internalLinks: string[];
- externalLinks: string[];
+ // Links (normalized, deduped by target)
+ links: PageLink[];
// Structured data
hasStructuredData: boolean;
@@ -84,15 +105,22 @@ export interface LighthouseResult {
payloadSizeBytes?: number | null;
}
-export interface StepPageResult {
+/**
+ * Full result of crawling one page. Persisted to D1 inside the crawl-batch
+ * step; never accumulated in memory or returned as durable step state.
+ */
+export interface CrawledPageResult {
id: string;
url: string;
statusCode: number;
+ fetchClass: PageFetchClass;
redirectUrl: string | null;
title: string;
metaDescription: string;
canonicalUrl: string | null;
robotsMeta: string | null;
+ xRobotsTag: string | null;
+ headerCanonicalUrl: string | null;
ogTitle: string | null;
ogDescription: string | null;
ogImage: string | null;
@@ -104,13 +132,37 @@ export interface StepPageResult {
h6Count: number;
headingOrder: number[];
wordCount: number;
+ contentHash: string | null;
+ /**
+ * True when an HTML document was fetched and analyzed. Gates the content
+ * checks in page reporters (an empty-shell HTML page must still be
+ * checked; a PDF must not). Transient — not persisted.
+ */
+ isHtml: boolean;
imagesTotal: number;
imagesMissingAlt: number;
images: Array<{ src: string | null; alt: string | null }>;
- internalLinks: string[];
- externalLinks: string[];
+ links: PageLink[];
hasStructuredData: boolean;
hreflangTags: string[];
isIndexable: boolean;
responseTimeMs: number;
+ /** null = not reached via links (e.g. sitemap-seeded). */
+ crawlDepth: number | null;
+ inSitemap: boolean;
+}
+
+/**
+ * Slim per-page summary returned as durable step state from a crawl batch.
+ * Keep this small: full page data lives in D1, not in Workflow step state.
+ */
+export interface StepPageSummary {
+ id: string;
+ url: string;
+ statusCode: number;
+ fetchClass: PageFetchClass;
+ redirectUrl: string | null;
+ title: string;
+ /** Normalized same-origin link targets, for frontier expansion. */
+ internalLinks: string[];
}
diff --git a/src/server/lib/audit/url-policy.ts b/src/server/lib/audit/url-policy.ts
index f5ddbb31..0c386fd7 100644
--- a/src/server/lib/audit/url-policy.ts
+++ b/src/server/lib/audit/url-policy.ts
@@ -186,6 +186,26 @@ async function hostnameResolvesToBlockedAddress(
}
}
+/**
+ * Synchronous SSRF check for URLs discovered mid-crawl (links, redirect
+ * targets, sitemap entries). Blocks non-http(s) schemes, private/loopback IP
+ * literals, and internal hostnames. DNS resolution is only performed for the
+ * start URL (see normalizeAndValidateStartUrl); per-link DoH lookups would be
+ * prohibitively slow.
+ */
+export function isCrawlableUrl(url: string): boolean {
+ let parsed: URL;
+ try {
+ parsed = new URL(url);
+ } catch {
+ return false;
+ }
+ if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
+ return false;
+ }
+ return !isBlockedHost(parsed.hostname);
+}
+
export async function normalizeAndValidateStartUrl(
input: string,
): Promise<string> {
diff --git a/src/server/mcp/server.ts b/src/server/mcp/server.ts
index 22d224cb..3d2d8d6b 100644
--- a/src/server/mcp/server.ts
+++ b/src/server/mcp/server.ts
@@ -22,6 +22,12 @@ import {
getSearchConsolePerformanceTool,
inspectUrlsTool,
} from "@/server/mcp/tools/search-console-tools";
+import {
+ getAuditIssuesTool,
+ getAuditPagesTool,
+ getAuditStatusTool,
+ runSiteAuditTool,
+} from "@/server/mcp/tools/site-audit-tools";
import { whoamiTool } from "@/server/mcp/tools/whoami";
// Each handler is wrapped with instrumentMcpToolHandler so failures reach
@@ -201,4 +207,24 @@ export function registerOpenSeoMcpTools(server: McpServer) {
inspectUrlsTool.handler,
),
);
+ server.registerTool(
+ runSiteAuditTool.name,
+ runSiteAuditTool.config,
+ runSiteAuditTool.handler,
+ );
+ server.registerTool(
+ getAuditStatusTool.name,
+ getAuditStatusTool.config,
+ getAuditStatusTool.handler,
+ );
+ server.registerTool(
+ getAuditIssuesTool.name,
+ getAuditIssuesTool.config,
+ getAuditIssuesTool.handler,
+ );
+ server.registerTool(
+ getAuditPagesTool.name,
+ getAuditPagesTool.config,
+ getAuditPagesTool.handler,
+ );
}
diff --git a/src/server/mcp/tools/site-audit-tools.ts b/src/server/mcp/tools/site-audit-tools.ts
new file mode 100644
index 00000000..488bc787
--- /dev/null
+++ b/src/server/mcp/tools/site-audit-tools.ts
@@ -0,0 +1,399 @@
+import { z } from "zod";
+import { AuditRepository } from "@/server/features/audit/repositories/AuditRepository";
+import { AuditService } from "@/server/features/audit/services/AuditService";
+import { AppError } from "@/server/lib/errors";
+import { captureServerEvent } from "@/server/lib/posthog";
+import {
+ AUDIT_ISSUE_TYPES,
+ getIssueDescriptor,
+ ISSUE_SEVERITY_ORDER,
+} from "@/shared/audit-issues";
+import { mcpResponse } from "@/server/mcp/formatters";
+import { buildProjectMeta } from "@/server/mcp/context";
+import {
+ looseObjectOutputSchema,
+ optionalMetaOutputSchema,
+} from "@/server/mcp/output-schemas";
+import { withMcpProjectAuth } from "@/server/mcp/project-auth";
+import { projectIdSchema } from "@/server/mcp/schemas";
+
+const auditIdSchema = z
+ .string()
+ .optional()
+ .describe("Audit ID. If omitted, uses the project's most recent audit.");
+
+async function resolveAudit(projectId: string, auditId?: string) {
+ const audit = auditId
+ ? await AuditRepository.getAuditForProject(auditId, projectId)
+ : await AuditRepository.getLatestAuditForProject(projectId);
+ if (!audit) {
+ throw new AppError(
+ "NOT_FOUND",
+ auditId
+ ? `Audit ${auditId} not found in this project.`
+ : "No audits exist for this project yet. Start one with run_site_audit.",
+ );
+ }
+ return audit;
+}
+
+function auditPath(projectId: string, auditId: string) {
+ return `/p/${projectId}/audit?auditId=${auditId}`;
+}
+
+// ─── run_site_audit ──────────────────────────────────────────────────────────
+
+const runInputSchema = {
+ projectId: projectIdSchema,
+ url: z.string().min(1).max(2048).describe("Start URL to crawl."),
+ maxPages: z
+ .number()
+ .int()
+ .min(10)
+ .max(10_000)
+ .optional()
+ .describe("Page budget for the crawl (default 50)."),
+ runLighthouse: z
+ .boolean()
+ .optional()
+ .describe(
+ "Run Lighthouse on a sample of up to 10 representative pages (default true).",
+ ),
+} as const;
+
+type RunArgs = z.infer<z.ZodObject<typeof runInputSchema>>;
+
+export const runSiteAuditTool = {
+ name: "run_site_audit",
+ config: {
+ title: "Run site audit",
+ description:
+ "Start a site audit: crawls the site (robots.txt-aware, same-origin), checks every page for SEO issues (broken links, duplicate/missing titles and descriptions, redirect chains, orphan pages, canonical problems, thin content, and more), and optionally runs Lighthouse on a sample of pages. Runs in the background — poll get_audit_status, then read get_audit_issues. If the site blocks our crawler, pages are honestly flagged as blocked rather than misreported.",
+ inputSchema: runInputSchema,
+ outputSchema: z
+ .object({
+ auditId: z.string(),
+ ...optionalMetaOutputSchema,
+ })
+ .passthrough(),
+ annotations: {
+ readOnlyHint: false,
+ openWorldHint: true,
+ destructiveHint: false,
+ },
+ },
+ handler: withMcpProjectAuth(async (args: RunArgs, context) => {
+ const lighthouseStrategy = (args.runLighthouse ?? true) ? "auto" : "none";
+ const limitTier = await AuditService.resolveAuditLimitTier(
+ context.auth.organizationId,
+ );
+ let auditId: string;
+ try {
+ ({ auditId } = await AuditService.startAudit({
+ actorUserId: context.auth.userId,
+ billingCustomer: context.billing,
+ projectId: args.projectId,
+ startUrl: args.url,
+ maxPages: args.maxPages,
+ lighthouseStrategy,
+ limitTier,
+ }));
+ } catch (error) {
+ if (
+ error instanceof AppError &&
+ error.code === "AUDIT_CAPACITY_REACHED"
+ ) {
+ return mcpResponse({
+ text: "Audit capacity reached for this account — delete old audits in the dashboard to free capacity, then try again.",
+ meta: buildProjectMeta(
+ context,
+ args.projectId,
+ `/p/${args.projectId}/audit`,
+ ),
+ });
+ }
+ throw error;
+ }
+
+ await captureServerEvent({
+ distinctId: context.auth.userId,
+ event: "site_audit:start",
+ organizationId: context.auth.organizationId,
+ properties: {
+ project_id: args.projectId,
+ max_pages: args.maxPages ?? 50,
+ run_lighthouse: lighthouseStrategy !== "none",
+ source: "mcp",
+ },
+ });
+
+ return mcpResponse({
+ text: `Audit ${auditId} started for ${args.url}. Poll get_audit_status until it completes, then call get_audit_issues for the prioritized issue report.`,
+ meta: buildProjectMeta(
+ context,
+ args.projectId,
+ auditPath(args.projectId, auditId),
+ ),
+ structuredContent: { auditId },
+ });
+ }),
+};
+
+// ─── get_audit_status ────────────────────────────────────────────────────────
+
+const statusInputSchema = {
+ projectId: projectIdSchema,
+ auditId: auditIdSchema,
+} as const;
+
+type StatusArgs = z.infer<z.ZodObject<typeof statusInputSchema>>;
+
+export const getAuditStatusTool = {
+ name: "get_audit_status",
+ config: {
+ title: "Get site audit status",
+ description:
+ "Check the progress of a site audit (phase, pages crawled, Lighthouse progress). Free — reads OpenSEO state. Omit auditId for the most recent audit.",
+ inputSchema: statusInputSchema,
+ outputSchema: z
+ .object({
+ status: looseObjectOutputSchema,
+ ...optionalMetaOutputSchema,
+ })
+ .passthrough(),
+ annotations: {
+ readOnlyHint: true,
+ openWorldHint: false,
+ destructiveHint: false,
+ },
+ },
+ handler: withMcpProjectAuth(async (args: StatusArgs, context) => {
+ // getStatus fetches (and self-heals) the audit row itself; only hit the
+ // DB here when we need to default to the most recent audit.
+ const auditId = args.auditId ?? (await resolveAudit(args.projectId)).id;
+ const status = await AuditService.getStatus(auditId, args.projectId);
+
+ const lighthouseNote =
+ status.lighthouseTotal > 0
+ ? `, lighthouse ${status.lighthouseCompleted + status.lighthouseFailed}/${status.lighthouseTotal}`
+ : "";
+ return mcpResponse({
+ text: `Audit ${status.id} (${status.startUrl}): ${status.status} — phase ${status.currentPhase}, ${status.pagesCrawled}/${status.pagesTotal} pages${lighthouseNote}.${status.status === "completed" ? " Call get_audit_issues for the issue report." : ""}`,
+ meta: buildProjectMeta(
+ context,
+ args.projectId,
+ auditPath(args.projectId, status.id),
+ ),
+ structuredContent: { status },
+ });
+ }),
+};
+
+// ─── get_audit_issues ────────────────────────────────────────────────────────
+
+const issuesInputSchema = {
+ projectId: projectIdSchema,
+ auditId: auditIdSchema,
+ severity: z
+ .enum(["critical", "warning", "info"])
+ .optional()
+ .describe("Only return issues of this severity."),
+ issueType: z
+ .string()
+ .optional()
+ .describe(
+ `Only return issues of this type. One of: ${Object.keys(AUDIT_ISSUE_TYPES).join(", ")}`,
+ ),
+ limit: z
+ .number()
+ .int()
+ .min(1)
+ .max(1_000)
+ .optional()
+ .describe("Max issues to return (default 200)."),
+} as const;
+
+type IssuesArgs = z.infer<z.ZodObject<typeof issuesInputSchema>>;
+
+export const getAuditIssuesTool = {
+ name: "get_audit_issues",
+ config: {
+ title: "Get site audit issues",
+ description:
+ "Read the prioritized issue report from a completed site audit. Every issue carries a how_to_fix with concrete remediation steps an agent can act on. Free — reads OpenSEO state. Omit auditId for the most recent audit.",
+ inputSchema: issuesInputSchema,
+ outputSchema: z
+ .object({
+ summary: z.array(looseObjectOutputSchema),
+ issues: z.array(looseObjectOutputSchema),
+ ...optionalMetaOutputSchema,
+ })
+ .passthrough(),
+ annotations: {
+ readOnlyHint: true,
+ openWorldHint: false,
+ destructiveHint: false,
+ },
+ },
+ handler: withMcpProjectAuth(async (args: IssuesArgs, context) => {
+ const audit = await resolveAudit(args.projectId, args.auditId);
+ const unsorted = await AuditRepository.getIssuesForAudit(audit.id, {
+ severity: args.severity,
+ issueType: args.issueType,
+ });
+ // Severity-first so truncation drops info rows, never critical ones.
+ const rows = unsorted.toSorted(
+ (a, b) =>
+ ISSUE_SEVERITY_ORDER[a.severity] - ISSUE_SEVERITY_ORDER[b.severity] ||
+ a.issueType.localeCompare(b.issueType),
+ );
+
+ const counts = new Map<string, number>();
+ for (const row of rows) {
+ counts.set(row.issueType, (counts.get(row.issueType) ?? 0) + 1);
+ }
+ const summary = Array.from(counts.entries())
+ .map(([issueType, count]) => {
+ const descriptor = getIssueDescriptor(issueType);
+ return {
+ issueType,
+ title: descriptor?.title ?? issueType,
+ severity: descriptor?.severity ?? "info",
+ count,
+ };
+ })
+ .toSorted(
+ (a, b) =>
+ ISSUE_SEVERITY_ORDER[a.severity] - ISSUE_SEVERITY_ORDER[b.severity] ||
+ b.count - a.count,
+ );
+
+ const limit = args.limit ?? 200;
+ const issues = rows.slice(0, limit).map((row) => {
+ const descriptor = getIssueDescriptor(row.issueType);
+ return {
+ severity: row.severity,
+ issueType: row.issueType,
+ title: descriptor?.title ?? row.issueType,
+ url: row.pageUrl,
+ details: row.detailsJson
+ ? (JSON.parse(row.detailsJson) as unknown)
+ : null,
+ howToFix: descriptor?.howToFix ?? null,
+ };
+ });
+
+ const text =
+ rows.length === 0
+ ? args.severity || args.issueType
+ ? `No issues found for audit ${audit.id} matching the given filters.`
+ : `No issues recorded for audit ${audit.id}. Note: audits run before issue checks existed have no issue data — re-run the audit with run_site_audit to get a real report.`
+ : [
+ `Audit ${audit.id} (${audit.startUrl}): ${rows.length} issues${rows.length > limit ? ` (showing ${limit})` : ""}.`,
+ "By type:",
+ ...summary.map(
+ (entry) =>
+ `- [${entry.severity}] ${entry.title} (${entry.issueType}): ${entry.count}`,
+ ),
+ "Full issue rows with how_to_fix instructions are in structuredContent.issues.",
+ ].join("\n");
+
+ return mcpResponse({
+ text,
+ meta: buildProjectMeta(
+ context,
+ args.projectId,
+ auditPath(args.projectId, audit.id),
+ ),
+ structuredContent: { summary, issues },
+ });
+ }),
+};
+
+// ─── get_audit_pages ─────────────────────────────────────────────────────────
+
+const pagesInputSchema = {
+ projectId: projectIdSchema,
+ auditId: auditIdSchema,
+ fetchClass: z
+ .enum(["ok", "blocked", "error"])
+ .optional()
+ .describe(
+ 'Filter by fetch outcome ("blocked" = the site\'s bot protection challenged the crawler).',
+ ),
+ statusCode: z
+ .number()
+ .int()
+ .optional()
+ .describe("Filter by exact HTTP status code."),
+ urlContains: z
+ .string()
+ .optional()
+ .describe("Filter to URLs containing this substring."),
+ limit: z
+ .number()
+ .int()
+ .min(1)
+ .max(1_000)
+ .optional()
+ .describe("Max pages to return (default 100)."),
+} as const;
+
+type PagesArgs = z.infer<z.ZodObject<typeof pagesInputSchema>>;
+
+export const getAuditPagesTool = {
+ name: "get_audit_pages",
+ config: {
+ title: "Get site audit pages",
+ description:
+ "List crawled pages from a site audit with per-page SEO data (status, title, description, word count, indexability, crawl depth, link counts). Free — reads OpenSEO state. Omit auditId for the most recent audit.",
+ inputSchema: pagesInputSchema,
+ outputSchema: z
+ .object({
+ pages: z.array(looseObjectOutputSchema),
+ total: z.number(),
+ ...optionalMetaOutputSchema,
+ })
+ .passthrough(),
+ annotations: {
+ readOnlyHint: true,
+ openWorldHint: false,
+ destructiveHint: false,
+ },
+ },
+ handler: withMcpProjectAuth(async (args: PagesArgs, context) => {
+ const audit = await resolveAudit(args.projectId, args.auditId);
+ const allPages = await AuditRepository.getPagesForAudit(audit.id);
+
+ const filtered = allPages.filter(
+ (page) =>
+ (!args.fetchClass || page.fetchClass === args.fetchClass) &&
+ (args.statusCode === undefined ||
+ page.statusCode === args.statusCode) &&
+ (!args.urlContains || page.url.includes(args.urlContains)),
+ );
+ const limit = args.limit ?? 100;
+ const pages = filtered.slice(0, limit);
+
+ const text = [
+ `Audit ${audit.id}: ${filtered.length} pages${filtered.length > limit ? ` (showing ${limit})` : ""}.`,
+ ...pages
+ .slice(0, 25)
+ .map(
+ (page) =>
+ `- ${page.statusCode} ${page.url}${page.fetchClass !== "ok" ? ` [${page.fetchClass}]` : ""} "${page.title ?? ""}"`,
+ ),
+ "Full rows are in structuredContent.pages.",
+ ].join("\n");
+
+ return mcpResponse({
+ text,
+ meta: buildProjectMeta(
+ context,
+ args.projectId,
+ auditPath(args.projectId, audit.id),
+ ),
+ structuredContent: { pages, total: filtered.length },
+ });
+ }),
+};
diff --git a/src/server/workflows/SiteAuditWorkflow.ts b/src/server/workflows/SiteAuditWorkflow.ts
index 2ca39245..b6fe8986 100644
--- a/src/server/workflows/SiteAuditWorkflow.ts
+++ b/src/server/workflows/SiteAuditWorkflow.ts
@@ -40,20 +40,25 @@ export class SiteAuditWorkflow extends WorkflowEntrypoint<Env, AuditParams> {
const { auditId, billingCustomer, projectId, startUrl, config } =
event.payload;
- const audit = await AuditRepository.getAuditForWorkflow(
- auditId,
- event.instanceId,
- );
+ try {
+ // Inside a step so the D1 read is retried and replay-cached; a bare
+ // read here would re-execute on every replay and a transient failure
+ // would kill the instance before the catch below exists.
+ await pgStep(step, "validate-context", undefined, async () => {
+ const audit = await AuditRepository.getAuditForWorkflow(
+ auditId,
+ event.instanceId,
+ );
- if (!audit) {
- throw new Error("Audit workflow context mismatch");
- }
+ if (!audit) {
+ throw new Error("Audit workflow context mismatch");
+ }
- if (audit.projectId !== projectId) {
- throw new Error("Audit workflow project mismatch");
- }
+ if (audit.projectId !== projectId) {
+ throw new Error("Audit workflow project mismatch");
+ }
+ });
- try {
await runAuditPhases(step, {
auditId,
workflowInstanceId: event.instanceId,
diff --git a/src/server/workflows/site-audit-workflow-helpers.ts b/src/server/workflows/site-audit-workflow-helpers.ts
index 5b616024..aac85068 100644
--- a/src/server/workflows/site-audit-workflow-helpers.ts
+++ b/src/server/workflows/site-audit-workflow-helpers.ts
@@ -1,110 +1,258 @@
-import type { StepPageResult } from "@/server/lib/audit/types";
-import { isSameOrigin, normalizeUrl } from "@/server/lib/audit/url-utils";
+import type {
+ CrawledPageResult,
+ PageFetchClass,
+} from "@/server/lib/audit/types";
+import { sha256Hex } from "@/server/lib/audit/ids";
+import { normalizeUrl } from "@/server/lib/audit/url-utils";
+
+const CRAWL_USER_AGENT = "OpenSEO-Audit/1.0";
+
+/**
+ * Markers of a bot-mitigation challenge page. We classify these honestly as
+ * "blocked" instead of recording the challenge HTML as if it were the page.
+ */
+const CHALLENGE_BODY_MARKERS = [
+ "just a moment...",
+ "challenge-platform",
+ "cf-browser-verification",
+ "attention required! | cloudflare",
+ "verifying you are human",
+];
+
+function classifyFetch(
+ statusCode: number,
+ headers: Headers,
+ bodySnippet: string,
+): PageFetchClass {
+ if (statusCode === 0) return "error";
+ if (headers.get("cf-mitigated")) return "blocked";
+ if (statusCode === 401 || statusCode === 403 || statusCode === 429) {
+ return "blocked";
+ }
+ if (statusCode === 503) {
+ const snippet = bodySnippet.toLowerCase();
+ if (CHALLENGE_BODY_MARKERS.some((marker) => snippet.includes(marker))) {
+ return "blocked";
+ }
+ }
+ return "ok";
+}
+
+/** Resolve a Location header against its base without normalizing. */
+function resolveRawUrl(location: string, base: string): string | null {
+ try {
+ return new URL(location, base).toString();
+ } catch {
+ return null;
+ }
+}
+
+/** Parse `Link: <url>; rel="canonical"` response headers. */
+function parseLinkHeaderCanonical(
+ linkHeader: string | null,
+ pageUrl: string,
+): string | null {
+ if (!linkHeader) return null;
+ for (const part of linkHeader.split(",")) {
+ const match = part.match(/<([^>]+)>\s*;([^]*)/);
+ if (!match) continue;
+ if (/rel\s*=\s*"?canonical"?/i.test(match[2])) {
+ return normalizeUrl(match[1].trim(), pageUrl);
+ }
+ }
+ return null;
+}
export async function crawlPage(
url: string,
- crawlOrigin: string,
-): Promise<StepPageResult | null> {
+ crawlDepth: number | null,
+ inSitemap: boolean,
+): Promise<CrawledPageResult> {
const startTime = Date.now();
try {
- const response = await fetch(url, {
- headers: {
- "User-Agent": "OpenSEO-Audit/1.0",
- Accept: "text/html,application/xhtml+xml",
- },
- redirect: "follow",
- signal: AbortSignal.timeout(15_000),
- });
+ // Manual redirect handling: each hop is recorded as its own page row and
+ // the target is enqueued by the frontier, so chains/loops are detectable.
+ // Exception: redirects whose target normalizes to this same URL (e.g.
+ // /docs -> /docs/ on slash-canonical sites — our normalizer strips the
+ // slash) are followed inline; recording them would create self-redirect
+ // rows the frontier can never resolve.
+ let fetchUrl = url;
+ let response: Response;
+ let hops = 0;
+ for (;;) {
+ response = await fetch(fetchUrl, {
+ headers: {
+ "User-Agent": CRAWL_USER_AGENT,
+ Accept: "text/html,application/xhtml+xml",
+ },
+ redirect: "manual",
+ signal: AbortSignal.timeout(15_000),
+ });
+
+ if (response.status < 300 || response.status >= 400) break;
+
+ const location = response.headers.get("location");
+ const rawTarget = location ? resolveRawUrl(location, fetchUrl) : null;
+ const normalizedTarget = location
+ ? normalizeUrl(location, fetchUrl)
+ : null;
+ const isSelfAfterNormalization =
+ normalizedTarget === url &&
+ rawTarget !== null &&
+ rawTarget !== fetchUrl;
+ if (!isSelfAfterNormalization || hops >= 3) break;
+
+ fetchUrl = rawTarget;
+ hops += 1;
+ }
const responseTimeMs = Date.now() - startTime;
const statusCode = response.status;
- const finalUrl = normalizeUrl(response.url) ?? response.url;
- if (!isSameOrigin(finalUrl, crawlOrigin)) return null;
+ const xRobotsTag = response.headers.get("x-robots-tag");
+ const headerCanonicalUrl = parseLinkHeaderCanonical(
+ response.headers.get("link"),
+ fetchUrl,
+ );
+
+ if (statusCode >= 300 && statusCode < 400) {
+ const location = response.headers.get("location");
+ const redirectUrl = location ? normalizeUrl(location, fetchUrl) : null;
+ return emptyPageResult({
+ url,
+ statusCode,
+ fetchClass: "ok",
+ redirectUrl,
+ responseTimeMs,
+ xRobotsTag,
+ headerCanonicalUrl,
+ crawlDepth,
+ inSitemap,
+ });
+ }
- const redirectUrl =
- response.redirected && response.url !== url ? response.url : null;
const contentType = response.headers.get("content-type") ?? "";
- if (!contentType.includes("text/html")) {
- return emptyPageResult(finalUrl, statusCode, redirectUrl, responseTimeMs);
+ const isHtml = contentType.includes("text/html");
+ const body = isHtml ? await response.text() : "";
+ const fetchClass = classifyFetch(
+ statusCode,
+ response.headers,
+ body.slice(0, 4_000),
+ );
+
+ if (!isHtml || fetchClass !== "ok" || statusCode >= 400) {
+ return emptyPageResult({
+ url,
+ statusCode,
+ fetchClass,
+ redirectUrl: null,
+ responseTimeMs,
+ xRobotsTag,
+ headerCanonicalUrl,
+ crawlDepth,
+ inSitemap,
+ });
}
- const html = await response.text();
+ // Resolve links/canonical against the URL that actually served the
+ // content (it may carry a trailing slash the recorded URL doesn't).
// Dynamic import keeps cheerio (page-analyzer's HTML parser) out of the
// worker's startup module graph: SiteAuditWorkflow is re-exported from
// src/server.ts, so a static import would evaluate cheerio in every
// isolate's baseline heap, not just when an audit actually crawls.
const { analyzeHtml } = await import("@/server/lib/audit/page-analyzer");
- const analysis = analyzeHtml(
- html,
- finalUrl,
- statusCode,
- responseTimeMs,
- redirectUrl,
- );
- const isIndexable = !(
- analysis.robotsMeta?.toLowerCase().includes("noindex") ?? false
- );
- const h2Count = analysis.headingOrder.filter((h) => h === 2).length;
- const h3Count = analysis.headingOrder.filter((h) => h === 3).length;
- const h4Count = analysis.headingOrder.filter((h) => h === 4).length;
- const h5Count = analysis.headingOrder.filter((h) => h === 5).length;
- const h6Count = analysis.headingOrder.filter((h) => h === 6).length;
+ const analysis = analyzeHtml(body, fetchUrl, statusCode, responseTimeMs);
+ const robotsDirectives = [analysis.robotsMeta, xRobotsTag]
+ .filter(Boolean)
+ .join(",")
+ .toLowerCase();
+ const isIndexable = !robotsDirectives.includes("noindex");
+ const headingCount = (level: number) =>
+ analysis.headingOrder.filter((h) => h === level).length;
return {
id: crypto.randomUUID(),
- url: finalUrl,
+ url,
statusCode,
- redirectUrl,
+ fetchClass,
+ redirectUrl: null,
title: analysis.title,
metaDescription: analysis.metaDescription,
- canonicalUrl: analysis.canonical,
+ canonicalUrl: analysis.canonical
+ ? (normalizeUrl(analysis.canonical, fetchUrl) ?? analysis.canonical)
+ : null,
robotsMeta: analysis.robotsMeta,
+ xRobotsTag,
+ headerCanonicalUrl,
ogTitle: analysis.ogTitle,
ogDescription: analysis.ogDescription,
ogImage: analysis.ogImage,
h1Count: analysis.h1s.length,
- h2Count,
- h3Count,
- h4Count,
- h5Count,
- h6Count,
+ h2Count: headingCount(2),
+ h3Count: headingCount(3),
+ h4Count: headingCount(4),
+ h5Count: headingCount(5),
+ h6Count: headingCount(6),
headingOrder: analysis.headingOrder,
wordCount: analysis.wordCount,
+ contentHash: analysis.bodyText
+ ? await sha256Hex(analysis.bodyText)
+ : null,
+ isHtml: true,
imagesTotal: analysis.images.length,
- imagesMissingAlt: analysis.images.filter(
- (img) => !img.alt || img.alt === "",
- ).length,
+ // Only a truly absent alt attribute counts: alt="" is the correct
+ // markup for decorative images.
+ imagesMissingAlt: analysis.images.filter((img) => img.alt === null)
+ .length,
images: analysis.images,
- internalLinks: analysis.internalLinks,
- externalLinks: analysis.externalLinks,
+ links: analysis.links,
hasStructuredData: analysis.hasStructuredData,
hreflangTags: analysis.hreflangTags,
isIndexable,
responseTimeMs,
+ crawlDepth,
+ inSitemap,
};
} catch (error) {
const responseTimeMs = Date.now() - startTime;
console.warn(`Failed to crawl ${url}:`, error);
- return emptyPageResult(url, 0, null, responseTimeMs);
+ return emptyPageResult({
+ url,
+ statusCode: 0,
+ fetchClass: "error",
+ redirectUrl: null,
+ responseTimeMs,
+ xRobotsTag: null,
+ headerCanonicalUrl: null,
+ crawlDepth,
+ inSitemap,
+ });
}
}
-function emptyPageResult(
- url: string,
- statusCode: number,
- redirectUrl: string | null,
- responseTimeMs: number,
-): StepPageResult {
+function emptyPageResult(input: {
+ url: string;
+ statusCode: number;
+ fetchClass: PageFetchClass;
+ redirectUrl: string | null;
+ responseTimeMs: number;
+ xRobotsTag: string | null;
+ headerCanonicalUrl: string | null;
+ crawlDepth: number | null;
+ inSitemap: boolean;
+}): CrawledPageResult {
return {
id: crypto.randomUUID(),
- url,
- statusCode,
- redirectUrl,
+ url: input.url,
+ statusCode: input.statusCode,
+ fetchClass: input.fetchClass,
+ redirectUrl: input.redirectUrl,
title: "",
metaDescription: "",
canonicalUrl: null,
robotsMeta: null,
+ xRobotsTag: input.xRobotsTag,
+ headerCanonicalUrl: input.headerCanonicalUrl,
ogTitle: null,
ogDescription: null,
ogImage: null,
@@ -116,14 +264,17 @@ function emptyPageResult(
h6Count: 0,
headingOrder: [],
wordCount: 0,
+ contentHash: null,
+ isHtml: false,
imagesTotal: 0,
imagesMissingAlt: 0,
images: [],
- internalLinks: [],
- externalLinks: [],
+ links: [],
hasStructuredData: false,
hreflangTags: [],
isIndexable: false,
- responseTimeMs,
+ responseTimeMs: input.responseTimeMs,
+ crawlDepth: input.crawlDepth,
+ inSitemap: input.inSitemap,
};
}
diff --git a/src/server/workflows/siteAuditWorkflowCrawl.ts b/src/server/workflows/siteAuditWorkflowCrawl.ts
index a915e34c..08b9871f 100644
--- a/src/server/workflows/siteAuditWorkflowCrawl.ts
+++ b/src/server/workflows/siteAuditWorkflowCrawl.ts
@@ -1,13 +1,21 @@
import type { WorkflowStep } from "cloudflare:workers";
import type { RobotsResult } from "@/server/lib/audit/discovery";
-import type { StepPageResult } from "@/server/lib/audit/types";
+import type { StepPageSummary } from "@/server/lib/audit/types";
import { isSameOrigin, normalizeUrl } from "@/server/lib/audit/url-utils";
+import { isCrawlableUrl } from "@/server/lib/audit/url-policy";
+import { deterministicAuditRowId } from "@/server/lib/audit/ids";
+import { runPageReporters } from "@/server/lib/audit/issues/page-reporters";
import { AuditRepository } from "@/server/features/audit/repositories/AuditRepository";
import { AuditProgressKV } from "@/server/lib/audit/progress-kv";
import { crawlPage } from "@/server/workflows/site-audit-workflow-helpers";
import { pgStep } from "@/server/workflows/pgStep";
const CRAWL_CONCURRENCY = 25;
+// Keep durable step state under the ~1MiB limit: full link lists live in D1;
+// the step return only carries new-to-the-frontier targets, deduped across
+// the batch and capped.
+const MAX_FRONTIER_LINKS_PER_BATCH = 2_000;
+const MAX_SUMMARY_TITLE_CHARS = 300;
function shouldQueueCrawlLink(
link: string,
@@ -18,12 +26,19 @@ function shouldQueueCrawlLink(
): boolean {
return (
isSameOrigin(link, origin) &&
+ isCrawlableUrl(link) &&
robots.isAllowed(link) &&
!visited.has(link) &&
!queued.has(link)
);
}
+interface QueueEntry {
+ url: string;
+ /** Clicks from the start URL; null when only reachable via sitemap. */
+ depth: number | null;
+}
+
type CrawlPhaseParams = {
auditId: string;
workflowInstanceId: string;
@@ -34,10 +49,19 @@ type CrawlPhaseParams = {
sitemapUrls: string[];
};
+/** What later phases need per page — no link lists (those stay in D1). */
+export type CrawledPageSummary = Omit<StepPageSummary, "internalLinks">;
+
+export type CrawlPhaseResult = {
+ pages: CrawledPageSummary[];
+ /** True when the frontier was exhausted before hitting maxPages. */
+ completed: boolean;
+};
+
export async function runCrawlPhase(
step: WorkflowStep,
params: CrawlPhaseParams,
-): Promise<StepPageResult[]> {
+): Promise<CrawlPhaseResult> {
const {
auditId,
workflowInstanceId,
@@ -48,43 +72,69 @@ export async function runCrawlPhase(
sitemapUrls,
} = params;
const visited = new Set<string>();
- const queue: string[] = [];
const queued = new Set<string>();
- const allPages: StepPageResult[] = [];
+ // Link-discovered URLs crawl first (BFS from the start URL); sitemap-only
+ // URLs drain last so link discovery isn't starved of page budget and
+ // orphan detection stays meaningful.
+ const linkQueue: QueueEntry[] = [];
+ const sitemapQueue: QueueEntry[] = [];
+ const sitemapSet = new Set<string>();
+ const summaries: CrawledPageSummary[] = [];
- seedCrawlQueue({
- startUrl,
- origin,
- robots,
- sitemapUrls,
- visited,
- queued,
- queue,
- });
+ const normalizedStart = normalizeUrl(startUrl) ?? startUrl;
+ if (
+ robots.isAllowed(normalizedStart) &&
+ isSameOrigin(normalizedStart, origin)
+ ) {
+ linkQueue.push({ url: normalizedStart, depth: 0 });
+ queued.add(normalizedStart);
+ }
+
+ for (const sitemapUrl of sitemapUrls) {
+ const normalized = normalizeUrl(sitemapUrl);
+ if (!normalized) continue;
+ sitemapSet.add(normalized);
+ if (!shouldQueueCrawlLink(normalized, origin, robots, visited, queued)) {
+ continue;
+ }
+ sitemapQueue.push({ url: normalized, depth: null });
+ queued.add(normalized);
+ }
let crawlBatchIndex = 0;
- while (queue.length > 0 && allPages.length < maxPages) {
- const urlsToCrawl = selectNextCrawlBatch(
- queue,
+ while (
+ (linkQueue.length > 0 || sitemapQueue.length > 0) &&
+ summaries.length < maxPages
+ ) {
+ const batchEntries = selectNextCrawlBatch({
+ linkQueue,
+ sitemapQueue,
queued,
visited,
robots,
- maxPages - allPages.length,
- );
- if (urlsToCrawl.length === 0) continue;
+ remaining: maxPages - summaries.length,
+ });
+ if (batchEntries.length === 0) continue;
crawlBatchIndex += 1;
- const crawledBatch = await runCrawlBatch(
- step,
+ const crawledBatch = await runCrawlBatch(step, {
crawlBatchIndex,
- urlsToCrawl,
- origin,
+ auditId,
+ batchEntries,
+ sitemapSet,
+ visited,
+ queued,
+ });
+ // Keep only the slim summary in memory: at 10k pages, retaining link
+ // lists for the whole crawl would not fit in the 128MB Worker heap.
+ summaries.push(
+ ...crawledBatch.map(({ internalLinks: _links, ...summary }) => summary),
);
- allPages.push(...crawledBatch);
enqueueDiscoveredLinks({
crawledBatch,
- queue,
+ batchEntries,
+ linkQueue,
queued,
visited,
origin,
@@ -96,110 +146,151 @@ export async function runCrawlPhase(
auditId,
workflowInstanceId,
crawledBatch,
- pagesCrawled: allPages.length,
+ pagesCrawled: summaries.length,
visitedCount: visited.size,
- queueLength: queue.length,
+ queueLength: linkQueue.length + sitemapQueue.length,
maxPages,
});
}
- return allPages;
+ return {
+ pages: summaries,
+ completed: linkQueue.length === 0 && sitemapQueue.length === 0,
+ };
}
-function seedCrawlQueue({
- startUrl,
- origin,
- robots,
- sitemapUrls,
- visited,
- queued,
- queue,
-}: {
- startUrl: string;
- origin: string;
- robots: RobotsResult;
- sitemapUrls: string[];
- visited: Set<string>;
+function selectNextCrawlBatch(params: {
+ linkQueue: QueueEntry[];
+ sitemapQueue: QueueEntry[];
queued: Set<string>;
- queue: string[];
+ visited: Set<string>;
+ robots: RobotsResult;
+ remaining: number;
}) {
- const normalizedStart = normalizeUrl(startUrl) ?? startUrl;
- if (
- robots.isAllowed(normalizedStart) &&
- isSameOrigin(normalizedStart, origin)
- ) {
- queue.push(normalizedStart);
- queued.add(normalizedStart);
- }
-
- for (const sitemapUrl of sitemapUrls) {
- const normalized = normalizeUrl(sitemapUrl);
- if (!normalized) continue;
- if (!shouldQueueCrawlLink(normalized, origin, robots, visited, queued)) {
- continue;
- }
- queue.push(normalized);
- queued.add(normalized);
- }
-}
-
-function selectNextCrawlBatch(
- queue: string[],
- queued: Set<string>,
- visited: Set<string>,
- robots: RobotsResult,
- remaining: number,
-) {
+ const { linkQueue, sitemapQueue, queued, visited, robots, remaining } =
+ params;
const batchSize = Math.min(CRAWL_CONCURRENCY, remaining);
- const urlsToCrawl: string[] = [];
-
- while (queue.length > 0 && urlsToCrawl.length < batchSize) {
- const url = queue.shift()!;
- queued.delete(url);
- if (visited.has(url)) continue;
- if (!robots.isAllowed(url)) continue;
- visited.add(url);
- urlsToCrawl.push(url);
+ const batchEntries: QueueEntry[] = [];
+
+ while (
+ (linkQueue.length > 0 || sitemapQueue.length > 0) &&
+ batchEntries.length < batchSize
+ ) {
+ const entry = (linkQueue.length > 0 ? linkQueue : sitemapQueue).shift()!;
+ queued.delete(entry.url);
+ if (visited.has(entry.url)) continue;
+ if (!robots.isAllowed(entry.url)) continue;
+ visited.add(entry.url);
+ batchEntries.push(entry);
}
- return urlsToCrawl;
+ return batchEntries;
}
async function runCrawlBatch(
step: WorkflowStep,
- crawlBatchIndex: number,
- urlsToCrawl: string[],
- origin: string,
-): Promise<StepPageResult[]> {
- return step.do(`crawl-batch-${crawlBatchIndex}`, async () => {
- const settled = await Promise.allSettled(
- urlsToCrawl.map((url) => crawlPage(url, origin)),
+ input: {
+ crawlBatchIndex: number;
+ auditId: string;
+ batchEntries: QueueEntry[];
+ sitemapSet: Set<string>;
+ visited: Set<string>;
+ queued: Set<string>;
+ },
+): Promise<StepPageSummary[]> {
+ const {
+ crawlBatchIndex,
+ auditId,
+ batchEntries,
+ sitemapSet,
+ visited,
+ queued,
+ } = input;
+ return pgStep(step, `crawl-batch-${crawlBatchIndex}`, undefined, async () => {
+ const pages = await Promise.all(
+ batchEntries.map((entry) =>
+ crawlPage(entry.url, entry.depth, sitemapSet.has(entry.url)),
+ ),
);
- return settled.flatMap((result) => {
- if (result.status === "fulfilled" && result.value) {
- return [result.value];
+
+ // Deterministic ids keep the D1 writes idempotent across step retries.
+ for (const page of pages) {
+ page.id = await deterministicAuditRowId(auditId, page.url);
+ }
+
+ const issues = pages.flatMap((page) => runPageReporters(page));
+ await AuditRepository.insertCrawledBatch(auditId, pages, issues);
+
+ // Frontier candidates only: drop targets already visited/queued and
+ // dedupe across the batch, so the step return stays far under the
+ // ~1MiB durable-state limit even on mega-menu sites.
+ const seenTargets = new Set<string>();
+ return pages.map((page) => {
+ const internalLinks: string[] = [];
+ for (const link of page.links) {
+ if (!link.isInternal) continue;
+ if (seenTargets.size >= MAX_FRONTIER_LINKS_PER_BATCH) break;
+ if (visited.has(link.targetUrl) || queued.has(link.targetUrl)) continue;
+ if (seenTargets.has(link.targetUrl)) continue;
+ seenTargets.add(link.targetUrl);
+ internalLinks.push(link.targetUrl);
}
- return [];
+ return {
+ id: page.id,
+ url: page.url,
+ statusCode: page.statusCode,
+ fetchClass: page.fetchClass,
+ redirectUrl: page.redirectUrl,
+ title: page.title.slice(0, MAX_SUMMARY_TITLE_CHARS),
+ internalLinks,
+ };
});
});
}
function enqueueDiscoveredLinks(params: {
- crawledBatch: StepPageResult[];
- queue: string[];
+ crawledBatch: StepPageSummary[];
+ batchEntries: QueueEntry[];
+ linkQueue: QueueEntry[];
queued: Set<string>;
visited: Set<string>;
origin: string;
robots: RobotsResult;
}) {
- const { crawledBatch, queue, queued, visited, origin, robots } = params;
+ const {
+ crawledBatch,
+ batchEntries,
+ linkQueue,
+ queued,
+ visited,
+ origin,
+ robots,
+ } = params;
+ const depthByUrl = new Map(
+ batchEntries.map((entry) => [entry.url, entry.depth]),
+ );
+
for (const pageResult of crawledBatch) {
- for (const link of pageResult.internalLinks.filter((candidate) =>
- shouldQueueCrawlLink(candidate, origin, robots, visited, queued),
- )) {
- queue.push(link);
+ const pageDepth = depthByUrl.get(pageResult.url) ?? null;
+ const childDepth = pageDepth === null ? null : pageDepth + 1;
+
+ for (const link of pageResult.internalLinks) {
+ if (!shouldQueueCrawlLink(link, origin, robots, visited, queued)) {
+ continue;
+ }
+ linkQueue.push({ url: link, depth: childDepth });
queued.add(link);
}
+
+ // Redirect targets continue the same navigation path: same depth.
+ const redirectTarget = pageResult.redirectUrl;
+ if (
+ redirectTarget &&
+ shouldQueueCrawlLink(redirectTarget, origin, robots, visited, queued)
+ ) {
+ linkQueue.push({ url: redirectTarget, depth: pageDepth });
+ queued.add(redirectTarget);
+ }
}
}
@@ -208,7 +299,7 @@ async function persistCrawlProgress(params: {
crawlBatchIndex: number;
auditId: string;
workflowInstanceId: string;
- crawledBatch: StepPageResult[];
+ crawledBatch: StepPageSummary[];
pagesCrawled: number;
visitedCount: number;
queueLength: number;
@@ -225,23 +316,24 @@ async function persistCrawlProgress(params: {
queueLength,
maxPages,
} = params;
- await step.do(`kv-progress-batch-${crawlBatchIndex}`, async () => {
- await AuditProgressKV.pushCrawledUrls(
- auditId,
- crawledBatch.map((pageResult) => ({
- url: pageResult.url,
- statusCode: pageResult.statusCode,
- title: pageResult.title,
- crawledAt: Date.now(),
- })),
- );
- });
-
+ // KV push + D1 progress in one step — merging them halves the per-batch
+ // step count against the ~1k step budget. The D1 update is idempotent; the
+ // KV push can duplicate entries on a partial retry, which is acceptable for
+ // an ephemeral progress feed (capped list, short TTL).
await pgStep(
step,
`progress-batch-${crawlBatchIndex}`,
undefined,
async () => {
+ await AuditProgressKV.pushCrawledUrls(
+ auditId,
+ crawledBatch.map((pageResult) => ({
+ url: pageResult.url,
+ statusCode: pageResult.statusCode,
+ title: pageResult.title,
+ crawledAt: Date.now(),
+ })),
+ );
await AuditRepository.updateAuditProgress(auditId, workflowInstanceId, {
pagesCrawled,
pagesTotal: Math.min(visitedCount + queueLength, maxPages),
diff --git a/src/server/workflows/siteAuditWorkflowPhases.ts b/src/server/workflows/siteAuditWorkflowPhases.ts
index 034a481f..19468025 100644
--- a/src/server/workflows/siteAuditWorkflowPhases.ts
+++ b/src/server/workflows/siteAuditWorkflowPhases.ts
@@ -1,6 +1,6 @@
import type { WorkflowStep } from "cloudflare:workers";
import type { BillingCustomerContext } from "@/server/billing/subscription";
-import { discoverUrls, fetchRobotsTxt } from "@/server/lib/audit/discovery";
+import { discoverUrls, parseRobotsTxt } from "@/server/lib/audit/discovery";
import {
fetchAndStoreLighthouseResult,
selectLighthouseSample,
@@ -8,13 +8,14 @@ import {
import { getOrigin } from "@/server/lib/audit/url-utils";
import { AuditRepository } from "@/server/features/audit/repositories/AuditRepository";
import { AuditProgressKV } from "@/server/lib/audit/progress-kv";
-import type {
- AuditConfig,
- LighthouseResult,
- StepPageResult,
-} from "@/server/lib/audit/types";
+import { runMultipageChecks } from "@/server/lib/audit/issues/multipage";
+import type { AuditConfig } from "@/server/lib/audit/types";
import { captureServerEvent } from "@/server/lib/posthog";
-import { runCrawlPhase } from "@/server/workflows/siteAuditWorkflowCrawl";
+import {
+ runCrawlPhase,
+ type CrawledPageSummary,
+ type CrawlPhaseResult,
+} from "@/server/workflows/siteAuditWorkflowCrawl";
import { pgStep } from "@/server/workflows/pgStep";
const LIGHTHOUSE_URL_BATCH_SIZE = 10;
@@ -35,22 +36,6 @@ function capSitemapSeeds(urls: string[], maxPages: number): string[] {
return seeds;
}
-function countLighthouseBatchResults(results: LighthouseResult[]): {
- completed: number;
- failed: number;
-} {
- let completed = 0;
- let failed = 0;
- for (const result of results) {
- if (result.errorMessage) {
- failed += 1;
- continue;
- }
- completed += 1;
- }
- return { completed, failed };
-}
-
type AuditPhasesParams = {
auditId: string;
workflowInstanceId: string;
@@ -82,8 +67,11 @@ export async function runAuditPhases(
origin,
maxPages,
);
- const robots = await fetchRobotsTxt(origin);
- const allPages = await runCrawlPhase(step, {
+ // Parsed outside the step from checkpointed text, so replays see the exact
+ // robots rules the original run used (a live re-fetch could differ and
+ // desync the frontier from already-persisted crawl batches).
+ const robots = parseRobotsTxt(origin, discovery.robotsText);
+ const crawl = await runCrawlPhase(step, {
auditId,
workflowInstanceId,
origin,
@@ -92,14 +80,14 @@ export async function runAuditPhases(
robots,
sitemapUrls: discovery.sitemapUrls,
});
- const lighthouseResults = await runLighthousePhase(step, {
+ await runLighthousePhase(step, {
auditId,
workflowInstanceId,
billingCustomer,
projectId,
startUrl,
config,
- allPages,
+ pages: crawl.pages,
});
await finalizeAudit({
step,
@@ -107,9 +95,9 @@ export async function runAuditPhases(
workflowInstanceId,
billingCustomer,
projectId,
+ startUrl,
config,
- allPages,
- lighthouseResults,
+ crawl,
});
}
@@ -126,7 +114,10 @@ async function runDiscoveryPhase(
pagesTotal: Math.min(result.urls.length + 1, maxPages),
currentPhase: "crawling",
});
- return { sitemapUrls: capSitemapSeeds(result.urls, maxPages) };
+ return {
+ sitemapUrls: capSitemapSeeds(result.urls, maxPages),
+ robotsText: result.robotsText,
+ };
});
}
@@ -137,13 +128,13 @@ type LighthousePhaseParams = {
projectId: string;
startUrl: string;
config: AuditConfig;
- allPages: StepPageResult[];
+ pages: CrawledPageSummary[];
};
async function runLighthousePhase(
step: WorkflowStep,
params: LighthousePhaseParams,
-): Promise<LighthouseResult[]> {
+) {
const {
auditId,
workflowInstanceId,
@@ -151,20 +142,19 @@ async function runLighthousePhase(
projectId,
startUrl,
config,
- allPages,
+ pages,
} = params;
- if (config.lighthouseStrategy === "none") return [];
+ if (config.lighthouseStrategy === "none") return;
const lighthouseWork = await selectLighthousePages({
step,
auditId,
workflowInstanceId,
- allPages,
+ pages,
startUrl,
strategy: config.lighthouseStrategy,
});
- const lighthouseResults: LighthouseResult[] = [];
let completedChecks = 0;
let failedChecks = 0;
let lighthouseBatchIndex = 0;
@@ -172,47 +162,69 @@ async function runLighthousePhase(
for (let i = 0; i < lighthouseWork.length; i += LIGHTHOUSE_URL_BATCH_SIZE) {
const batch = lighthouseWork.slice(i, i + LIGHTHOUSE_URL_BATCH_SIZE);
lighthouseBatchIndex += 1;
- const lighthouseBatchResults = await runLighthouseBatch({
- step,
- lighthouseBatchIndex,
- batch,
- billingCustomer,
- projectId,
- auditId,
- });
+ const priorCompleted = completedChecks;
+ const priorFailed = failedChecks;
- lighthouseResults.push(...lighthouseBatchResults);
- const counts = countLighthouseBatchResults(lighthouseBatchResults);
- failedChecks += counts.failed;
- completedChecks += counts.completed;
- await pgStep(
+ // Fetch, store (R2 + D1) and update progress inside one step. The step
+ // returns only counts; full results live in D1.
+ const counts = await pgStep(
step,
- `lighthouse-progress-batch-${lighthouseBatchIndex}`,
+ `lighthouse-batch-${lighthouseBatchIndex}`,
undefined,
async () => {
+ const perUrlResults = await Promise.all(
+ batch.map(async ({ url, pageId }) => {
+ const [mobileResult, desktopResult] = await Promise.all([
+ fetchAndStoreLighthouseResult({
+ url,
+ pageId,
+ strategy: "mobile",
+ billingCustomer,
+ projectId,
+ auditId,
+ }),
+ fetchAndStoreLighthouseResult({
+ url,
+ pageId,
+ strategy: "desktop",
+ billingCustomer,
+ projectId,
+ auditId,
+ }),
+ ]);
+ return [mobileResult, desktopResult];
+ }),
+ );
+ const results = perUrlResults.flat();
+ await AuditRepository.insertLighthouseResults(auditId, results);
+
+ const failed = results.filter((result) => result.errorMessage).length;
+ const completed = results.length - failed;
await AuditRepository.updateAuditProgress(auditId, workflowInstanceId, {
- lighthouseCompleted: completedChecks,
- lighthouseFailed: failedChecks,
+ lighthouseCompleted: priorCompleted + completed,
+ lighthouseFailed: priorFailed + failed,
});
+ return { completed, failed };
},
);
- }
- return lighthouseResults;
+ completedChecks += counts.completed;
+ failedChecks += counts.failed;
+ }
}
async function selectLighthousePages(params: {
step: WorkflowStep;
auditId: string;
workflowInstanceId: string;
- allPages: StepPageResult[];
+ pages: CrawledPageSummary[];
startUrl: string;
strategy: AuditConfig["lighthouseStrategy"];
}) {
- const { step, auditId, workflowInstanceId, allPages, startUrl, strategy } =
+ const { step, auditId, workflowInstanceId, pages, startUrl, strategy } =
params;
return pgStep(step, "select-lighthouse-sample", undefined, async () => {
- const sample = selectLighthouseSample(allPages, startUrl, strategy);
+ const sample = selectLighthouseSample(pages, startUrl, strategy);
const selectedUrls = new Set(sample);
await AuditRepository.updateAuditProgress(auditId, workflowInstanceId, {
@@ -221,66 +233,21 @@ async function selectLighthousePages(params: {
lighthouseCompleted: 0,
lighthouseFailed: 0,
});
- return allPages.flatMap((page) =>
+ return pages.flatMap((page) =>
selectedUrls.has(page.url) ? [{ url: page.url, pageId: page.id }] : [],
);
});
}
-async function runLighthouseBatch(params: {
- step: WorkflowStep;
- lighthouseBatchIndex: number;
- batch: Array<{ url: string; pageId: string }>;
- billingCustomer: BillingCustomerContext;
- projectId: string;
- auditId: string;
-}) {
- const {
- step,
- lighthouseBatchIndex,
- batch,
- billingCustomer,
- projectId,
- auditId,
- } = params;
- return step.do(`lighthouse-batch-${lighthouseBatchIndex}`, async () => {
- const perUrlResults = await Promise.all(
- batch.map(async ({ url, pageId }) => {
- const [mobileResult, desktopResult] = await Promise.all([
- fetchAndStoreLighthouseResult({
- url,
- pageId,
- strategy: "mobile",
- billingCustomer,
- projectId,
- auditId,
- }),
- fetchAndStoreLighthouseResult({
- url,
- pageId,
- strategy: "desktop",
- billingCustomer,
- projectId,
- auditId,
- }),
- ]);
- return [mobileResult, desktopResult];
- }),
- );
-
- return perUrlResults.flat();
- });
-}
-
async function finalizeAudit(args: {
step: WorkflowStep;
auditId: string;
workflowInstanceId: string;
billingCustomer: BillingCustomerContext;
projectId: string;
+ startUrl: string;
config: AuditConfig;
- allPages: StepPageResult[];
- lighthouseResults: LighthouseResult[];
+ crawl: CrawlPhaseResult;
}) {
const {
step,
@@ -288,23 +255,42 @@ async function finalizeAudit(args: {
workflowInstanceId,
billingCustomer,
projectId,
+ startUrl,
config,
- allPages,
- lighthouseResults,
+ crawl,
} = args;
- await pgStep(step, "finalize", undefined, async () => {
+ await pgStep(step, "multipage-checks", undefined, async () => {
await AuditRepository.updateAuditProgress(auditId, workflowInstanceId, {
currentPhase: "finalizing",
});
- await AuditRepository.batchWriteResults(
+
+ // Integrity guard: pages are persisted inside crawl-batch steps. If the
+ // crawl claims pages but D1 has none (e.g. an instance started under the
+ // pre-incremental-persistence code was replayed under this code), fail
+ // loudly instead of completing with an empty audit.
+ if (
+ crawl.pages.length > 0 &&
+ !(await AuditRepository.hasPagesForAudit(auditId))
+ ) {
+ throw new Error(
+ `Audit ${auditId}: crawl reported ${crawl.pages.length} pages but none were persisted`,
+ );
+ }
+
+ const issues = await runMultipageChecks({
auditId,
- allPages,
- lighthouseResults,
- );
+ startUrl,
+ crawlCompleted: crawl.completed,
+ });
+ await AuditRepository.insertIssues(auditId, issues);
+ return { issueCount: issues.length };
+ });
+
+ await pgStep(step, "finalize", undefined, async () => {
await AuditRepository.completeAudit(auditId, workflowInstanceId, {
- pagesCrawled: allPages.length,
- pagesTotal: allPages.length,
+ pagesCrawled: crawl.pages.length,
+ pagesTotal: crawl.pages.length,
});
await captureServerEvent({
distinctId: billingCustomer.userId,
@@ -313,8 +299,12 @@ async function finalizeAudit(args: {
properties: {
project_id: projectId,
status: "completed",
- pages_crawled: allPages.length,
- pages_total: allPages.length,
+ pages_crawled: crawl.pages.length,
+ pages_total: crawl.pages.length,
+ crawl_completed: crawl.completed,
+ pages_blocked: crawl.pages.filter(
+ (page) => page.fetchClass === "blocked",
+ ).length,
run_lighthouse: config.lighthouseStrategy !== "none",
},
});
diff --git a/src/serverFunctions/audit.ts b/src/serverFunctions/audit.ts
index 050573e8..eb56fefc 100644
--- a/src/serverFunctions/audit.ts
+++ b/src/serverFunctions/audit.ts
@@ -1,14 +1,7 @@
import { createServerFn } from "@tanstack/react-start";
import { waitUntil } from "cloudflare:workers";
import { AuditService } from "@/server/features/audit/services/AuditService";
-import type { AuditLimitTier } from "@/server/features/audit/services/audit-capacity";
-import {
- customerHasManagedAccess,
- customerHasPaidPlan,
-} from "@/server/billing/subscription";
-import { AppError } from "@/server/lib/errors";
import { captureServerEvent } from "@/server/lib/posthog";
-import { isHostedServerAuthMode } from "@/server/lib/runtime-env";
import { requireProjectContext } from "@/serverFunctions/middleware";
import {
deleteAuditSchema,
@@ -23,21 +16,9 @@ export const startAudit = createServerFn({ method: "POST" })
.middleware(requireProjectContext)
.validator(startAuditSchema)
.handler(async ({ data, context }) => {
- // The crawler runs on our Workers compute and isn't credit-metered, so
- // plan-tier limits are the abuse bound in hosted mode: free accounts get
- // one small audit at a time, paid keeps the full limits, and customers
- // with no Autumn product at all are turned away. Self-hosted isn't gated.
- let limitTier: AuditLimitTier = "paid";
- if (await isHostedServerAuthMode()) {
- const [hasManagedAccess, hasPaidPlan] = await Promise.all([
- customerHasManagedAccess(context.organizationId),
- customerHasPaidPlan(context.organizationId),
- ]);
- if (!hasManagedAccess) {
- throw new AppError("PAYMENT_REQUIRED", "Subscribe to run site audits");
- }
- limitTier = hasPaidPlan ? "paid" : "free";
- }
+ const limitTier = await AuditService.resolveAuditLimitTier(
+ context.organizationId,
+ );
const result = await AuditService.startAudit({
actorUserId: context.userId,
diff --git a/src/shared/audit-issues.ts b/src/shared/audit-issues.ts
new file mode 100644
index 00000000..c3f13bad
--- /dev/null
+++ b/src/shared/audit-issues.ts
@@ -0,0 +1,235 @@
+/**
+ * Registry of site-audit issue types.
+ *
+ * Shared between the server (issue engine, MCP tools) and the client
+ * (issues UI, CSV export). Each issue row in `audit_issues` references one
+ * of these types by id.
+ */
+
+type IssueSeverity = "critical" | "warning" | "info";
+
+interface AuditIssueDescriptor {
+ severity: IssueSeverity;
+ title: string;
+ explanation: string;
+ howToFix: string;
+}
+
+export const AUDIT_ISSUE_TYPES = {
+ "blocked-page": {
+ severity: "critical",
+ title: "Crawler was blocked",
+ explanation:
+ "The site returned a bot challenge or access denial (e.g. a Cloudflare challenge, 403, or 429) instead of the page. We report this honestly rather than pretending the page is broken — but it means this page could not be audited, and other crawlers like search engines may face similar friction.",
+ howToFix:
+ 'If you own this site, allowlist the "OpenSEO-Audit" user agent in your WAF/bot-protection settings (on Cloudflare: a WAF custom rule that skips bot protection when the user agent contains "OpenSEO-Audit"; on some free tiers you may need to relax bot protection). Then re-run the audit.',
+ },
+ "server-error": {
+ severity: "critical",
+ title: "Server error (5xx)",
+ explanation:
+ "The page returned a 5xx server error. Search engines that repeatedly see server errors will crawl the site less and may drop the page from the index.",
+ howToFix:
+ "Check the server logs for this URL and fix the underlying error. If the page is gone, return a 404/410 or redirect it to a relevant page instead of erroring.",
+ },
+ "broken-internal-link": {
+ severity: "critical",
+ title: "Broken internal link",
+ explanation:
+ "This page links to an internal URL that returns an error status (4xx/5xx). Broken links waste crawl budget, leak link equity, and frustrate users — they are among the most common and most damaging technical SEO issues.",
+ howToFix:
+ "Update the link to point at the correct live URL, or remove it. If the target was moved, prefer linking directly to the new URL rather than relying on a redirect.",
+ },
+ "missing-title": {
+ severity: "critical",
+ title: "Missing title tag",
+ explanation:
+ "The page has no <title>. The title is the strongest on-page relevance signal and the headline shown in search results; without it search engines generate one themselves, usually badly.",
+ howToFix:
+ "Add a unique, descriptive <title> of roughly 50–60 characters that includes the page's primary topic.",
+ },
+ "broken-page": {
+ severity: "warning",
+ title: "Page returns an error (4xx)",
+ explanation:
+ "This crawled URL returned a client error (e.g. 404). If it is referenced from your sitemap or other pages, crawlers keep wasting requests on it.",
+ howToFix:
+ "If the page should exist, restore it. If it is intentionally gone, remove it from the sitemap and internal links, and consider a 301 redirect to the closest live page.",
+ },
+ "duplicate-title": {
+ severity: "warning",
+ title: "Duplicate title",
+ explanation:
+ "Multiple pages share the same title tag. Search engines use titles to differentiate pages; duplicates make pages compete with each other and depress click-through rates.",
+ howToFix:
+ "Write a unique title for each page describing its specific content. For templated pages, include the distinguishing attribute (name, category, location) in the template.",
+ },
+ "duplicate-meta-description": {
+ severity: "warning",
+ title: "Duplicate meta description",
+ explanation:
+ "Multiple pages share the same meta description, so search results show identical snippets and users cannot tell the pages apart.",
+ howToFix:
+ "Write a unique meta description per page, or remove the duplicated one entirely — search engines will generate a snippet from page content, which beats a wrong duplicate.",
+ },
+ "duplicate-content": {
+ severity: "warning",
+ title: "Duplicate page content",
+ explanation:
+ "Two or more URLs serve byte-identical visible text. Search engines pick one version to index and ignore the rest, and ranking signals get split across the duplicates.",
+ howToFix:
+ "Consolidate duplicates: pick the canonical URL, add rel=canonical from the others, and 301-redirect duplicate URLs where possible (common causes: trailing-slash variants, URL parameters, http/https or www variants).",
+ },
+ "missing-meta-description": {
+ severity: "warning",
+ title: "Missing meta description",
+ explanation:
+ "The page has no meta description. Search engines will assemble a snippet from page text, which is often less compelling and hurts click-through rate.",
+ howToFix:
+ "Add a meta description of roughly 70–160 characters that summarizes the page and gives a reason to click.",
+ },
+ "missing-h1": {
+ severity: "warning",
+ title: "Missing H1 heading",
+ explanation:
+ "The page has no H1. The H1 tells users and search engines what the page is about; pages without one tend to have weaker topical clarity.",
+ howToFix:
+ "Add a single H1 that states the page's main topic, consistent with the title tag.",
+ },
+ "multiple-h1": {
+ severity: "warning",
+ title: "Multiple H1 headings",
+ explanation:
+ "The page has more than one H1, which dilutes the main-topic signal and usually indicates a templating mistake (e.g. a logo and a headline both marked up as H1).",
+ howToFix:
+ "Keep one H1 for the page's main heading and demote the others to H2/H3 (or unstyled elements for non-headings like logos).",
+ },
+ "redirect-chain": {
+ severity: "warning",
+ title: "Redirect chain",
+ explanation:
+ "Reaching the final page requires two or more consecutive redirects. Each hop adds latency, leaks link equity, and burns crawl budget; long chains may not be followed at all.",
+ howToFix:
+ "Point the first URL (and any internal links) directly at the final destination so there is at most one redirect.",
+ },
+ "redirect-loop": {
+ severity: "warning",
+ title: "Redirect loop",
+ explanation:
+ "This redirect eventually points back to itself, so the URL never resolves. Browsers and crawlers give up with an error.",
+ howToFix:
+ "Trace the redirect rules for this URL and break the cycle so the chain terminates at a real 200 page.",
+ },
+ "canonical-conflict": {
+ severity: "warning",
+ title: "Conflicting canonical signals",
+ explanation:
+ "The page declares different canonical URLs in its HTML <link rel=canonical> and its HTTP Link header. When signals conflict, search engines ignore both and choose their own canonical.",
+ howToFix:
+ "Pick one canonical URL and declare it in exactly one place (HTML head is the most common); remove or align the other declaration.",
+ },
+ "thin-content": {
+ severity: "warning",
+ title: "Thin content",
+ explanation:
+ "The page has very little visible text. Thin pages rarely rank, can drag down sitewide quality assessments, and (if the site renders client-side) may indicate content invisible to plain-HTML crawlers.",
+ howToFix:
+ "Either expand the page with genuinely useful content, noindex it, or consolidate it into a stronger page. If the content exists but is rendered by JavaScript, ensure it is server-rendered or pre-rendered.",
+ },
+ "images-missing-alt": {
+ severity: "warning",
+ title: "Images missing alt text",
+ explanation:
+ "One or more images on the page lack alt attributes. Alt text is an accessibility requirement and the main way search engines understand images.",
+ howToFix:
+ 'Add descriptive alt text to meaningful images; use an empty alt (alt="") only for purely decorative ones.',
+ },
+ "orphan-page": {
+ severity: "warning",
+ title: "Orphan page",
+ explanation:
+ "No crawled page links to this URL — it was only discoverable via the sitemap. Pages without internal links receive little crawl attention and no internal link equity, and users can't find them by browsing.",
+ howToFix:
+ "Link to this page from relevant pages (navigation, related content, hub pages), or remove it from the sitemap if it shouldn't be indexed.",
+ },
+ "title-too-long": {
+ severity: "info",
+ title: "Title too long",
+ explanation:
+ "The title exceeds ~60 characters, so search results will truncate it and the ending may be cut off mid-phrase.",
+ howToFix:
+ "Shorten the title to roughly 50–60 characters, front-loading the most important words.",
+ },
+ "title-too-short": {
+ severity: "info",
+ title: "Title too short",
+ explanation:
+ "The title is under ~10 characters, which is usually too generic to describe the page or attract clicks.",
+ howToFix:
+ "Expand the title into a descriptive phrase (roughly 30–60 characters) that states what the page offers.",
+ },
+ "meta-description-too-long": {
+ severity: "info",
+ title: "Meta description too long",
+ explanation:
+ "The meta description exceeds ~160 characters, so search engines will truncate the snippet.",
+ howToFix:
+ "Trim the description to roughly 70–160 characters while keeping the core message and call to action.",
+ },
+ "heading-order-skip": {
+ severity: "info",
+ title: "Heading levels skip",
+ explanation:
+ "The heading hierarchy skips levels (e.g. an H4 directly after an H2). This weakens document structure for accessibility tools and content parsing.",
+ howToFix:
+ "Adjust heading levels so they descend one step at a time (H1 → H2 → H3) without skipping.",
+ },
+ "slow-response": {
+ severity: "info",
+ title: "Slow server response",
+ explanation:
+ "The HTML response took over 1.5 seconds. Slow time-to-first-byte drags down every downstream performance metric and reduces crawl rate on large sites.",
+ howToFix:
+ "Investigate server/database time and caching for this route; serving cached or statically generated HTML usually fixes it.",
+ },
+ "noindex-page": {
+ severity: "info",
+ title: "Page is noindex",
+ explanation:
+ "The page asks search engines not to index it (via robots meta tag or X-Robots-Tag header). That's often intentional — this is a heads-up, not an error.",
+ howToFix:
+ "If this page should rank, remove the noindex directive. If it's intentional (admin, thank-you, filter pages), no action is needed.",
+ },
+ "canonicalized-page": {
+ severity: "info",
+ title: "Canonicalized to another URL",
+ explanation:
+ "The page declares a different URL as its canonical, telling search engines to index that URL instead. Fine when intentional (parameter pages, syndication) — a problem if this page was meant to rank.",
+ howToFix:
+ "If this page should rank on its own, set its canonical to itself. Otherwise no action is needed.",
+ },
+ "deep-page": {
+ severity: "info",
+ title: "Page is deep in the site structure",
+ explanation:
+ "The page is 5+ clicks from the homepage. Deep pages get crawled less often and receive less link equity.",
+ howToFix:
+ "Add links from higher-level pages (hubs, category pages, navigation) to flatten the path to this page.",
+ },
+} as const satisfies Record<string, AuditIssueDescriptor>;
+
+export type AuditIssueType = keyof typeof AUDIT_ISSUE_TYPES;
+
+export const ISSUE_SEVERITY_ORDER: Record<IssueSeverity, number> = {
+ critical: 0,
+ warning: 1,
+ info: 2,
+};
+
+const issueRegistry: Record<string, AuditIssueDescriptor> = AUDIT_ISSUE_TYPES;
+
+export function getIssueDescriptor(
+ issueType: string,
+): AuditIssueDescriptor | null {
+ return issueRegistry[issueType] ?? null;
+}
diff --git a/src/types/schemas/audit.ts b/src/types/schemas/audit.ts
index b12c783c..864f3f4b 100644
--- a/src/types/schemas/audit.ts
+++ b/src/types/schemas/audit.ts
@@ -17,10 +17,7 @@ export const startAuditSchema = z.object({
.max(PAID_MAX_AUDIT_PAGES)
.optional()
.default(DEFAULT_AUDIT_PAGES),
- lighthouseStrategy: z
- .enum(["auto", "manual", "none"])
- .optional()
- .default("auto"),
+ lighthouseStrategy: z.enum(["auto", "none"]).optional().default("auto"),
});
export const getAuditStatusSchema = z.object({