π¬ A browser extension and desktop app that turns your clicks into step-by-step guides with annotated screenshots π
TypeScript browser-extensionchrome-extensiondocumentationfirefox-extensionguide-generatorlocal-firstmanifest-v3privacy-firstproductivityreactscreenshotstep-by-step-guidetutorialtypescriptworkflowwxt
docs: update AGENTS.md with current architecture
Luis Ortiz0f0cbc91855e
diff --git a/AGENTS.md b/AGENTS.mdindex 6d9f330..66f8f48 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -22,12 +22,32 @@ You click "Record," perform a workflow in your browser, and Mimik automatically```src/βββ core/ # Business logic (no UI dependencies)-β βββ capture/ # Recording pipeline, event detection, screenshots+β βββ capture/ # Recording pipeline+β β βββ ai/ # AI description + title generation (Vercel AI SDK)+β β β βββ description.ts # getAIDescription (DOM context β AI β step text)+β β β βββ title.ts # generateGuideTitle (steps β AI β guide name)+β β β βββ models.ts # AI_PROVIDERS config (OpenAI/Anthropic model lists)+β β β βββ prompts.ts # Prompt templates+β β β βββ provider.ts # createModel factory (OpenAI/Anthropic)+β β βββ dom/ # DOM extraction utilities+β β β βββ context.ts # DOMContext extraction + serialization+β β β βββ element-meta.ts # extractElementMeta (selector, text, aria, rect)+β β β βββ element-utils.ts # findFocusableAncestor, isTextField, etc.+β β βββ events/ # Event capture system+β β β βββ handlers.ts # CaptureController class + startCapture+β β β βββ highlight.ts # HighlightManager (dashed border overlay)+β β β βββ input-session.ts # InputSession (typing lifecycle)+β β βββ machine.ts # xstate capture state machine+β β βββ rrweb-recorder.ts # DOM recording for replay+β β βββ session.ts # CaptureSession (lifecycle manager)+β β βββ spa-nav.ts # SPA navigation tracking+β β βββ start-notification.ts # Recording notification overlay+β β βββ step-description.ts # Fallback rule-based descriptionsβ βββ export/ # HTML, PDF, Markdown export generatorsβ βββ guides/ # Data layer: types, Dexie DB, CRUD serviceβββ entrypoints/ # Chrome extension entry points (WXT)β βββ background/ # Service worker: state machine, message handlers, tab management-β βββ content.ts # Content script: event capture, overlay, rrweb recording+β βββ content.ts # Content script: CaptureSession, event listeners, rrwebβ βββ sidepanel/ # Side panel React mountβ βββ fullview/ # Full-page view React mountβ βββ options/ # Settings page React mount@@ -59,6 +79,8 @@ src/β βββ ExportMenu.tsxβ βββ BlurCanvas.tsxβ βββ ZoomScreenshot.tsx+ βββ shared/ # Shared UI components+ β βββ SettingsView.tsx # AI settings (provider, model, API key)βββ options/ # Settings pageβββ App.tsx```@@ -72,16 +94,17 @@ src/| Persistence | Dexie (IndexedDB) | Guides, steps, screenshots, rrweb chunks || Service worker recovery | sessionStorage | xstate machine snapshot persistence || Background β Sidepanel | Port messaging | Real-time state broadcast |+| Cross-context sync | BroadcastChannel | Guide mutations (star, delete) across sidepanel/fullview |## Extension Entry Points| Entry Point | File | Purpose ||-------------|------|---------|-| Background | `entrypoints/background/` | Service worker: xstate actor, message handlers, tab management, navigation tracking |-| Content Script | `entrypoints/content.ts` | Injected into all tabs: CaptureSession, event listeners, rrweb, highlight overlay |-| Side Panel | `entrypoints/sidepanel/` | Recording controls, library, guide editor |+| Background | `entrypoints/background/` | Service worker: xstate actor, message handlers, tab management |+| Content Script | `entrypoints/content.ts` | Injected into all tabs: CaptureSession, event listeners, rrweb |+| Side Panel | `entrypoints/sidepanel/` | Recording controls, library, guide editor, settings || Full View | `entrypoints/fullview/` | Dashboard: library browse, guide viewer, Ctrl+K search |-| Options | `entrypoints/options/` | AI API key settings (OpenAI / Anthropic) |+| Options | `entrypoints/options/` | Settings page (shared SettingsView in centered card) |## Messaging@@ -92,13 +115,15 @@ Content Script ββ Background Service Worker ββ Sidepanel / Fullview**Extension messages** (webext-core, `lib/messaging.ts`):- `getState` β current capture state, step count, guide ID- `startRecording({url})` β creates guide, returns guideId-- `stopRecording()` β finalizes guide-- `userAction({guideId, action, elementMeta})` β processes captured step+- `stopRecording()` β finalizes guide, generates AI title+- `captureStep({guideId, action, elementMeta, domContext})` β screenshots + creates step+- `updateInputStep({stepId, description})` β updates typing step description+- `finalizeInputStep({stepId, elementMeta, domContext})` β final screenshot + AI description- `rrwebChunk({guideId, events, timestamp})` β stores DOM recording chunk**Tab messages** (content script β background, `lib/tab-messages.ts`):- `PING` / `START_CAPTURE` / `STOP_CAPTURE` β lifecycle-- `HIDE_OVERLAY` / `SHOW_OVERLAY` β overlay toggle before/after screenshot+- `HIDE_OVERLAY` / `SHOW_OVERLAY` β overlay toggle (fallback, used for non-click captures)- `SHOW_NOTIFICATION` β "Recording started" overlay- `URL_CHANGED` / `GET_ROUTE` β SPA navigation tracking@@ -108,21 +133,47 @@ Content Script ββ Background Service Worker ββ Sidepanel / Fullview1. User clicks "Start Capture" in sidepanel2. Background transitions xstate machine IDLE β RECORDING3. Creates Guide in IndexedDB, broadcasts `START_CAPTURE` to all tabs-4. Content scripts initialize CaptureSession (event listeners + rrweb)+4. Content scripts create CaptureSession β CaptureController (event listeners) + rrweb5. Shows recording notification overlay on active tab-**Capture a step:**-1. Content script detects user action (click, input, drag)-2. Extracts ElementMeta (selector, text, aria labels, position)-3. Sends `userAction` to background-4. Background hides overlay β `captureVisibleTab` β shows overlay-5. Optionally sends screenshot to AI for description (OpenAI/Anthropic)-6. Saves Step + Screenshot to IndexedDB+**Capture a click:**+1. Content script's CaptureController detects click via DOM event listener+2. Click handler pushes async work into PQueue (concurrency: 1)+3. Queue processes: hides overlay instantly β sends `captureStep` to background β waits+4. Background calls `captureVisibleTab` (overlay is hidden, page hasn't reacted yet)+5. Saves Screenshot + Step to IndexedDB+6. Optionally generates AI description from DOM context text (not screenshot)+7. Returns `{ stepId }` β content script shows overlay β queue processes next event++**Capture text input (typing):**+1. Click on text field β CaptureController starts InputSession β `captureStep` with initial screenshot+2. Each keystroke β InputSession.update() β `updateInputStep` (description only, fire-and-forget)+3. Enter/Escape/focusout β InputSession.finalize() β `finalizeInputStep` β final screenshot replaces initial + AI description+4. Result: one step for entire typing interaction**Stop recording:**1. Background transitions RECORDING β IDLE-2. Broadcasts `STOP_CAPTURE`, content scripts flush rrweb events-3. Opens fullview dashboard with the guide+2. Broadcasts `STOP_CAPTURE`, content scripts flush pending input sessions + rrweb events+3. Background generates guide title from step descriptions + URLs via AI+4. Opens fullview dashboard with the guide++## DOM Context (AI Input)++Instead of sending screenshots to the AI for step descriptions, Mimik extracts a lightweight DOM context (~50-100 tokens) from around the target element:++```+Page: "Public profile - Settings" /settings/profile+Container: form "Public profile"+Heading: "Public profile"+Siblings: input "Name", input "Email", textarea "Bio", button "Update profile"+β Target: button "Update profile" (click)+```++Extraction walks up from the target element to find:+- Page title + URL path+- Nearest semantic container (form, nav, dialog, section) or 3 levels up+- Nearest heading+- Sibling interactive elements in the same container (max 10)## Export Formats@@ -147,7 +198,8 @@ Content Script ββ Background Service Worker ββ Sidepanel / Fullview| Messaging | webext-core || Session recording | rrweb || Export | jsPDF, client-side HTML/Markdown |-| AI (optional) | OpenAI SDK, Anthropic SDK |+| AI (optional) | Vercel AI SDK (`ai`, `@ai-sdk/openai`, `@ai-sdk/anthropic`) |+| Event queue | p-queue (concurrency: 1) || Icons | Lucide React || Dates | dayjs || DOM utils | css-selector-generator |@@ -172,10 +224,13 @@ Font: Poppins (loaded via `@fontsource/poppins`).## Key Technical Details-- **Screenshot capture** uses `chrome.tabs.captureVisibleTab` with overlay hidden to avoid capturing the highlight-- **Event deduplication** merges fast clicks within 300ms on same element; max 4 clicks per 800ms window+- **Async event queue** in content script (PQueue, concurrency 1) serializes capture work β each action awaits the full background round-trip before the next starts+- **Overlay hidden from content script side** (instant `display:none`) before sending capture message β no round-trip delay for overlay toggle+- **Input session** aggregates all typing on a field into one step β click creates it, keystrokes update description, finalize takes final screenshot+- **DOM context** sent as text to AI instead of screenshots β 15-30x cheaper per step- **Highlight overlay** uses a custom web component (`<mimik-highlight>`) with closed Shadow DOM at max z-index- **Content script injection** pings first, falls back to `chrome.scripting.executeScript()` for tabs without the script- **xstate snapshot** persisted to sessionStorage so the state machine survives service worker restarts- **Recording notification** uses `animationend` event (not hardcoded delays) for timing- **Font loading** uses `@fontsource/poppins` (CSP-safe, no CDN dependency)+- **Cross-context sync** via BroadcastChannel β star/delete events update other views without full reload