π¬ A browser extension and desktop app that turns your clicks into step-by-step guides with annotated screenshots π
TypeScript browser-extensionchrome-extensiondocumentationfirefox-extensionguide-generatorlocal-firstmanifest-v3privacy-firstproductivityreactscreenshotstep-by-step-guidetutorialtypescriptworkflowwxt
Restore AGENTS.MD file
Alvin1777b8daab85
diff --git a/AGENTS.md b/AGENTS.mdindex 47dc3e3..13cc107 100755--- a/AGENTS.md+++ b/AGENTS.md@@ -1 +1,247 @@-AGENTS.md\ No newline at end of file+# Mimik++Open-source Chrome extension that auto-captures browser workflows and generates step-by-step guides. No backend, no account, no data leaves the browser.++## What It Does++You click "Record," perform a workflow in your browser, and Mimik automatically captures each action as a step with an annotated screenshot and description. You can edit the guide, replay it on a live page, or export it as a file.++**Core loop: Record β Edit β Replay or Export.**++## Architecture++**Everything runs in the Chrome extension. No backend.**++- Storage: IndexedDB via Dexie.js (browser-local)+- AI descriptions: optional, user provides their own API key in settings+- Export: generated client-side (no server rendering)+- No auth, no database, no hosting, no Docker++### Directory Structure++```+src/+βββ core/ # Business logic (no UI dependencies)+β βββ capture/ # Recording pipeline+β β βββ ai/ # AI description + title generation (Vercel AI SDK)+β β β βββ description.ts # getAIDescription (DOM context β AI β step text)+β β β βββ title.ts # generateGuideTitle (steps β AI β guide name)+β β β βββ models.ts # AI_PROVIDERS config (OpenAI/Anthropic model lists)+β β β βββ prompts.ts # Prompt templates+β β β βββ provider.ts # createModel factory (OpenAI/Anthropic)+β β βββ dom/ # DOM extraction utilities+β β β βββ context.ts # DOMContext extraction + serialization+β β β βββ element-meta.ts # extractElementMeta (selector, text, aria, rect)+β β β βββ element-utils.ts # findFocusableAncestor, isTextField, etc.+β β βββ events/ # Event capture system+β β β βββ handlers.ts # CaptureController class + startCapture+β β β βββ input-session.ts # InputSession (typing lifecycle)+β β βββ machine.ts # xstate capture state machine+β β βββ session.ts # CaptureSession (lifecycle manager)+β β βββ spa-nav.ts # SPA navigation tracking+β β βββ start-notification.ts # Recording notification overlay+β β βββ step-description.ts # Fallback rule-based descriptions+β βββ blur/ # Smart blur: regex presets, DOM scanner, element picker, panel UI+β βββ export/ # HTML, PDF, DOCX, Markdown, video generators + shared utils+β βββ guides/ # Data layer: types, Dexie DB, CRUD service+βββ entrypoints/ # Chrome extension entry points (WXT)+β βββ background/ # Service worker: state machine, message handlers, tab management+β βββ content.ts # Content script: CaptureSession, event listeners+β βββ sidepanel/ # Side panel React mount+β βββ fullview/ # Full-page view React mount+β βββ onboarding/ # Onboarding wizard (opens on first install)+β βββ options/ # Settings page React mount+βββ lib/ # Shared utilities+β βββ messaging.ts # Extension messaging protocol (webext-core)+β βββ port.ts # Long-lived port: background β sidepanel+β βββ browser-api.ts # Chrome API wrappers+β βββ tab-messages.ts # Content script message types+β βββ logger.ts # Logging utility+β βββ utils.ts # Shared helpers (dates, URLs, cn)+βββ stores/ # Zustand state stores+β βββ fullview.ts # Fullview UI state (search, counts, guide data)+βββ ui/ # React components+ βββ components/ui/ # shadcn/ui primitives (button, input, dialog, badge)+ βββ fullview/ # Full-page dashboard+ β βββ components/ # Extracted sub-components (grid, list, search, etc.)+ β βββ App.tsx+ β βββ TopNav.tsx+ β βββ SearchModal.tsx+ β βββ GuideContent.tsx+ β βββ LibraryContent.tsx+ β βββ router.ts+ βββ sidepanel/ # Side panel UI+ β βββ App.tsx+ β βββ LibraryView.tsx+ β βββ GuideEditor.tsx+ β βββ RecordingView.tsx+ β βββ StepCard.tsx+ β βββ ExportMenu.tsx+ β βββ BlurCanvas.tsx+ β βββ ZoomScreenshot.tsx+ βββ onboarding/ # Onboarding wizard UI+ β βββ App.tsx # 5-step wizard (welcome, AI, blur, pin, done)+ βββ shared/ # Shared UI components+ β βββ SettingsView.tsx # AI settings (provider, model, API key)+ βββ options/ # Settings page+ βββ App.tsx+```++## State Management++| Layer | Tool | Purpose |+|-------|------|---------|+| Capture lifecycle | xstate | State machine (IDLE β RECORDING) in background service worker |+| Fullview UI | Zustand | Search modal, guide counts, active guide data |+| Persistence | Dexie (IndexedDB) | Guides, steps, screenshots, snapshots |+| Service worker recovery | sessionStorage | xstate machine snapshot persistence |+| Background β Sidepanel | Port messaging | Real-time state broadcast |+| Cross-context sync | BroadcastChannel | Guide mutations (star, delete) across sidepanel/fullview |++## Extension Entry Points++| Entry Point | File | Purpose |+|-------------|------|---------|+| Background | `entrypoints/background/` | Service worker: xstate actor, message handlers, tab management |+| Content Script | `entrypoints/content.ts` | Injected into all tabs: CaptureSession, event listeners |+| Side Panel | `entrypoints/sidepanel/` | Recording controls, library, guide editor, settings |+| Full View | `entrypoints/fullview/` | Dashboard: library browse, guide viewer, Ctrl+K search |+| Onboarding | `entrypoints/onboarding/` | First-install wizard: AI setup, smart blur, pin extension |+| Options | `entrypoints/options/` | Settings page (shared SettingsView in centered card) |++## Messaging++```+Content Script ββ Background Service Worker ββ Sidepanel / Fullview+```++**Extension messages** (webext-core, `lib/messaging.ts`):+- `getState` β current capture state, step count, guide ID+- `startRecording({url})` β creates guide, returns guideId+- `stopRecording()` β finalizes guide, generates AI title+- `captureStep({guideId, action, elementMeta, domContext})` β screenshots + creates step+- `updateInputStep({stepId, description})` β updates typing step description+- `finalizeInputStep({stepId, elementMeta, domContext})` β final screenshot + AI description++**Tab messages** (content script β background, `lib/tab-messages.ts`):+- `PING` / `START_CAPTURE` / `STOP_CAPTURE` β lifecycle+- `SHOW_NOTIFICATION` β "Recording started" overlay+- `URL_CHANGED` / `GET_ROUTE` β SPA navigation tracking++## Capture Pipeline++**Start recording:**+1. User clicks "Start Capture" in sidepanel+2. Background transitions xstate machine IDLE β RECORDING+3. Creates Guide in IndexedDB, broadcasts `START_CAPTURE` to all tabs+4. Content scripts create CaptureSession β CaptureController (event listeners)+5. Shows recording notification overlay on active tab++**Capture a click:**+1. Content script's CaptureController detects click via DOM event listener+2. Click handler pushes async work into PQueue (concurrency: 1)+3. Enqueueing hides the hover ring (`pointerdown` already did, on mouse paths); queue waits 3 frames then sends `captureStep`+4. Background calls `captureVisibleTab` (ring is hidden, page hasn't reacted yet)+5. Saves Screenshot + Step to IndexedDB+6. Queues the AI description from DOM context text (serialized in the background; the step carries `aiPending` until it lands) rather than awaiting it+7. Returns `{ stepId }` β queue processes next event; the ring returns only once the queue is fully drained++**Capture text input (typing):**+1. Click on text field β CaptureController starts InputSession β `captureStep` with initial screenshot+2. Each keystroke β InputSession.update() β `updateInputStep` (description only, fire-and-forget)+3. Enter/Escape/focusout β InputSession.finalize() β `finalizeInputStep` β final screenshot replaces initial + queued AI description+4. Result: one step for entire typing interaction++**Stop recording:**+1. Background transitions RECORDING β IDLE+2. Broadcasts `STOP_CAPTURE`, content scripts flush pending input sessions+3. Background drains queued step descriptions (20s cap), then generates the guide title from step descriptions + URLs via AI+4. Opens fullview dashboard with the guide++## DOM Context (AI Input)++Instead of sending screenshots to the AI for step descriptions, Mimik extracts a lightweight DOM context (~50-100 tokens) from around the target element:++```+Page: "Public profile - Settings" /settings/profile+Container: form "Public profile"+Heading: "Public profile"+Siblings: input "Name", input "Email", textarea "Bio", button "Update profile"+β Target: button "Update profile" (click)+```++Extraction walks up from the target element to find:+- Page title + URL path+- Nearest semantic container (form, nav, dialog, section) or 3 levels up+- Nearest heading+- Sibling interactive elements in the same container (max 10)++## Export Formats++| Format | Generator | Details |+|--------|-----------|---------|+| HTML | `core/export/html-export.ts` | Self-contained, base64 images, inline CSS |+| PDF | `core/export/pdf-export.ts` | jsPDF, A4 portrait, auto page breaks |+| Markdown | `core/export/markdown-export.ts` | Standard MD with base64 image data URLs |+| DOCX | `core/export/docx-export.ts` | Lazy-imported, Word-compatible |+| Video | `core/export/video-export.ts` | WebCodecs via mediabunny (lazy), mp4/H.264 with WebM/VP9 fallback |++Video frames reuse `renderScreenshot`, so the auto-crop, click-target outline, annotations and+redactions are already baked in. Each step holds 1.5s wide, eases into a crop around the target+over 0.73s, holds 3s close, and consecutive steps cross-dissolve over 0.33s at 30fps. Capability+probing lives in `video-support.ts`, which must stay free of mediabunny imports because both+export UIs load it eagerly.++## Tech Stack++| Layer | Technology |+|-------|-----------|+| Extension framework | WXT (Manifest V3) |+| Language | TypeScript |+| UI | React 19 |+| Styling | Tailwind CSS v4 |+| Components | shadcn/ui |+| State (capture) | xstate |+| State (UI) | Zustand |+| Storage | Dexie.js (IndexedDB) |+| Messaging | webext-core |+| Export | jsPDF, docx, mediabunny (WebCodecs video), client-side HTML/Markdown |+| AI (optional) | Vercel AI SDK (`ai`, `@ai-sdk/openai`, `@ai-sdk/anthropic`) |+| Event queue | p-queue (concurrency: 1) |+| Icons | Lucide React |+| Dates | dayjs |+| DOM utils | css-selector-generator |++## Design System++All colors are defined as CSS variables in `src/ui/global.css` and used via Tailwind classes:++| Token | Color | Usage |+|-------|-------|-------|+| `--color-foreground` | `#1E1B4B` | Primary text (deep navy) |+| `--color-muted-foreground` | `#6B7280` | Secondary text |+| `--color-border` | `#C7D2FE` | Borders, dividers (lavender) |+| `--color-secondary` | `#EEF2FF` | Light wash backgrounds |+| `--color-accent` | `#4F46E5` | Icons, links, focus rings, toggles, meters (indigo) β never a button fill |+| `--color-primary` | `#1E1B4B` | Primary buttons, badges, dark backgrounds |+| `--color-primary-foreground` | `#C7D2FE` | Text on dark backgrounds |+| `--color-lavender` | `#C7D2FE` | Soft accent |+| `--color-purple` | `#4F46E5` | Primary indigo |+| `--color-deep` | `#1E1B4B` | Deepest navy |+| `--color-violet` | `#38BDF8` | Sky blue accent |+| `--color-success` | `#059669` | Success green |++Font: Poppins (loaded via `@fontsource/poppins`).++## Key Technical Details++- **Async event queue** in content script (PQueue, concurrency 1) serializes capture work β each action awaits the background round-trip (screenshot + step write, not the AI description) before the next starts+- **Hover ring hidden when work is enqueued** (`CaptureController.enqueue`, instant `display:none`), and `show()` stays suppressed until the queue drains β a second click while the first capture is still in flight can't bring the ring back before the screenshot. `pointerdown` hides it earlier still, on top of `captureAction`'s 3-frame wait+- **Input session** aggregates all typing on a field into one step β click creates it, keystrokes update description, finalize takes final screenshot+- **DOM context** sent as text to AI instead of screenshots β 15-30x cheaper per step+- **Hover ring** (`lib/hover-ring.ts`) is a closed-Shadow-DOM host marked `data-mimik-ignore`, shared by recording and the blur picker (purple). Recording reads the user's `targetColor` so the live ring matches the dashed target baked into screenshots. Never drawn on `iframe`/`embed`/`object` β a capture inside a subframe can't hide the top frame's ring+- **Content script injection** pings first, falls back to `chrome.scripting.executeScript()` for tabs without the script+- **xstate snapshot** persisted to sessionStorage so the state machine survives service worker restarts+- **Recording notification** uses `animationend` event (not hardcoded delays) for timing+- **Font loading** uses `@fontsource/poppins` (CSP-safe, no CDN dependency)+- **Cross-context sync** via BroadcastChannel β star/delete events update other views without full reload\ No newline at end of file