Skip to content

armgit / mimik

Public Imported from GitHub
0

🎬 A browser extension and desktop app that turns your clicks into step-by-step guides with annotated screenshots πŸ“–

Restore AGENTS.MD file

Alvin1777b8daab85
diff --git a/AGENTS.md b/AGENTS.md
index 47dc3e3..13cc107 100755
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1 +1,247 @@
-AGENTS.md
\ No newline at end of file
+# Mimik
+
+Open-source Chrome extension that auto-captures browser workflows and generates step-by-step guides. No backend, no account, no data leaves the browser.
+
+## What It Does
+
+You click "Record," perform a workflow in your browser, and Mimik automatically captures each action as a step with an annotated screenshot and description. You can edit the guide, replay it on a live page, or export it as a file.
+
+**Core loop: Record β†’ Edit β†’ Replay or Export.**
+
+## Architecture
+
+**Everything runs in the Chrome extension. No backend.**
+
+- Storage: IndexedDB via Dexie.js (browser-local)
+- AI descriptions: optional, user provides their own API key in settings
+- Export: generated client-side (no server rendering)
+- No auth, no database, no hosting, no Docker
+
+### Directory Structure
+
+```
+src/
+β”œβ”€β”€ core/ # Business logic (no UI dependencies)
+β”‚ β”œβ”€β”€ capture/ # Recording pipeline
+β”‚ β”‚ β”œβ”€β”€ ai/ # AI description + title generation (Vercel AI SDK)
+β”‚ β”‚ β”‚ β”œβ”€β”€ description.ts # getAIDescription (DOM context β†’ AI β†’ step text)
+β”‚ β”‚ β”‚ β”œβ”€β”€ title.ts # generateGuideTitle (steps β†’ AI β†’ guide name)
+β”‚ β”‚ β”‚ β”œβ”€β”€ models.ts # AI_PROVIDERS config (OpenAI/Anthropic model lists)
+β”‚ β”‚ β”‚ β”œβ”€β”€ prompts.ts # Prompt templates
+β”‚ β”‚ β”‚ └── provider.ts # createModel factory (OpenAI/Anthropic)
+β”‚ β”‚ β”œβ”€β”€ dom/ # DOM extraction utilities
+β”‚ β”‚ β”‚ β”œβ”€β”€ context.ts # DOMContext extraction + serialization
+β”‚ β”‚ β”‚ β”œβ”€β”€ element-meta.ts # extractElementMeta (selector, text, aria, rect)
+β”‚ β”‚ β”‚ └── element-utils.ts # findFocusableAncestor, isTextField, etc.
+β”‚ β”‚ β”œβ”€β”€ events/ # Event capture system
+β”‚ β”‚ β”‚ β”œβ”€β”€ handlers.ts # CaptureController class + startCapture
+β”‚ β”‚ β”‚ └── input-session.ts # InputSession (typing lifecycle)
+β”‚ β”‚ β”œβ”€β”€ machine.ts # xstate capture state machine
+β”‚ β”‚ β”œβ”€β”€ session.ts # CaptureSession (lifecycle manager)
+β”‚ β”‚ β”œβ”€β”€ spa-nav.ts # SPA navigation tracking
+β”‚ β”‚ β”œβ”€β”€ start-notification.ts # Recording notification overlay
+β”‚ β”‚ └── step-description.ts # Fallback rule-based descriptions
+β”‚ β”œβ”€β”€ blur/ # Smart blur: regex presets, DOM scanner, element picker, panel UI
+β”‚ β”œβ”€β”€ export/ # HTML, PDF, DOCX, Markdown, video generators + shared utils
+β”‚ └── guides/ # Data layer: types, Dexie DB, CRUD service
+β”œβ”€β”€ entrypoints/ # Chrome extension entry points (WXT)
+β”‚ β”œβ”€β”€ background/ # Service worker: state machine, message handlers, tab management
+β”‚ β”œβ”€β”€ content.ts # Content script: CaptureSession, event listeners
+β”‚ β”œβ”€β”€ sidepanel/ # Side panel React mount
+β”‚ β”œβ”€β”€ fullview/ # Full-page view React mount
+β”‚ β”œβ”€β”€ onboarding/ # Onboarding wizard (opens on first install)
+β”‚ └── options/ # Settings page React mount
+β”œβ”€β”€ lib/ # Shared utilities
+β”‚ β”œβ”€β”€ messaging.ts # Extension messaging protocol (webext-core)
+β”‚ β”œβ”€β”€ port.ts # Long-lived port: background ↔ sidepanel
+β”‚ β”œβ”€β”€ browser-api.ts # Chrome API wrappers
+β”‚ β”œβ”€β”€ tab-messages.ts # Content script message types
+β”‚ β”œβ”€β”€ logger.ts # Logging utility
+β”‚ └── utils.ts # Shared helpers (dates, URLs, cn)
+β”œβ”€β”€ stores/ # Zustand state stores
+β”‚ └── fullview.ts # Fullview UI state (search, counts, guide data)
+└── ui/ # React components
+ β”œβ”€β”€ components/ui/ # shadcn/ui primitives (button, input, dialog, badge)
+ β”œβ”€β”€ fullview/ # Full-page dashboard
+ β”‚ β”œβ”€β”€ components/ # Extracted sub-components (grid, list, search, etc.)
+ β”‚ β”œβ”€β”€ App.tsx
+ β”‚ β”œβ”€β”€ TopNav.tsx
+ β”‚ β”œβ”€β”€ SearchModal.tsx
+ β”‚ β”œβ”€β”€ GuideContent.tsx
+ β”‚ β”œβ”€β”€ LibraryContent.tsx
+ β”‚ └── router.ts
+ β”œβ”€β”€ sidepanel/ # Side panel UI
+ β”‚ β”œβ”€β”€ App.tsx
+ β”‚ β”œβ”€β”€ LibraryView.tsx
+ β”‚ β”œβ”€β”€ GuideEditor.tsx
+ β”‚ β”œβ”€β”€ RecordingView.tsx
+ β”‚ β”œβ”€β”€ StepCard.tsx
+ β”‚ β”œβ”€β”€ ExportMenu.tsx
+ β”‚ β”œβ”€β”€ BlurCanvas.tsx
+ β”‚ └── ZoomScreenshot.tsx
+ β”œβ”€β”€ onboarding/ # Onboarding wizard UI
+ β”‚ └── App.tsx # 5-step wizard (welcome, AI, blur, pin, done)
+ β”œβ”€β”€ shared/ # Shared UI components
+ β”‚ └── SettingsView.tsx # AI settings (provider, model, API key)
+ └── options/ # Settings page
+ └── App.tsx
+```
+
+## State Management
+
+| Layer | Tool | Purpose |
+|-------|------|---------|
+| Capture lifecycle | xstate | State machine (IDLE ↔ RECORDING) in background service worker |
+| Fullview UI | Zustand | Search modal, guide counts, active guide data |
+| Persistence | Dexie (IndexedDB) | Guides, steps, screenshots, snapshots |
+| Service worker recovery | sessionStorage | xstate machine snapshot persistence |
+| Background β†’ Sidepanel | Port messaging | Real-time state broadcast |
+| Cross-context sync | BroadcastChannel | Guide mutations (star, delete) across sidepanel/fullview |
+
+## Extension Entry Points
+
+| Entry Point | File | Purpose |
+|-------------|------|---------|
+| Background | `entrypoints/background/` | Service worker: xstate actor, message handlers, tab management |
+| Content Script | `entrypoints/content.ts` | Injected into all tabs: CaptureSession, event listeners |
+| Side Panel | `entrypoints/sidepanel/` | Recording controls, library, guide editor, settings |
+| Full View | `entrypoints/fullview/` | Dashboard: library browse, guide viewer, Ctrl+K search |
+| Onboarding | `entrypoints/onboarding/` | First-install wizard: AI setup, smart blur, pin extension |
+| Options | `entrypoints/options/` | Settings page (shared SettingsView in centered card) |
+
+## Messaging
+
+```
+Content Script ←→ Background Service Worker ←→ Sidepanel / Fullview
+```
+
+**Extension messages** (webext-core, `lib/messaging.ts`):
+- `getState` β†’ current capture state, step count, guide ID
+- `startRecording({url})` β†’ creates guide, returns guideId
+- `stopRecording()` β†’ finalizes guide, generates AI title
+- `captureStep({guideId, action, elementMeta, domContext})` β†’ screenshots + creates step
+- `updateInputStep({stepId, description})` β†’ updates typing step description
+- `finalizeInputStep({stepId, elementMeta, domContext})` β†’ final screenshot + AI description
+
+**Tab messages** (content script ↔ background, `lib/tab-messages.ts`):
+- `PING` / `START_CAPTURE` / `STOP_CAPTURE` β€” lifecycle
+- `SHOW_NOTIFICATION` β€” "Recording started" overlay
+- `URL_CHANGED` / `GET_ROUTE` β€” SPA navigation tracking
+
+## Capture Pipeline
+
+**Start recording:**
+1. User clicks "Start Capture" in sidepanel
+2. Background transitions xstate machine IDLE β†’ RECORDING
+3. Creates Guide in IndexedDB, broadcasts `START_CAPTURE` to all tabs
+4. Content scripts create CaptureSession β†’ CaptureController (event listeners)
+5. Shows recording notification overlay on active tab
+
+**Capture a click:**
+1. Content script's CaptureController detects click via DOM event listener
+2. Click handler pushes async work into PQueue (concurrency: 1)
+3. Enqueueing hides the hover ring (`pointerdown` already did, on mouse paths); queue waits 3 frames then sends `captureStep`
+4. Background calls `captureVisibleTab` (ring is hidden, page hasn't reacted yet)
+5. Saves Screenshot + Step to IndexedDB
+6. Queues the AI description from DOM context text (serialized in the background; the step carries `aiPending` until it lands) rather than awaiting it
+7. Returns `{ stepId }` β†’ queue processes next event; the ring returns only once the queue is fully drained
+
+**Capture text input (typing):**
+1. Click on text field β†’ CaptureController starts InputSession β†’ `captureStep` with initial screenshot
+2. Each keystroke β†’ InputSession.update() β†’ `updateInputStep` (description only, fire-and-forget)
+3. Enter/Escape/focusout β†’ InputSession.finalize() β†’ `finalizeInputStep` β†’ final screenshot replaces initial + queued AI description
+4. Result: one step for entire typing interaction
+
+**Stop recording:**
+1. Background transitions RECORDING β†’ IDLE
+2. Broadcasts `STOP_CAPTURE`, content scripts flush pending input sessions
+3. Background drains queued step descriptions (20s cap), then generates the guide title from step descriptions + URLs via AI
+4. Opens fullview dashboard with the guide
+
+## DOM Context (AI Input)
+
+Instead of sending screenshots to the AI for step descriptions, Mimik extracts a lightweight DOM context (~50-100 tokens) from around the target element:
+
+```
+Page: "Public profile - Settings" /settings/profile
+Container: form "Public profile"
+Heading: "Public profile"
+Siblings: input "Name", input "Email", textarea "Bio", button "Update profile"
+β†’ Target: button "Update profile" (click)
+```
+
+Extraction walks up from the target element to find:
+- Page title + URL path
+- Nearest semantic container (form, nav, dialog, section) or 3 levels up
+- Nearest heading
+- Sibling interactive elements in the same container (max 10)
+
+## Export Formats
+
+| Format | Generator | Details |
+|--------|-----------|---------|
+| HTML | `core/export/html-export.ts` | Self-contained, base64 images, inline CSS |
+| PDF | `core/export/pdf-export.ts` | jsPDF, A4 portrait, auto page breaks |
+| Markdown | `core/export/markdown-export.ts` | Standard MD with base64 image data URLs |
+| DOCX | `core/export/docx-export.ts` | Lazy-imported, Word-compatible |
+| Video | `core/export/video-export.ts` | WebCodecs via mediabunny (lazy), mp4/H.264 with WebM/VP9 fallback |
+
+Video frames reuse `renderScreenshot`, so the auto-crop, click-target outline, annotations and
+redactions are already baked in. Each step holds 1.5s wide, eases into a crop around the target
+over 0.73s, holds 3s close, and consecutive steps cross-dissolve over 0.33s at 30fps. Capability
+probing lives in `video-support.ts`, which must stay free of mediabunny imports because both
+export UIs load it eagerly.
+
+## Tech Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Extension framework | WXT (Manifest V3) |
+| Language | TypeScript |
+| UI | React 19 |
+| Styling | Tailwind CSS v4 |
+| Components | shadcn/ui |
+| State (capture) | xstate |
+| State (UI) | Zustand |
+| Storage | Dexie.js (IndexedDB) |
+| Messaging | webext-core |
+| Export | jsPDF, docx, mediabunny (WebCodecs video), client-side HTML/Markdown |
+| AI (optional) | Vercel AI SDK (`ai`, `@ai-sdk/openai`, `@ai-sdk/anthropic`) |
+| Event queue | p-queue (concurrency: 1) |
+| Icons | Lucide React |
+| Dates | dayjs |
+| DOM utils | css-selector-generator |
+
+## Design System
+
+All colors are defined as CSS variables in `src/ui/global.css` and used via Tailwind classes:
+
+| Token | Color | Usage |
+|-------|-------|-------|
+| `--color-foreground` | `#1E1B4B` | Primary text (deep navy) |
+| `--color-muted-foreground` | `#6B7280` | Secondary text |
+| `--color-border` | `#C7D2FE` | Borders, dividers (lavender) |
+| `--color-secondary` | `#EEF2FF` | Light wash backgrounds |
+| `--color-accent` | `#4F46E5` | Icons, links, focus rings, toggles, meters (indigo) β€” never a button fill |
+| `--color-primary` | `#1E1B4B` | Primary buttons, badges, dark backgrounds |
+| `--color-primary-foreground` | `#C7D2FE` | Text on dark backgrounds |
+| `--color-lavender` | `#C7D2FE` | Soft accent |
+| `--color-purple` | `#4F46E5` | Primary indigo |
+| `--color-deep` | `#1E1B4B` | Deepest navy |
+| `--color-violet` | `#38BDF8` | Sky blue accent |
+| `--color-success` | `#059669` | Success green |
+
+Font: Poppins (loaded via `@fontsource/poppins`).
+
+## Key Technical Details
+
+- **Async event queue** in content script (PQueue, concurrency 1) serializes capture work β€” each action awaits the background round-trip (screenshot + step write, not the AI description) before the next starts
+- **Hover ring hidden when work is enqueued** (`CaptureController.enqueue`, instant `display:none`), and `show()` stays suppressed until the queue drains β€” a second click while the first capture is still in flight can't bring the ring back before the screenshot. `pointerdown` hides it earlier still, on top of `captureAction`'s 3-frame wait
+- **Input session** aggregates all typing on a field into one step β€” click creates it, keystrokes update description, finalize takes final screenshot
+- **DOM context** sent as text to AI instead of screenshots β€” 15-30x cheaper per step
+- **Hover ring** (`lib/hover-ring.ts`) is a closed-Shadow-DOM host marked `data-mimik-ignore`, shared by recording and the blur picker (purple). Recording reads the user's `targetColor` so the live ring matches the dashed target baked into screenshots. Never drawn on `iframe`/`embed`/`object` β€” a capture inside a subframe can't hide the top frame's ring
+- **Content script injection** pings first, falls back to `chrome.scripting.executeScript()` for tabs without the script
+- **xstate snapshot** persisted to sessionStorage so the state machine survives service worker restarts
+- **Recording notification** uses `animationend` event (not hardcoded delays) for timing
+- **Font loading** uses `@fontsource/poppins` (CSP-safe, no CDN dependency)
+- **Cross-context sync** via BroadcastChannel β€” star/delete events update other views without full reload
\ No newline at end of file