🎬 A browser extension and desktop app that turns your clicks into step-by-step guides with annotated screenshots 📖
TypeScript browser-extensionchrome-extensiondocumentationfirefox-extensionguide-generatorlocal-firstmanifest-v3privacy-firstproductivityreactscreenshotstep-by-step-guidetutorialtypescriptworkflowwxt
docs: record how the narration transcript works
davbio401cff11ffb8
diff --git a/AGENTS.md b/AGENTS.mdindex f88d93a..9ea8bdc 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -36,6 +36,7 @@ src/│ │ ├── events/ # Event capture system│ │ │ ├── handlers.ts # CaptureController class + startCapture│ │ │ └── input-session.ts # InputSession (typing lifecycle)+│ │ ├── voice/ # Narration: energy gate, batching, transcribe, attribute, transcript│ │ ├── machine.ts # xstate capture state machine│ │ ├── session.ts # CaptureSession (lifecycle manager)│ │ ├── spa-nav.ts # SPA navigation tracking@@ -50,7 +51,7 @@ src/│ │ ├── scrub.ts # scrubValues (removes typed values from prose)│ │ ├── bundle.ts # exportGuideAsBundle (zip via fflate)│ │ └── parse.ts # readBundle (unzip + validate)-│ └── guides/ # Data layer: types, Dexie DB, CRUD service+│ └── guides/ # Data layer: types, Dexie DB, CRUD service, transcript timeline├── entrypoints/ # Chrome extension entry points (WXT)│ ├── background/ # Service worker: state machine, message handlers, tab management│ ├── content.ts # Content script: CaptureSession, event listeners@@ -100,7 +101,7 @@ src/|-------|------|---------|| Capture lifecycle | xstate | State machine (IDLE ↔ RECORDING ↔ PAUSED) in background service worker || Fullview UI | Zustand | Search modal, guide counts, active guide data |-| Persistence | Dexie (IndexedDB) | Guides, steps, screenshots, snapshots, cached voice clips |+| Persistence | Dexie (IndexedDB) | Guides, steps, screenshots, snapshots, cached voice clips, narration transcripts || Service worker recovery | sessionStorage | xstate machine snapshot persistence || Background → Sidepanel | Port messaging | Real-time state broadcast || Cross-context sync | BroadcastChannel | Guide mutations (star, delete) across sidepanel/fullview |@@ -186,6 +187,59 @@ Extraction walks up from the target element to find:- Nearest heading- Sibling interactive elements in the same container (max 10)+## Narration transcript (`core/capture/voice/`)++Attribution is a guess, so the words are kept independently of it. `assignSegments` returns a+`TranscriptLine` for **every** segment the transcriber returned — the ones a step took, the ones+nothing claimed (`stepId: null`), and the ones `rejectReason` threw away, with the reason on the+line. `runNarrationPipeline` collects them into `NarrationResult.transcript`, stamped with the audio+slice's `audioEpochMs`, and `applyNarration` in the background writes that to the `transcripts`+table (Dexie `version(4)`) *before* it applies anything to steps — a recording where nothing was+attributed is exactly the case this exists for, and it writes no step at all, so `saveTranscript`+announces itself on the guides channel rather than relying on `applyNarrationToSteps` to do it.++One row per transcribed slice, because mid-recording flushes land separately from the tail.+`flushedUpToSeconds` in the voice host guarantees the slices never overlap, so `mergeTranscripts`+(`core/guides/transcript.ts`) can order them by wall clock with nothing to de-duplicate. The+`epochMs` on each row is what makes that possible: a flush slice's seconds are relative to its own+start, not the recording's.++`Step.narratedDescription` holds what narration heard, verbatim. `applyNarrationToSteps` writes it,+and only `deleteTranscripts` clears it, so an edit replaces `description` and leaves it, and `restoreNarratedDescription`+can put the spoken text back with its `narration` source. That is the whole answer to "an edit+overwrote what I said": the step editor offers the restore only while the two actually differ.++The dashboard's transcript panel is the answer to the other half. It shares the side-panel column+with version history, so the store keeps them mutually exclusive, and the TopNav button only appears+when `hasTranscript` says there is one — which is re-read on every guide reload, because transcription+is still running when the dashboard opens after a recording. An unattributed line offers to append+itself to the nearest step: `nearestStepId` looks **forward** first, since onboarding asks users to+say what they are about to do and then do it. Accepting that writes the step onto the line+(`attachTranscriptLine`, flagged `addedByHand`) as well as into the description, so the line reads as+used from then on and a second visit to the panel cannot append the same words twice. Restoring that+step's spoken original deletes the appended sentence, so `restoreNarratedDescription` releases the+lines it just wiped out and they return to unused — otherwise the panel would claim a step still+carries words it no longer has, with no way left to re-add them. Only hand-added lines are released;+an attribution narration made itself is a record of what the pipeline decided and stays put.++Privacy is the cost of keeping this. A verbatim transcript is more sensitive than per-step text —+smart blur cannot reach speech, so a password read aloud lands in it as plain text. It is therefore+local-only: never in an exported guide, deletable on its own from the panel — which also clears+`narratedDescription` from the steps and from the snapshots that copied them, so “gone for good” is+true — and swept by `permanentlyDeleteGuide` alongside snapshots. `mergeGuideInto` re-keys+transcripts onto the guide it merges into and remembers the redirect, and `saveTranscript` resolves+the owning guide before it writes — by that redirect, or by the guide a line's step now sits on —+because an insert-recording deletes its staging guide before narration lands, and a slice where+nothing was attributed has no step to resolve through. Stripping the spoken text rewrites the stored+snapshot steps, so `deleteTranscripts` recomputes each `contentHash` too, or version history would+stop collapsing two identical saves into one row. Nothing was added to any export path, and nothing+should be.++What is *not* recoverable: speech the energy gate never detected, and a mis-transcription. Both need+the audio, and the audio is released at `recorder.stop()`. Keeping it is still a separate bet (see+the voice-over section) — what the transcript buys is the paid-for text that used to be thrown away+between the API response and the step write.+## Export Formats| Format | Generator | Details |diff --git a/README.es.md b/README.es.mdindex 1b64d04..8c5031d 100644--- a/README.es.md+++ b/README.es.md@@ -160,6 +160,12 @@ Habla en voz alta mientras grabas y Mimik convierte lo que dijiste en las descripasos. El audio se transcribe con tu propia key (OpenAI o Groq) y se empareja con el paso al quecorresponde, así narras una vez en lugar de escribir cada paso a mano.+Se guarda la transcripción completa, no solo lo que acabó en un paso. Abre **Transcripción** en una+guía para ver todo lo que dijiste, incluidas las frases que no se emparejaron con nada, y añade+cualquiera de ellas a un paso. Editar un paso tampoco destruye el original hablado: el editor+siempre puede restaurar lo que dijiste. La transcripción se queda en tu dispositivo, nunca forma+parte de una guía exportada y se elimina junto con la guía.+<img src="https://github.com/user-attachments/assets/061fddc7-da65-4641-8b39-d30b80c36531" alt="Narración por voz" width="800" /><div align="right">diff --git a/README.fr.md b/README.fr.mdindex b951418..f309b03 100644--- a/README.fr.md+++ b/README.fr.md@@ -160,6 +160,12 @@ Parle à voix haute pendant que tu enregistres et Mimik transforme ce que tu asd'étapes. L'audio est transcrit avec ta propre clé (OpenAI ou Groq) puis rattaché à l'étapecorrespondante, donc tu narres une fois au lieu d'écrire chaque étape à la main.+La transcription complète est conservée, pas seulement ce qui a atterri dans une étape. Ouvre+**Transcription** sur un guide pour voir tout ce que tu as dit, y compris les phrases rattachées à+rien, et ajoute celles que tu veux à une étape. Modifier une étape ne détruit pas non plus+l'original parlé : l'éditeur peut toujours remettre ce que tu as réellement dit. La transcription+reste sur ton appareil, ne fait jamais partie d'un guide exporté et est supprimée avec le guide.+<img src="https://github.com/user-attachments/assets/061fddc7-da65-4641-8b39-d30b80c36531" alt="Narration vocale" width="800" /><div align="right">diff --git a/README.md b/README.mdindex 99bcfd8..e33a04b 100644--- a/README.md+++ b/README.md@@ -162,6 +162,12 @@ Talk through the workflow out loud while you record and Mimik turns what you saidescriptions. Audio is transcribed with your own key (OpenAI or Groq) and matched to the steps itbelongs to, so you narrate once instead of writing every step by hand.+The full transcript is kept, not just the parts that made it into a step. Open **Transcript** on a+guide to see everything you said, including the lines nothing was matched to, and add any of them to+a step. Editing a step never destroys the spoken original either — the step editor can always put+back what you actually said. The transcript stays on your device, is never part of an exported guide, and is+deleted with the guide.+<img src="https://github.com/user-attachments/assets/061fddc7-da65-4641-8b39-d30b80c36531" alt="Voice narration" width="800" /><div align="right">diff --git a/README.pt-BR.md b/README.pt-BR.mdindex 8871843..c8bfd34 100644--- a/README.pt-BR.md+++ b/README.pt-BR.md@@ -160,6 +160,12 @@ Fala em voz alta enquanto grava e o Mimik transforma o que tu disse nas descriçáudio é transcrito com a tua própria key (OpenAI ou Groq) e casado com o passo a que pertence,então tu narra uma vez em vez de escrever cada passo na mão.+A transcrição inteira fica guardada, não só o que virou passo. Abre **Transcrição** num guia pra ver+tudo o que tu falou, inclusive as falas que não casaram com nada, e adiciona qualquer uma delas a um+passo. Editar um passo também não destrói o original falado: o editor sempre consegue devolver o que+tu realmente disse. A transcrição fica no teu dispositivo, nunca faz parte de um guia exportado e é apagada+junto com o guia.+<img src="https://github.com/user-attachments/assets/061fddc7-da65-4641-8b39-d30b80c36531" alt="Narração por voz" width="800" /><div align="right">diff --git a/README.zh-CN.md b/README.zh-CN.mdindex e83116e..ee288ec 100644--- a/README.zh-CN.md+++ b/README.zh-CN.md@@ -158,6 +158,8 @@ Mimik 可以在几秒内把任何重复性的浏览器任务变成一份可分录制时直接把流程讲出来,Mimik 会把你的语音转换成步骤描述。音频会用你自己的 key(OpenAI 或 Groq)转写,并匹配到对应步骤上,所以你只需要讲一遍,不用手动给每一步写说明。+完整的语音记录都会保留下来,而不只是进入步骤的那部分。在指南里打开 **语音记录**,就能看到你说过的全部内容,包括没有匹配到任何步骤的那些,并把其中任意一句加到某个步骤上。编辑步骤也不会毁掉原始口述内容 —— 步骤编辑器始终可以把你真正说过的话放回去。语音记录只留在你的设备上,不会出现在导出的指南里,并会随指南一起删除。+<img src="https://github.com/user-attachments/assets/061fddc7-da65-4641-8b39-d30b80c36531" alt="语音讲解" width="800" /><div align="right">