Skip to content

armgit / mimik

Public Imported from GitHub
0

🎬 A browser extension and desktop app that turns your clicks into step-by-step guides with annotated screenshots πŸ“–

docs: describe the API keys section, the abortable main-process fetch, the player, the loader and the desktop CI

Luis Ortizeb22c62af421
diff --git a/AGENTS.md b/AGENTS.md
index 3a177cc..f53c93e 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -1042,7 +1042,10 @@ renderer names the url and main is the one holding the network. A text, JSON or
response comes back as text and anything else as base64, which the renderer decodes into bytes:
voice-over audio went through the same path as JSON and arrived as mangled text. The voice-over
client calls `coreFetch` for that reason rather than the global `fetch`, which a renderer origin
-cannot use against a provider.
+cannot use against a provider. The request's abort signal crosses too: `mainFetch` stops waiting the moment the signal fires and
+sends `mimik:ai:abort` with the request's id, and main aborts its `net.fetch`. It used to drop the
+signal, so neither the 60-second limit on a voice-over clip nor cancelling the export ever reached the
+request, and one stalled clip held the export on "Narrating step 3 of 6" forever.
Ask AI and Generate description reach the model through `messages.send`, which the extension answers
from its background worker. The desktop has no worker, and its adapter's `send` used to throw for
@@ -1346,7 +1349,7 @@ settings were on disk when it finishes.
## Desktop Settings
-Five sections behind one left nav: General, Capturing, AI descriptions, Export Branding, Shortcuts, and
+Six sections behind one left nav: General, Capturing, AI, Export Branding, Shortcuts and API keys, and
the dialog opens on General, the first, as desktop settings conventionally do. General holds three cards named
for what they hold β€” Language, Startup and Updates β€” rather than one card named after the section,
which put "General" in the list and again as the pane's only heading. Startup and Updates are the
@@ -1354,11 +1357,40 @@ tray menu's own "Start at login" and "Check for updates", in the tray's words an
`setOpenAtLogin` and `checkForUpdates`, so the tray and the settings cannot disagree; the update check
does nothing in an unpackaged build, where the updater never runs. The split is the
same one that decides where a file lives. Capturing and Shortcuts describe things the extension has
-no concept of, so they are written in `apps/desktop` against `capture-settings.json`. AI descriptions
-and Branding are identical on both surfaces, so `AiSettings` and `BrandingSettings` live in
-`packages/ui` and both apps mount them. `AiSettings` takes a `validate` function β€” the desktop passes
-core's `validateApiKey` directly, and the extension goes through its background messaging, which is
-the only part that differs.
+no concept of, so they are written in `apps/desktop` against `capture-settings.json`. AI descriptions,
+video voice-over, the API keys and Branding are identical on both surfaces, so `AiSettings`,
+`VoiceoverSettings`, `ApiKeysSettings` and `BrandingSettings` live in `packages/ui` and both apps mount
+them; the desktop's AI section is the first two. `ApiKeysSettings` takes a `validate` function β€” the
+desktop passes core's `validateApiKey` directly, and the extension goes through its background
+messaging, which is the only part that differs.
+
+**One key per provider, entered once.** Descriptions, narration and voice-over each used to keep a
+key of their own β€” `aiApiKeys`, `voiceApiKey` and `voiceoverApiKeys` β€” and borrowed OpenAI's across
+features only when both sides happened to be OpenAI, so one person pasted the same key up to three
+times. The keys now live in one `apiKeys` map, one entry per provider in `KEY_PROVIDERS`, and every
+feature reads the key of the provider it uses through `readApiKeys`. Until that map exists,
+`readApiKeys` assembles it from the three old settings, the descriptions key winning where two
+features held different OpenAI keys; once it exists, even empty, the old settings are ignored, so a
+key cleared in the new section cannot come back from an old one. The API keys section is a field per
+provider with its logo, checked when the field loses focus and on opening, with a tick, a spinner or
+the reason it failed beside it. The feature cards hold no key field at all: each picks its provider
+through `ProviderSelect`, which lists every provider the feature supports with its logo and greys out
+the ones without a key β€” your own server without an address β€” each with an Add key link to the
+keys. The link is a button inside a disabled Radix item, so it takes `pointer-events-auto` back from
+the item and closes the select itself; on the desktop it opens the API keys section, in the
+extension it scrolls to the card at the top of the page. A provider saved before its key was cleared
+stays selected and says "Add one in API keys" under the select. The logos are the monochrome marks from the MIT-licensed lobehub
+icon set, copied into `ProviderLogo`, because the set would be a dependency of hundreds of icons for
+six paths.
+
+Your own server is not a key, it is an address: the second card of the section holds its base URL,
+an optional key and whether it speaks the OpenAI or the Anthropic API, stored as `aiServerUrl`,
+`apiKeys.server` and `aiServerProtocol`, and AI descriptions picks it like any provider. The check
+for it asks the server's `/models` without a model, so Ollama or LM Studio can be confirmed before
+anything is chosen; it used to need a model and a key first, which meant typing a key a local server
+never reads. It used to be a switch that replaced the selected provider's base URL, stored in
+`aiBaseUrl`. `resolveServer` still reads that, and `useApiKeys` moves it into the new settings the
+first time the settings open.
`BrandingSettings` is the extension's Brand colour card and Branding card β€” the colour of the click
highlight and export accent, the logo, the footer line and the attribution β€” lifted out of
@@ -1376,7 +1408,8 @@ leading slash on a `file://` page is `file:///mimik-mark.png`. Neither held befo
failing quietly returned no logo, so no desktop export ever had one. `check:pipeline` fetches it.
Every card is `SettingsCard` from `packages/ui`: the border, radius, padding, white face and 28px icon
-header, with an optional hint under the title and an optional control beside it. The extension's AI,
+header, with an optional hint under the title and an optional control beside it. The header is
+optional too, for the desktop's key list, whose section name already says what it holds. The extension's AI,
Brand colour, Export Branding, Voice narration and Smart blur cards and the desktop's `Card` all use
it; they had been six hand-copied frames, one of them the desktop's. The desktop `Card` only adds the
divided rows inside. Rows inside a
@@ -1387,10 +1420,10 @@ on screen at once, and matching them would mean branching that shared component
Capturing is two cards: Screenshots, and Typing and keys.
Each kind of value has one control, and it shows the value rather than hiding it behind a click. A
-choice among a few is `Segmented`, a row of buttons with the chosen one filled; that is the capture
-mode and the zoom, which offers Automatic, 1Γ—, 1.5Γ—, 2Γ—, 3Γ—, 4Γ— and 5Γ— and adds the stored level as one more
-button when an older build saved one in between. An on/off is `Switch`, the same switch the AI card's
-own-server toggle is, moved into `packages/ui` so the two cannot drift. A duration is `Slider`, a
+choice among a few is `Segmented`, a row of buttons with the chosen one filled, in `packages/ui` since
+the API keys section uses it for the server's API; that is the capture mode and the zoom, which offers Automatic, 1Γ—, 1.5Γ—, 2Γ—, 3Γ—, 4Γ— and 5Γ— and adds the stored level as one more
+button when an older build saved one in between. An on/off is `Switch`, shared with the extension's
+cards from `packages/ui` so the two cannot drift. A duration is `Slider`, a
native range input tinted with `accent-accent` beside a chip reading the value in ms or seconds,
because a number field hid what the default was and drew spinner arrows on Windows. A setting that cannot apply is disabled rather than hidden, so the rows
do not jump: outside clicks without Area mode and the typing pause without typing.
@@ -1420,13 +1453,13 @@ what has `host_permissions`, and the desktop calls `validateApiKey` directly on
process fetch.
The rest of `SettingsView` stayed put. Voice narration, smart blur and the microphone picker have no
-desktop meaning, and it reaches into `@/lib/browser-api/`. It mounts `BrandingSettings` where its own
-two cards were and passes the autosave's `queue` as `onChange`, which is only there to raise the
-Saved badge β€” the card has already written the value. Splitting it did mean the
-autosave had to change shape: the AI fields left the parent's snapshot, so `AiSettings` reports its
-own changes through `onChange` and both halves queue into the same debounced flush. `SettingsView`
-still keeps the provider and key in state for one reason β€” `resolveVoiceApiKey` falls back to the AI
-key when no voice key is set β€” and it updates them from the patches the card sends up.
+desktop meaning, and it reaches into `@/lib/browser-api/`. It mounts the API keys card at the top,
+then `AiSettings`, `BrandingSettings` and `VoiceoverSettings` where its own cards were, and passes the
+autosave's `queue` as `onChange`, which is only there to raise the Saved badge β€” the card has already
+written the value. It holds `useApiKeys` itself and hands the state to every card, because the
+extension shows them on one page and a key pasted at the top has to clear the note under the
+narration provider at once; the desktop's `SettingsPanel` holds it for the same reason across
+sections.
The shortcut recorder reads a keystroke and writes an Electron accelerator. It refuses a bare key,
because a global accelerator with no modifier takes that key from every application on the machine,
@@ -1485,6 +1518,17 @@ before `captureKey` writes anything, and it compares by parts rather than by str
`Shift+Alt+P` and `Alt+Shift+P` are the same shortcut. `CommandOrControl` resolves to Control
everywhere but macOS.
+## Desktop Input Hook and CI
+
+`uiohook-napi` needs no compiler on Windows: the package ships `prebuilds/win32-x64/uiohook-napi.node`,
+an N-API build that loads in any Electron, and pnpm never runs its `node-gyp-build` install script
+because the package is not in `onlyBuiltDependencies`.
+
+`.github/workflows/desktop.yml` runs on the `desktop` branch, which the extension's `pr-test.yml`
+does not cover: lint, typecheck, the tests, the extension build and the storage, pipeline and
+overlay checks under `xvfb-run` on Linux, then on Windows the addon, the storage and pipeline checks
+and an unsigned NSIS installer uploaded as the run's artifact.
+
## Export Formats
| Format | Generator | Details |
@@ -1529,10 +1573,10 @@ WebM).
`providers.ts` is the registry, shaped like `AI_PROVIDERS`: OpenAI (`/audio/speech`, fixed voice
list) and ElevenLabs (`/text-to-speech/{voice}`, per-account catalog via `listVoices`). Both return
mp3. OpenAI is the default because most users already hold that key, and `resolveVoiceoverConfig`
-borrows it from AI descriptions on the same terms `resolveVoiceApiKey` uses β€” both sides OpenAI, or
-nothing. **Holding a key never turns narration on**; `exportOptions.voiceover` defaults to false and
-only the user flips it. Keys are stored per provider (`voiceoverApiKeys`) so switching does not lose
-one, and a voice id is validated against the provider that issued it β€” an ElevenLabs id selected
+reads whichever key the API keys section holds for the chosen provider β€” the same one descriptions
+and narration use. **Holding a key never turns narration on**; `exportOptions.voiceover` defaults to
+false and only the user flips it. Switching provider loses nothing, since each provider keeps its own
+key, and a voice id is validated against the provider that issued it β€” an ElevenLabs id selected
under OpenAI falls back to OpenAI's default rather than being sent and rejected. ElevenLabs ids are
taken on trust, since the account catalog is larger than the shipped list.
@@ -1583,6 +1627,30 @@ bar with no volume affordance. Narrated previews flip both β€” a browser will no
so a narrated preview waits for the play button and that gesture buys it audio from the first frame,
while a silent one still autoplays muted as before. The mute toggle is there in both cases.
+The player sits on the app's light background rather than a dark one, in two white cards: the video
+with its controls under it, and the step list beside it. Only the frame itself stays dark
+(`FRAME_FILL`), sized by container query units to the largest 16:9 that fits the card. The timeline
+under the video is one segment per step, sized by the step's length and filled as it plays, and each
+segment jumps to its step. The list numbers the steps the way the guide does and marks each one with
+`StepKindBadge` β€” Click, Type, Key, Page or Note, in the style of the AI and Basic badges. It used to
+be a coloured dot per kind, and the click dot was the accent, which became the panel's own navy when
+the palette changed, so clicks showed no dot at all and read as a different kind from typing.
+
+While the video is made, `VideoLoader` shows the stage it is in. Narrating, the mascot talks, with
+its mouth and sound waves moving, over the line being read β€” `renderVoiceover` reports the next
+segment's text with its progress for this. Encoding, the camera mascot films over a running strip of
+frames, over the step count. Both stages share one bar with the percentage beside it, and it is the
+export's single overall figure, so it carries on from narrating into encoding rather than starting
+again: a segmented bar for narration and a percentage for encoding read as two different loaders.
+The stage is `useVideoPreview`'s `stage`, not inferred from narration progress: with voice-over on
+it starts on `voice` and turns to `video` only on the first encoded frame, because the containers
+are probed and the clip cache read before the first clip is asked for, and a loader that took "no
+narration yet" for encoding showed the camera first. Until that first clip it reads "Preparing the
+voice-over". `LoaderMascot` carries the change as one drawing rather than swapping two: the mouth
+stops, the waves fade, the camera drops onto the cap and the film strip slides into a space kept
+for it, so nothing below it moves. The bar is `role="progressbar"` with that percentage, which is
+what the modal's tests read.
+
The export panel shows the voice-over control under Video quality wherever video export is possible,
not only on the video preview tab β€” a guide can be exported to video from the format list without
ever opening that tab. The length estimate appears only once voice-over is on; the estimate's second figure comes from the rendered chapters, so it is the