Claude's answers, rendered: Markdown, math, charts and diagrams drawn in the Claude Code terminal. Independent project, not affiliated with Anthropic.
TypeScript claude-code-mod
Compile: a lean font set for plain text (--ignore-system-fonts with a folder of the needed fonts), about 3x faster per block
razorback16ee5d9bb88bb7
diff --git a/docs/development.md b/docs/development.mdindex 11906eb..1d1ebe5 100644--- a/docs/development.md+++ b/docs/development.md@@ -21,7 +21,7 @@ Set `IUI_DEBUG=1` before you start Claude Code. The mod then writes trace lines## Module layout-The hooks module is `hooks/register.tsx`. It connects the hooks to the modules in `hooks/lib/`. Each module has one topic, for example `markdown.ts` (blocks and segments), `mode.ts` (light or dark), `typst.ts` (compile), `draw.tsx` (the pages of an answer and the link rows under them), `links.ts` (the links, file paths and commands of a text), `answer.ts` (one text through the compile and cache steps), `plan.tsx` (the plan approval question and the plan rows), `frame.ts` (the terminal size and the watcher process, which also reports the clipboard changes), `copy.ts` (the text under a mouse selection of pictures), `probe.ts` (the drawn pages, and the words of a page that a copy reads, kept in memory) and `updates.ts` (the daily check for a new version). The values that change are in one object, `S`, in `state.ts`. Each `$` method has one call site, a wrapper in `api.ts`; the other modules call the wrappers. This keeps the `calls:` note of `claude plugin validate` short (the validator cuts a note at about 1,000 characters, and the catalogue scan then cannot read the footprint). The tests import from `hooks/register.tsx`, which exports the names that they use.+The hooks module is `hooks/register.tsx`. It connects the hooks to the modules in `hooks/lib/`. Each module has one topic, for example `markdown.ts` (blocks and segments), `mode.ts` (light or dark), `typst.ts` (compile), `fonts.ts` (the lean font set: the fonts that a compile of plain text loads), `draw.tsx` (the pages of an answer and the link rows under them), `links.ts` (the links, file paths and commands of a text), `answer.ts` (one text through the compile and cache steps), `plan.tsx` (the plan approval question and the plan rows), `frame.ts` (the terminal size and the watcher process, which also reports the clipboard changes), `copy.ts` (the text under a mouse selection of pictures), `probe.ts` (the drawn pages, and the words of a page that a copy reads, kept in memory) and `updates.ts` (the daily check for a new version). The values that change are in one object, `S`, in `state.ts`. Each `$` method has one call site, a wrapper in `api.ts`; the other modules call the wrappers. This keeps the `calls:` note of `claude plugin validate` short (the validator cuts a note at about 1,000 characters, and the catalogue scan then cannot read the footprint). The tests import from `hooks/register.tsx`, which exports the names that they use.The Typst template is `render/md.typ`. The themes are in `render/themes/` (refer to [`render/themes/README.md`](../render/themes/README.md)). The vendored Typst packages are in `render/packages/preview/`.diff --git a/docs/how-it-works.md b/docs/how-it-works.mdindex df324a6..cfa4485 100644--- a/docs/how-it-works.md+++ b/docs/how-it-works.md@@ -5,7 +5,7 @@## The steps1. A `ui.render` hook on `AssistantMessage` gets each block of the answer as Markdown.-2. While Claude writes a block, the block is hidden. The block is final when the model request starts its next block (a tool call, more text) or ends. If the mod did not see the request, the block is final after its text stops changing for 400 ms. Then the mod writes the block to a JSON file and runs `typst compile` with `render/md.typ`. Long answers are cut at block boundaries into pages. One page is one PNG.+2. While Claude writes a block, the block is hidden. The block is final when the model request starts its next block (a tool call, more text) or ends. If the mod did not see the request, the block is final after its text stops changing for 400 ms. Then the mod writes the block to a JSON file and runs `typst compile` with `render/md.typ`. For plain text, Typst loads only some fonts (refer to [The lean font set](#the-lean-font-set)). Long answers are cut at block boundaries into pages. One page is one PNG.3. The hook draws one `Image` per PNG, in place of the text.4. Text size: the mod finds the terminal font (iTerm2 profile, Ghostty, kitty or Rio config, else Menlo metrics) and the cell size in pixels. Typst then measures both fonts so that 100% looks as large as the terminal text.5. If a compile fails (a bad Typst block or an unknown LaTeX command), the mod compiles once more in safe mode: Typst blocks and math show as code. A dim line under the picture names the error. If that also fails, the normal text shows.@@ -14,6 +14,27 @@8. The plan: a `ui.render` hook on the ExitPlanMode `ToolUse` row draws the plan through the same steps. Claude Code shows that row above the approval question. The approved result row then shows only its header, so the plan does not show two times. A `tool.call` hook on ExitPlanMode asks the mod's question (`$.ui.ask`) before Claude Code asks for permission. Then a `classic.PermissionRequest` hook gives the answer to Claude Code at once, so the Claude Code dialog does not show. For an approval, the hook sends the plan (`updatedInput`) and the new permission mode (`setMode`). For the person's text, the `tool.call` hook refuses the call with that text.9. Copy: one long-lived `perl` process watches the terminal size (at each SIGWINCH) and, on macOS, the clipboard. For the clipboard, it uses `/usr/bin/perl` with Apple's PerlObjCBridge and reads the change counter of the clipboard (`NSPasteboard changeCount`) every 0.3 s. This read is in the process and starts no other process (approximately 0.05% of one CPU core). The clipboard part starts only while the mod is on and after the mod drew its first picture in this session; then the mod starts the process again with the clipboard part (and again without it after `/iui off`). When the counter changes, the mod reads the clipboard one time (`pbpaste`). The first counter after a start is not a change, so an old clipboard is not copied. A picture cell holds its row and its column (kitty placeholder U+10EEEE plus two combining marks). The mod finds the drawn page that has these rows, at the size that it is drawn. Then it finds the position of each word on that page: it runs the probe (`typst query` on the same template, with the same inputs as the PNG compile, plus `probe=1` and `page=<n>`) for the selected pages only. The mod keeps the positions of the last 64 pages in memory. It writes nothing to disk for this, and the PNGs do not change. The mod takes the words in the selected cells. Immediately before it writes the text with `$.ui.copy`, it reads the clipboard again. If the clipboard has a newer copy, the mod does not write over it, and it reads that copy next. If the process stops, the mod starts it again after 1 s (3 times at most). At `/clear` or a resume, the process continues without the clipboard part. At other session ends (for example `/exit`), the mod stops the process and does not start a new one. A `ui.copy` hook also changes a copy with picture cells to the text.+## The lean font set++At each start, Typst reads all the fonts of the computer. On macOS, this is approximately 70 ms of each compile. Plain text does not need most of these fonts. Plain text has only ASCII characters, no math outside code, no live Typst block, no image and no HTML entity. In the answers that we measured, approximately 3 of 4 blocks are plain text.++For plain text, Typst loads only these fonts:++- The terminal font.+- The body fonts and the code fonts of the theme, but not the CJK fonts and the emoji fonts.+- The fonts in Typst (for example, the math font).++The mod puts links to these font files in one folder in the cache (`fonts-<hash>`). It gives Typst `--ignore-system-fonts --font-path <folder>`. Then a block of plain text compiles in approximately 0.03–0.07 s, not 0.1–0.15 s. The PNG does not change. For ASCII text, Typst takes each character from the first font of the list that has it, and that font is in the folder. Other text can need a character that no font of the list has. Then Typst looks in all the fonts of the computer, so other text compiles with all the fonts.++The mod makes the folder one time in each session, after the first compile of plain text. `typst fonts --variants` gives the files. The mod does not use the folder in these conditions:++- The first installed font of a list is a CJK font or an emoji font.+- A list has no installed font.+- One style of a font is in two files (then the order in which Typst reads the files can change the font that it uses).+- The list markers or the numbering of the theme are not ASCII.++If a compile with the folder fails, the mod compiles again with all the fonts. The probe of a copy (step 9) uses the same fonts as the PNG compile. If the folder is not there (the cleanup at start deletes folders that are older than 2 days), the mod uses all the fonts and makes the folder again.+## How `auto` is resolvedClaude Code finds the light or dark mode for `auto` by asking the terminal for its background colour (OSC 11). A mod cannot see that answer, so Intelligent UI uses these sources, in this order. The first source that gives an answer is used:diff --git a/docs/limitations.md b/docs/limitations.mdindex acd02bf..2294633 100644--- a/docs/limitations.md+++ b/docs/limitations.md@@ -4,7 +4,7 @@- **Copying text from a picture** (refer to [Copy text from a picture](usage.md#copy-text-from-a-picture)) has these limits:- The automatic copy works only on macOS, and not over ssh (`pbpaste` reads the clipboard of the computer that runs Claude Code). It also needs `/usr/bin/perl` with Apple's PerlObjCBridge (macOS has it), which reads the change counter of the clipboard. If the bridge does not load, the automatic copy is off: the mod does not read the clipboard at intervals in its place. `/iui details` shows the state on the line `Auto copy:`. When the automatic copy is off, the clipboard keeps the picture cells (U+10EEEE plus combining marks), and a paste gives strange characters. Use `/iui grab`, Claude Code's `/copy` (the last answer as Markdown, `/copy 2` for the one before it), or turn the mod off (`/iui off`) to select text.- - The first copy from a page takes 0.15–0.5 s (2–3 s for a page with a large Typst drawing), and no toast shows during this time. If you paste before the toast "copied N chars of text" shows, the paste gives the picture cells.+ - The first copy from a page takes approximately 0.05 s for plain text (refer to [The lean font set](how-it-works.md#the-lean-font-set)), 0.15–0.5 s for other text, and 2–3 s for a page with a large Typst drawing. No toast shows during this time. If you paste before the toast "copied N chars of text" shows, the paste gives the picture cells.- The mod copies whole words. It copies a word when the selection covers the middle of the word. In a code block, in a long word (30 characters or more, for example a path or a URL) and in CJK text, it copies each character whose middle is in the selection.- The mod does not copy math, the text in an image, or the text in a Typst drawing.- The cells do not identify the answer. The mod uses the answer of the selected row if it knows it, else the newest answer whose pages fit the selection. A selection in one page of a long answer fits each page that is tall enough, and the mod uses the first one. Thus the text can come from a different page or an older answer of the same width. `/iui dump` shows the page that the mod used.@@ -20,7 +20,7 @@- A level-4 to level-6 heading at 100% (and a level-3 or level-4 heading at approximately 117%) has the same gap above it and below it (1 row), because the two gaps are nearest to 1 row.- If inline math makes the first line of an answer taller, the reply bullet can be one row above the text.- **Search does not find text in a picture.**-- **While Claude writes**, the block is hidden, and Claude Code's spinner shows the progress. When the block is final, its picture shows after the compile (0.15–0.6 s). A long answer shows only when it is done. If the picture is not ready 2 s after the block is final, the normal text shows until the picture is ready. If the compile fails, the normal text stays. An earlier answer that is not in the cache (for example, after a resume) is blank for up to 1 s, then shows as text until its picture is ready.+- **While Claude writes**, the block is hidden, and Claude Code's spinner shows the progress. When the block is final, its picture shows after the compile: approximately 0.03–0.07 s for plain text (refer to [The lean font set](how-it-works.md#the-lean-font-set)), 0.15–0.6 s for other text, and 2–3 s for a large Typst drawing. A long answer shows only when it is done. If the picture is not ready 2 s after the block is final, the normal text shows until the picture is ready. If the compile fails, the normal text stays. An earlier answer that is not in the cache (for example, after a resume) is blank for up to 1 s, then shows as text until its picture is ready.- **Short summaries** between tool calls stay as text.- **Plan approval:** a mod cannot draw the Claude Code approval dialog, so the mod shows its own question. That question can have 4 options at most. Thus it has no option to keep planning (use Type something or Esc). The edit option cannot use a terminal editor (vim, nano), because a mod has no terminal. Where an organization manages Claude Code, `classic.PermissionRequest` hooks of a plugin that you install do not run. Then the Claude Code dialog shows after the mod's question.- **Typst blocks:** `#page`, `#pagebreak` and `#set page` are errors. A package that is not in the mod cannot be downloaded (the network is blocked for Typst), so its block shows as code.diff --git a/docs/usage.md b/docs/usage.mdindex 1ab2394..8e6ab69 100644--- a/docs/usage.md+++ b/docs/usage.md@@ -78,7 +78,7 @@ The mod does not take web links from code blocks, inline code or math. It takesSelect the text in a picture (an answer, a plan or a Markdown file that Claude sent) with the mouse, as you do with normal text. The selection puts picture cells on the clipboard. On macOS, while the mod is on and has drawn a picture in this session, the mod sees each change of the clipboard (in 0.3 s or less). When the clipboard has picture cells, the mod puts the text under your selection on the clipboard, and a toast shows the number of characters. Wait for this toast, then paste. In fullscreen mode, if this session has a selection, the mod changes the clipboard only when the clipboard holds that selection. Thus a copy from a different Claude Code session usually stays as it is. The mod does not change a clipboard that has no picture cells.-The mod finds the position of each word on a page when you copy from that page for the first time. This takes 0.15–0.5 s, and more (2–3 s) for a page with a large Typst drawing. No toast shows while it does this. Only the toast of the copy shows. The mod keeps the positions in memory, so the next copy from the same page is fast.+The mod finds the position of each word on a page when you copy from that page for the first time. This takes approximately 0.05 s for plain text (only ASCII characters, no math, no Typst block and no image), 0.15–0.5 s for other text, and more (2–3 s) for a page with a large Typst drawing. No toast shows while it does this. Only the toast of the copy shows. The mod keeps the positions in memory, so the next copy from the same page is fast.If the clipboard did not change, use `/iui grab`. It reads the last mouse selection of Claude Code (fullscreen mode only) and copies its text. If the copied text is not correct, `/iui dump` shows the cells that the mod found and the page that it used. It also writes all of this to `copy-debug.json` in the cache folder.diff --git a/hooks/lib/fonts.ts b/hooks/lib/fonts.tsnew file mode 100644index 0000000..f7aaf9c--- /dev/null+++ b/hooks/lib/fonts.ts@@ -0,0 +1,162 @@+// The lean font set: a compile of plain text loads only the theme's Latin+// fonts and the terminal font, from a folder of links, and not every system+// font. Typst reads all system fonts at each start (about 70 ms on macOS);+// the lean set takes about 10 ms. The PNG does not change: for ASCII text,+// Typst takes each glyph from the first font of the list that has it, and+// that font is in the lean set. A glyph that no listed font has (Typst then+// looks in all fonts) can come only from other text: emoji, CJK, symbols,+// math. Such a text compiles with all fonts.+import type { EngineInterface } from 'claude-code'++import { leanDirs, type Env, type LeanFonts } from './state'+import { FENCE, isTypstInfo } from './markdown'+import { hashOf } from './draw'+import { logDebug } from './log'+import { fsExists, processRun } from './api'++// The CJK and emoji fonts: large to read, and plain text never needs them.+export const SKIPPED_FONTS = /emoji|cjk|pingfang|hiragino|gothic|han sans|han serif|yahei|jhenghei|meiryo|malgun|simsun|simhei|wenquanyi|droid sans fallback/i++// One font face of `typst fonts --variants`: its file (null: in Typst+// itself) and its variant (`Normal, Weight: 400, Stretch: 100%`).+export type Face = { file: string | null; variant: string }++// `typst fonts --variants` as faces by family (lower case, as Typst+// matches a family name).+export function fontListOf(stdout: string): Map<string, Face[]> {+ const families = new Map<string, Face[]>()+ let faces: Face[] | null = null+ let file: string | null = null+ for (const line of stdout.split('\n')) {+ if (line && !/^\s/.test(line)) {+ const family = line.trim().toLowerCase()+ faces = families.get(family) ?? []+ families.set(family, faces)+ continue+ }+ const at = /^\s+[├└]\S*\s+(.*)$/.exec(line)+ if (at) {+ file = at[1] === '(Embedded)' ? null : at[1]!.trim()+ continue+ }+ const style = /^\s+\S*\s+Style:\s*(.*)$/.exec(line)+ if (style && faces) faces.push({ file, variant: style[1]!.trim() })+ }+ return families+}++// The font files of the lean set, in the order of the list: the faces of+// the terminal font and of each body and code font that is not skipped.+// Null when the lean set can draw differently: the first font of a list+// that is installed is a skipped one, a list has no installed font, or a+// family has the same variant in two files (Typst takes the first one it+// read, and the two reads go in a different order).+export function leanFiles(list: ReadonlyMap<string, readonly Face[]>, termFont: string, fonts: LeanFonts): string[] | null {+ const has = (family: string) => (list.get(family.toLowerCase())?.length ?? 0) > 0+ // The code font list with and without the terminal font (a theme can+ // turn it off).+ for (const each of [fonts.body, fonts.code, [termFont, ...fonts.code].filter(Boolean)]) {+ const first = each.find(has)+ if (!first || (first !== termFont && SKIPPED_FONTS.test(first))) return null+ }+ const files: string[] = []+ const seen = new Set<string>()+ for (const family of [termFont, ...fonts.body, ...fonts.code]) {+ const key = family.toLowerCase()+ if (!family || seen.has(key) || (family !== termFont && SKIPPED_FONTS.test(family))) continue+ seen.add(key)+ const where = new Map<string, string>()+ for (const face of list.get(key) ?? []) {+ if (!face.file) continue+ const other = where.get(face.variant)+ if (other !== undefined && other !== face.file) return null+ where.set(face.variant, face.file)+ if (!files.includes(face.file)) files.push(face.file)+ }+ }+ return files.length ? files : null+}++// The lean folder of a set of files: in the cache, by the files.+export const leanDirOf = (cache: string, files: readonly string[]) => `${cache}/fonts-${hashOf(files)}`++// Makes the lean folder: one link for each file (named by its place, so two+// files of one name do not clash). A folder that is there is kept (its name+// says its files). Made in a temporary folder first, so a compile never+// sees a half folder; when another session made it first, ours goes.+export const LEAN_SH = [+ 'd=$1; shift; [ -d "$d" ] && exit 0',+ 't="$d.$$"; rm -rf "$t"; mkdir -p "$t" || exit 1',+ 'i=0; for f in "$@"; do i=$((i+1)); ln -s "$f" "$t/$i-${f##*/}" || { rm -rf "$t"; exit 1; }; done',+ 'mv "$t" "$d" 2>/dev/null; rm -rf "$t" "$d/${t##*/}"; [ -d "$d" ]',+].join('\n')++// Plain text for the lean set: ASCII only, no live Typst block (its code+// can name any font or symbol), no image (an image alt has an emoji), no+// HTML entity or <img>, and no math outside code (a math symbol that the+// math font does not have comes from any font). Math is `$`; the `$` of+// code (fenced blocks, code spans) does not count.+export function isPlain(md: string): boolean {+ if (!/^[\x20-\x7e\t\n\r]*$/.test(md)) return false+ if (/!\[|<img|&(#\d+|#x[0-9a-f]+|[a-z][a-z0-9]*);/i.test(md)) return false+ const prose: string[] = []+ let open: { char: string; len: number } | null = null+ for (const line of md.split(/\r\n?|\n/)) {+ const m = FENCE.exec(line)+ if (open) {+ if (m && m[2]![0] === open.char && m[2]!.length >= open.len && line.trim() === m[2]) open = null+ } else if (m) {+ if (isTypstInfo(m[3] ?? '')) return false+ open = { char: m[2]![0]!, len: m[2]!.length }+ } else prose.push(line)+ }+ const text = prose.join('\n')+ // A code span: a run of backticks to the same run, in one line (a span+ // over lines can be two paragraphs). With an escaped backtick, no span+ // is cut out (each `$` counts).+ const outside = text.includes('\\`') ? text : text.replace(/(?<!`)(`+)(?!`)[^\n]*?(?<!`)\1(?!`)/g, '')+ return !outside.includes('$')+}++// The lean folder for a compile of `segments`: '' when the text is not+// plain, the theme's lists are not, or the folder is not made yet. The+// first call for a set of fonts starts the make (in the background), so+// that compile uses all fonts. `vars`: the environment of a compile.+export async function leanFontsFor($: EngineInterface, env: Env, segments: readonly string[], vars: Record<string, string>): Promise<string> {+ const fonts = env.metrics.fonts+ if (!env.typst || !fonts?.plain || !segments.every(isPlain)) return ''+ const want = JSON.stringify([env.typst, env.termFont, fonts.body, fonts.code])+ const dir = leanDirs.get(want)+ if (dir === undefined) {+ leanDirs.set(want, '')+ void makeLean($, env, env.typst, fonts, want, vars)+ return ''+ }+ if (!dir) return ''+ // The cleanup at start removes the folders not used for 2 days: one gone+ // is made again.+ if (!(await fsExists($, dir).catch(() => false))) {+ leanDirs.delete(want)+ return ''+ }+ return dir+}++// Lists the fonts as a compile sees them (its environment), then links the+// files of the lean set. A failure is kept: no lean set for these fonts in+// this session.+async function makeLean($: EngineInterface, env: Env, typst: string, fonts: LeanFonts, want: string, vars: Record<string, string>) {+ const listed = await processRun($, [typst, 'fonts', '--variants'], { timeoutMs: 10000, env: vars }).catch(() => null)+ const files = listed?.exitCode === 0 ? leanFiles(fontListOf(listed.stdout), env.termFont, fonts) : null+ if (!files) {+ await logDebug($, env.cache, 'lean fonts: not used (the font list does not allow it)')+ return+ }+ const dir = leanDirOf(env.cache, files)+ const made = await processRun($, ['sh', '-c', LEAN_SH, 'sh', dir, ...files], { timeoutMs: 10000 }).catch(() => null)+ if (made?.exitCode !== 0) {+ await logDebug($, env.cache, `lean fonts: the folder ${dir} was not made:\n${made?.stderr ?? ''}`)+ return+ }+ if (leanDirs.get(want) === '') leanDirs.set(want, dir)+}diff --git a/hooks/lib/state.ts b/hooks/lib/state.tsindex 06c1903..7c3af1a 100644--- a/hooks/lib/state.ts+++ b/hooks/lib/state.ts@@ -33,10 +33,14 @@ export type Room = { heightPt: number; design: Design; designKey: string; isNew:// Per 1 em (render/measure.typ): `adv` the width of "0", `xh` the x-height,// `avg` the mean advance over a lowercase pangram.export type FontMetrics = { adv: number; xh: number; avg: number }+// The theme's font lists (merged over vscode), and `plain`: its list+// markers and numbering are ASCII (fonts.ts: the lean font set).+export type LeanFonts = { body: string[]; code: string[]; plain: boolean }// `term` the terminal's font (null when unknown or not installed), `menlo`// the template's code font, `body` its text font, `grid` the row grid's-// numbers (DEFAULT_GRID when not given).-export type Metrics = { term: FontMetrics | null; menlo: FontMetrics; body: FontMetrics; grid?: GridMetrics }+// numbers (DEFAULT_GRID when not given), `fonts` the theme's font lists+// (none: no lean font set).+export type Metrics = { term: FontMetrics | null; menlo: FontMetrics; body: FontMetrics; grid?: GridMetrics; fonts?: LeanFonts }// Measured with Typst on macOS: Menlo and SF Pro Text.export const DEFAULT_METRICS: Metrics = {term: null,@@ -267,6 +271,11 @@ export const clip = {last: null as unknown,}+// The lean font sets (fonts.ts) by the fonts they are for (Typst, the+// terminal font and the theme's lists): the folder once made ('' while it+// is made, or when the lean set is not used for these fonts).+export const leanDirs = new Map<string, string>()+// Sets a cache entry as the newest; the oldest go past `max`.export function keep<T>(cache: Map<string, T>, key: string, value: T, max: number) {cache.delete(key)diff --git a/hooks/lib/typst.ts b/hooks/lib/typst.tsindex 3c0b421..ee9749e 100644--- a/hooks/lib/typst.ts+++ b/hooks/lib/typst.ts@@ -3,7 +3,7 @@import type { EngineInterface } from 'claude-code'import { COMPILE_TIMEOUT_MS, MAX_IMAGE_ROWS, NO_NETWORK, SEGMENT_ROWS, TOUCH_FLUSH_MS, TYPST_FALLBACKS } from './settings'-import { DEFAULT_METRICS, type Cell, type FontMetrics, type Job, type Metrics, type Page, type Result } from './state'+import { DEFAULT_METRICS, type Cell, type FontMetrics, type Job, type LeanFonts, type Metrics, type Page, type Result } from './state'import { firstLine, imagePathOf, localImages, plainOf, splitInTwo, splitSegments } from './markdown'import { inlineBudget } from './terminal'import { fontFit, placeFirstLine, sizing } from './draw'@@ -11,10 +11,12 @@ import { DEFAULT_GRID, gridOf } from './grid'import { HEADERS_SH } from './host'import { logDebug } from './log'import { positionsOf, type Mark, type Positions } from './copy'+import { leanFontsFor } from './fonts'import { clockAfter, envGet, fsExists, fsList, fsRead, fsWrite, processRun, processSpawn } from './api'// The typst command and its environment. Network off (see NO_NETWORK) and a-// package cache of our own, so only the vendored packages load.+// package cache of our own, so only the vendored packages load. `fonts`:+// the lean font folder (fonts.ts) in place of all system fonts.export function typstCall(o: {typst: stringroot: string@@ -22,11 +24,13 @@ export function typstCall(o: {ppi: numberinputs: Readonly<Record<string, string | number>>out: string+ fonts?: string}): { argv: string[]; env: Record<string, string> } {const argv = [o.typst, 'compile','--root', '/','--package-path', `${o.root}/render/packages`,+ ...fontArgs(o.fonts),'--format', 'png','--diagnostic-format', 'short','--ppi', o.ppi.toFixed(4),@@ -37,6 +41,9 @@ export function typstCall(o: {return { argv, env: noNetworkEnv(o.cache) }}+// The arguments that load only the fonts of a lean folder (and Typst's own).+export const fontArgs = (fonts?: string) => (fonts ? ['--ignore-system-fonts', '--font-path', fonts] : [])+// The typst command that makes a PNG of a JPEG or WebP (render/convert.typ)// that fits `box` (in pixels), never larger than its own pixel size: at 72// ppi, 1 pt is 1 pixel.@@ -124,6 +131,13 @@ export const measureCall = (typst: string, root: string, family: string, themeDi'--input', `term-font=${family}`, '--input', `theme-dir=${themeDir}`,]+// The theme's font lists of a measure, when they are lists of names.+export function leanFontsOf(v: unknown): LeanFonts | undefined {+ const f = v as Partial<LeanFonts> | null | undefined+ const names = (x: unknown): x is string[] => Array.isArray(x) && x.every(n => typeof n === 'string')+ return f && names(f.body) && names(f.code) && typeof f.plain === 'boolean' ? { body: f.body, code: f.code, plain: f.plain } : undefined+}+// The fonts' metrics from render/measure.typ; the macOS defaults when that fails.export async function measureFonts($: EngineInterface, typst: string, root: string, cache: string, family: string, themeDir: string): Promise<Metrics> {const argv = measureCall(typst, root, family, themeDir)@@ -131,7 +145,8 @@ export async function measureFonts($: EngineInterface, typst: string, root: stritry {const m = JSON.parse(got?.stdout ?? '') as Metricsconst ok = (x: FontMetrics | null | undefined) => !!x && x.adv > 0 && x.xh > 0 && x.avg > 0- if (ok(m.menlo) && ok(m.body)) return { term: ok(m.term) ? m.term : null, menlo: m.menlo, body: m.body, grid: gridOf(m.grid) ?? DEFAULT_GRID }+ const fonts = leanFontsOf(m.fonts)+ if (ok(m.menlo) && ok(m.body)) return { term: ok(m.term) ? m.term : null, menlo: m.menlo, body: m.body, grid: gridOf(m.grid) ?? DEFAULT_GRID, ...(fonts ? { fonts } : {}) }} catch {}return DEFAULT_METRICS}@@ -308,22 +323,43 @@ export function typstInputs(job: Job, segments: readonly string[], dir: string,return { inputs, ppi }}+// The lean font folder for a job's segments ('' none: all fonts), noted as in use.+async function leanFor($: EngineInterface, job: Job, segments: readonly string[]): Promise<string> {+ const fonts = await leanFontsFor($, job.env, segments, noNetworkEnv(job.env.cache)).catch(() => '')+ if (fonts) touchLater($, fonts)+ return fonts+}++// Runs a typst command with the lean fonts, else with all fonts. A lean+// run that fails (not one that was cancelled or timed out) runs once more+// with all fonts, so the lean set never makes a failure of its own.+async function runFonts($: EngineInterface, job: Job, fonts: string, run: (fonts?: string) => Promise<Ran>): Promise<Ran> {+ if (!fonts) return run()+ const ran = await run(fonts)+ if (ran.exitCode === 0 || ran.isCancelled || ran.stderr === 'timed out') return ran+ await logDebug($, job.env.cache, `typst for ${job.key} with the lean fonts failed (now with all fonts):\n${ran.stderr}`)+ return run()+}+export async function runTypst($: EngineInterface, job: Job, segments: readonly string[], dir: string, tag: string, isSafe: boolean, signal?: AbortSignal): Promise<Ran> {const { inputs, ppi } = typstInputs(job, segments, dir, isSafe)- const { argv, env: vars } = typstCall({ typst: job.env.typst!, root: job.env.root, cache: job.env.cache, ppi, inputs, out: `${dir}/${tag}-{0p}.png` })- return runKillable($, argv, { env: vars, timeoutMs: COMPILE_TIMEOUT_MS, ...(signal ? { signal } : {}) })+ return runFonts($, job, await leanFor($, job, segments), fonts => {+ const { argv, env: vars } = typstCall({ typst: job.env.typst!, root: job.env.root, cache: job.env.cache, ppi, inputs, out: `${dir}/${tag}-{0p}.png`, ...(fonts ? { fonts } : {}) })+ return runKillable($, argv, { env: vars, timeoutMs: COMPILE_TIMEOUT_MS, ...(signal ? { signal } : {}) })+ })}// ---- The probe: where the words are ---------------------------------------// `typst query` of the template's <mdpos> marks with probe=1: the PNG-// compile's own inputs (and `page`), root, package path and env (no ppi,-// format or out).-export function probeCall(o: { typst: string; root: string; cache: string; inputs: Readonly<Record<string, string | number>> }): { argv: string[]; env: Record<string, string> } {+// compile's own inputs (and `page`), root, package path, fonts and env (no+// ppi, format or out).+export function probeCall(o: { typst: string; root: string; cache: string; inputs: Readonly<Record<string, string | number>>; fonts?: string }): { argv: string[]; env: Record<string, string> } {const argv = [o.typst, 'query','--root', '/','--package-path', `${o.root}/render/packages`,+ ...fontArgs(o.fonts),'--diagnostic-format', 'short',...Object.entries({ ...o.inputs, probe: 1 }).flatMap(([k, v]) => ['--input', `${k}=${v}`]),`${o.root}/render/md.typ`,@@ -352,9 +388,12 @@ export async function probePage(return null}if (!Array.isArray(segments) || segments.length !== o.pages || !job.env.typst) return null+ const typst = job.env.typstconst { inputs, ppi } = typstInputs(job, segments as string[], dir, o.isSafe)- const { argv, env } = probeCall({ typst: job.env.typst, root: job.env.root, cache: job.env.cache, inputs: { ...inputs, page: o.page + 1 } })- const ran = await runKillable($, argv, { env, timeoutMs: COMPILE_TIMEOUT_MS })+ const ran = await runFonts($, job, await leanFor($, job, segments as string[]), fonts => {+ const { argv, env } = probeCall({ typst, root: job.env.root, cache: job.env.cache, inputs: { ...inputs, page: o.page + 1 }, ...(fonts ? { fonts } : {}) })+ return runKillable($, argv, { env, timeoutMs: COMPILE_TIMEOUT_MS })+ })if (ran.exitCode !== 0) {await logDebug($, job.env.cache, `probe ${dir} page ${o.page + 1} failed:\n${ran.stderr}`)return nulldiff --git a/render/measure.typ b/render/measure.typindex cf35565..6a9e720 100644--- a/render/measure.typ+++ b/render/measure.typ@@ -10,10 +10,15 @@// `code-size` that size in body text (none: the `code` input); `pad` the// padding in pt over the first line of a quote, a code block and a table// cell (md.typ rounds it to whole rows).+//+// `fonts`: the theme's `body` and `code` font lists (merged over vscode),+// and `plain`: its list markers and numbering are ASCII only (fonts.ts uses+// the lean font set only then).#import "fonts.typ": sans, mono-base, line-ink#import "theme.typ": T, at-level#let fam = sys.inputs.at("term-font", default: "")#let probe = "the quick brown fox jumps over the lazy dog"+#let is-ascii(v) = type(v) != str or v.match(regex("^[ -~]*$")) != none#let m(fonts, body, edge) = measure(text(font: fonts, size: 100pt, top-edge: edge, bottom-edge: "baseline", fallback: false, body,))@@ -48,5 +53,6 @@code-size: if type(T.code-block.size) in (int, float) { float(T.code-block.size) } else { none },pad: (quote: T.quote.pad-y-em * 11, code: T.code-block.pad-px * px, table: T.table.pad-y-px * px),),+ fonts: (body: sans, code: mono-base, plain: (T.list.markers + (T.list.numbering,)).all(is-ascii)),)) <m>]}diff --git a/tests/fonts.test.ts b/tests/fonts.test.tsnew file mode 100644index 0000000..3279ce7--- /dev/null+++ b/tests/fonts.test.ts@@ -0,0 +1,300 @@+import { expect, test } from 'claude-code/testing'++import { fontListOf, isPlain, LEAN_SH, leanDirOf, leanFiles, leanFontsFor } from '../hooks/lib/fonts'+import { compile, forgetTouches, leanFontsOf, probeCall, typstCall } from '../hooks/lib/typst'+import { DEFAULT_METRICS, leanDirs, type Env, type Job, type LeanFonts } from '../hooks/lib/state'+import { DEFAULT_CELL } from '../hooks/lib/settings'++// `typst fonts --variants` as Typst 0.15 prints it.+const LISTING = [+ 'Apple Color Emoji',+ ' └ /System/Library/Fonts/Apple Color Emoji.ttc',+ ' Style: Normal, Weight: 400, Stretch: 100%',+ 'DejaVu Sans Mono',+ ' ├ (Embedded)',+ ' │ Style: Normal, Weight: 700, Stretch: 100%',+ ' └ (Embedded)',+ ' Style: Normal, Weight: 400, Stretch: 100%',+ 'Helvetica',+ ' ├ /System/Library/Fonts/Helvetica.ttc',+ ' │ Style: Normal, Weight: 400, Stretch: 100%',+ ' └ /System/Library/Fonts/Helvetica.ttc',+ ' Style: Normal, Weight: 400, Stretch: 100%',+ 'Menlo',+ ' ├ /System/Library/Fonts/Menlo.ttc',+ ' │ Style: Normal, Weight: 400, Stretch: 100%',+ ' └ /System/Library/Fonts/Menlo.ttc',+ ' Style: Normal, Weight: 700, Stretch: 100%',+ 'MesloLGS NF',+ ' ├ /Users/me/Library/Fonts/MesloLGS NF Regular.ttf',+ ' │ Style: Normal, Weight: 400, Stretch: 100%',+ ' └ /Users/me/Library/Fonts/MesloLGS NF Bold.ttf',+ ' Style: Normal, Weight: 700, Stretch: 100%',+ 'PingFang SC',+ ' └ /System/Library/AssetsV2/PingFang.ttc',+ ' Style: Normal, Weight: 400, Stretch: 100%',+ 'SF Pro Text',+ ' ├ /Library/Fonts/SF-Pro-Text-Regular.otf',+ ' │ Style: Normal, Weight: 400, Stretch: 100%',+ ' └ /Library/Fonts/SF-Pro-Text-Bold.otf',+ ' Style: Normal, Weight: 700, Stretch: 100%',+ '',+].join('\n')++const FONTS: LeanFonts = {+ body: ['SF Pro Text', 'Helvetica', 'Noto Sans', 'PingFang SC', 'Apple Color Emoji'],+ code: ['Menlo', 'SF Mono', 'DejaVu Sans Mono', 'PingFang SC', 'Apple Color Emoji'],+ plain: true,+}++test('the font list reads each family with its files and variants', () => {+ const list = fontListOf(LISTING)+ expect(list.get('menlo')).toEqual([+ { file: '/System/Library/Fonts/Menlo.ttc', variant: 'Normal, Weight: 400, Stretch: 100%' },+ { file: '/System/Library/Fonts/Menlo.ttc', variant: 'Normal, Weight: 700, Stretch: 100%' },+ ])+ expect(list.get('dejavu sans mono')).toEqual([+ { file: null, variant: 'Normal, Weight: 700, Stretch: 100%' },+ { file: null, variant: 'Normal, Weight: 400, Stretch: 100%' },+ ])+ expect(list.get('apple color emoji')?.length).toBe(1)+ expect([...list.keys()].length).toBe(7)+})++test('the lean set has the terminal font and the Latin fonts, not the CJK and emoji fonts', () => {+ expect(leanFiles(fontListOf(LISTING), 'MesloLGS NF', FONTS)).toEqual([+ '/Users/me/Library/Fonts/MesloLGS NF Regular.ttf',+ '/Users/me/Library/Fonts/MesloLGS NF Bold.ttf',+ '/Library/Fonts/SF-Pro-Text-Regular.otf',+ '/Library/Fonts/SF-Pro-Text-Bold.otf',+ '/System/Library/Fonts/Helvetica.ttc',+ '/System/Library/Fonts/Menlo.ttc',+ ])+ // No terminal font: the theme's fonts only.+ expect(leanFiles(fontListOf(LISTING), '', FONTS)).not.toContain('/Users/me/Library/Fonts/MesloLGS NF Regular.ttf')+})++test('no lean set when it could draw differently', () => {+ const list = fontListOf(LISTING)+ // The first installed body font is a CJK font: plain text would use it.+ expect(leanFiles(list, '', { ...FONTS, body: ['Noto Sans', 'PingFang SC', 'Helvetica'] })).toBeNull()+ // No body font is installed (Typst then takes any font).+ expect(leanFiles(list, '', { ...FONTS, body: ['Inter', 'Noto Sans'] })).toBeNull()+ // One variant in two files: Typst takes the one it read first.+ const twice = `${LISTING}SF Pro Text\n └ /Users/me/Library/Fonts/SF-Pro-Text-Regular.otf\n Style: Normal, Weight: 400, Stretch: 100%\n`+ expect(leanFiles(fontListOf(twice), '', FONTS)).toBeNull()+ // The same variant twice in one file (a collection) is no problem.+ expect(leanFiles(list, '', FONTS)).toContain('/System/Library/Fonts/Helvetica.ttc')+ // A code list of embedded fonts only is installed.+ expect(leanFiles(list, '', { ...FONTS, code: ['DejaVu Sans Mono'] })).not.toBeNull()+})++test('the lean folder is named by its files', () => {+ expect(leanDirOf('/c', ['/a.ttf'])).toBe(leanDirOf('/c', ['/a.ttf']))+ expect(leanDirOf('/c', ['/a.ttf'])).not.toBe(leanDirOf('/c', ['/b.ttf']))+ expect(leanDirOf('/c', ['/a.ttf']).startsWith('/c/fonts-')).toBe(true)+ expect(LEAN_SH).toContain('ln -s')+})++test('only plain text uses the lean set', () => {+ for (const md of [+ '# Title\n\nSome **bold** text, `code` and [a link](https://x.y).',+ '- [x] done\n- [ ] not\n\n| a | b |\n|--|--|\n| 1 | 2 |',+ '```sh\necho $HOME\n```',+ 'Run `echo $PATH` now.',+ 'Run ``a ` $b`` now.',+ '```js\nconst a = `$x`\n```\n\nText.',+ ]) expect(isPlain(md)).toBe(true)+ for (const md of [+ 'café',+ 'Done ✅',+ 'An arrow → here',+ 'Math $x^2$ here',+ '$$\n\\Box\n$$',+ '```typst\n#circle()\n```',+ '~~~typ\n#circle()\n~~~',+ '',+ '<img src="a.png">',+ 'An entity → and →',+ // An escaped backtick: no code span is cut out.+ 'Not code \\` $x$ \\`',+ // Two backticks open; one closes: no span, so $x$ is math.+ '``$x$`',+ // A code span does not go over lines here.+ 'One `a\n$x$ b` two',+ ]) expect(isPlain(md)).toBe(false)+})++test('the lean folder goes into the compile and probe calls only when given', () => {+ const o = { typst: '/t', root: '/r', cache: '/c', ppi: 144, inputs: { theme: 'dark' }, out: '/o/a-{0p}.png' }+ expect(typstCall(o).argv).not.toContain('--ignore-system-fonts')+ const lean = typstCall({ ...o, fonts: '/c/fonts-1' }).argv+ expect(lean.slice(lean.indexOf('--ignore-system-fonts'), lean.indexOf('--ignore-system-fonts') + 3)).toEqual(['--ignore-system-fonts', '--font-path', '/c/fonts-1'])+ expect(probeCall({ ...o, fonts: '/c/fonts-1' }).argv).toContain('/c/fonts-1')+ expect(probeCall(o).argv).not.toContain('--ignore-system-fonts')+})++test('the measure gives the font lists only when they are lists of names', () => {+ expect(leanFontsOf(FONTS)).toEqual(FONTS)+ expect(leanFontsOf({ body: ['A'], code: [1], plain: true })).toBeUndefined()+ expect(leanFontsOf({ body: ['A'], code: ['B'] })).toBeUndefined()+ expect(leanFontsOf(undefined)).toBeUndefined()+})++// ---- On the host ------------------------------------------------------------++const HEAD = '89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 06 40 00 00 00 40'++const envOf = (fonts?: LeanFonts): Env => ({+ typst: '/usr/bin/typst', canDraw: true, inline: false, cache: '/c', cwd: '/w', root: '/r', isMac: true, stamp: '1', home: '/h',+ claudeTmp: '', termFont: 'MesloLGS NF', designs: '/d', metrics: { ...DEFAULT_METRICS, ...(fonts ? { fonts } : {}) },+})+let keys = 0+const jobOf = (env: Env, text: string): Job => ({+ key: `k${++keys}`, requestId: 'r', text, columns: 100, cell: DEFAULT_CELL, theme: 'dark', size: 100, grid: true, themeDir: '/r/render/themes/vscode', env, epoch: 0,+})++// A host with typst (an engine of its own: these modules are this test's+// copy, not the plugin's). `fails(argv)` says when a compile fails. It+// notes the argv of each compile, the font lists asked for, and the+// folders made.+function host(fails: (argv: string[]) => boolean = () => false) {+ const compiles: string[][] = []+ const made: string[] = []+ const listEnvs: Record<string, string>[] = []+ let listed = 0+ const files = new Map<string, string>()+ const run = (stdout: string, exitCode = 0) => ({ exitCode, stdout, stderr: '' })+ const $ = {+ fs: {+ read: async (path: string) => {+ if (!files.has(path)) throw new Error('ENOENT')+ return files.get(path)!+ },+ write: async (path: string, text: string) => {+ files.set(path, text)+ },+ exists: async (path: string) => made.includes(path),+ list: async () => [{ name: 'a-1.png', kind: 'file', size: 1, mtimeMs: 1, isLink: false }],+ },+ process: {+ run: async (argv: string[], init?: { env?: Record<string, string> }) => {+ if (argv[0] === '/usr/bin/typst' && argv[1] === 'fonts') return listed++, listEnvs.push(init?.env ?? {}), run(LISTING)+ if (argv[0] === 'sh' && argv[2] === LEAN_SH) return made.push(argv[4]!), run('')+ if (argv.join(' ').includes('od -An')) return run(argv.filter(a => a.endsWith('.png')).map(() => `${HEAD}\n@\n`).join(''))+ return run('')+ },+ spawn: (request: { argv: string[] }) => {+ const argv = request.argv.slice(4)+ compiles.push(argv)+ const fail = fails(argv)+ return (async function* () {+ yield { stream: 'stdout' as const, text: '4242\n' }+ if (fail) yield { stream: 'stderr' as const, text: 'error: boom' }+ return { code: fail ? 1 : 0, signal: null }+ })()+ },+ },+ clock: { after: () => ({ cancel() {} }) },+ } as any+ return { $, compiles, made, listEnvs, listed: () => listed }+}++// Lets the make of the lean folder (in the background) end.+async function made() {+ for (let i = 0; i < 1000 && ![...leanDirs.values()].some(Boolean); i++) await Promise.resolve()+}++const isLean = (argv: string[]) => argv.includes('--ignore-system-fonts')+const reset = () => {+ leanDirs.clear()+ forgetTouches()+}++test('the first plain compile makes the lean folder; the next ones use it', async () => {+ reset()+ const h = host()+ const $ = h.$+ const env = envOf(FONTS)+ expect(await compile($, jobOf(env, 'First answer.'))).not.toBe('failed')+ expect(isLean(h.compiles[0]!)).toBe(false)+ // The make runs in the background: wait for it.+ await made()+ expect(h.made.length).toBe(1)+ expect([...leanDirs.values()]).toEqual([h.made[0]!])+ expect(await compile($, jobOf(env, 'Second answer.'))).not.toBe('failed')+ expect(isLean(h.compiles[1]!)).toBe(true)+ expect(h.compiles[1]).toContain(h.made[0]!)+ // Text that is not plain uses all fonts.+ expect(await compile($, jobOf(env, 'Math $x$.'))).not.toBe('failed')+ expect(isLean(h.compiles[2]!)).toBe(false)+ expect(h.listed()).toBe(1)+ // The fonts are listed as a compile sees them.+ expect(h.listEnvs[0]!.TYPST_PACKAGE_CACHE_PATH).toBe('/c/pkg-cache')+ reset()+})++test('a lean compile that fails runs again with all fonts', async () => {+ reset()+ const h = host(isLean)+ const $ = h.$+ const env = envOf(FONTS)+ await compile($, jobOf(env, 'Warm up.'))+ await made()+ const result = await compile($, jobOf(env, 'Plain text.'))+ expect(result).not.toBe('failed')+ expect(typeof result === 'object' && result.note).toBeFalsy()+ expect(h.compiles.slice(1).map(isLean)).toEqual([true, false])+ reset()+})++test('no lean set without the font lists, with a theme that is not plain, or when its folder is gone', async () => {+ reset()+ const h = host()+ const $ = h.$+ expect(await leanFontsFor($, envOf(), ['Text.'], {})).toBe('')+ expect(await leanFontsFor($, envOf({ ...FONTS, plain: false }), ['Text.'], {})).toBe('')+ expect(h.listed()).toBe(0)+ const env = envOf(FONTS)+ expect(await leanFontsFor($, env, ['Text.'], {})).toBe('')+ await made()+ expect(await leanFontsFor($, env, ['Text.'], {})).toBe(h.made[0]!)+ // The cleanup removed it: all fonts, and the next call makes it again.+ h.made.length = 0+ expect(await leanFontsFor($, env, ['Text.'], {})).toBe('')+ expect(await leanFontsFor($, env, ['Text.'], {})).toBe('')+ await made()+ expect(h.made.length).toBe(1)+ expect(await leanFontsFor($, env, ['Text.'], {})).toBe(h.made[0]!)+ reset()+})++test('each set of fonts keeps its own folder', async () => {+ reset()+ const h = host()+ const a = envOf(FONTS)+ const b = { ...envOf(FONTS), termFont: '' }+ expect(await leanFontsFor(h.$, a, ['Text.'], {})).toBe('')+ expect(await leanFontsFor(h.$, b, ['Text.'], {})).toBe('')+ for (let i = 0; i < 1000 && [...leanDirs.values()].filter(Boolean).length < 2; i++) await Promise.resolve()+ expect(h.made.length).toBe(2)+ expect(h.made[0]).not.toBe(h.made[1])+ expect(await leanFontsFor(h.$, a, ['Text.'], {})).toBe(h.made[0]!)+ expect(await leanFontsFor(h.$, b, ['Text.'], {})).toBe(h.made[1]!)+ expect(await leanFontsFor(h.$, a, ['Text.'], {})).toBe(h.made[0]!)+ expect(h.listed()).toBe(2)+ reset()+})++test('a font list that does not allow the lean set is asked once, and all fonts are used', async () => {+ reset()+ const h = host()+ const env = envOf({ ...FONTS, body: ['PingFang SC', 'Helvetica'] })+ for (let i = 0; i < 3; i++) {+ expect(await leanFontsFor(h.$, env, ['Text.'], {})).toBe('')+ for (let j = 0; j < 200; j++) await Promise.resolve()+ }+ expect(h.listed()).toBe(1)+ expect(h.made.length).toBe(0)+ reset()+})