Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

feat: add modular text and script surfaces

kashorin39392a5d10d9
diff --git a/README.md b/README.md
index 94cd707e..894a5d59 100644
--- a/README.md
+++ b/README.md
@@ -1,8 +1,11 @@
# SVML
> **v2 bootstrap:** `packages/protocol`, `packages/core` and `packages/driver-node`
-> now implement the domain-free typed state kernel and an explicitly registered Node Host.
-> Frontend, `.svml` and `.svs` parsing are intentionally not part of this slice. The existing
+> implement the domain-free typed state kernel and an explicitly registered Node Host.
+> `packages/text` and `packages/script` now implement the first official authoring vertical:
+> a minimal Text Frontend that learns raw/structured Surfaces from locked module manifests, plus
+> an ordinary Script Surface package using named Segments such as `<opening>...</opening>`.
+> Third-party parser sandboxing, source imports and `.svs` are not implemented yet. The existing
> root compiler and standard library remain the executable v1 research oracle during migration.
Semantic Video Markup Language is a semantic source language for information-flow
diff --git a/packages/core/README.md b/packages/core/README.md
index 289bc225..c4425547 100644
--- a/packages/core/README.md
+++ b/packages/core/README.md
@@ -6,5 +6,7 @@ The package accepts an already resolved module closure, typed authored modules a
finite build plan. It validates identities, schemas, immutable records, Needs, Receipts and
Derivations, then advances a serializable `BuildState` with pure `reduce(state, event)` calls.
-It does not parse source, resolve package locations, execute implementations, access artifact
-bytes, select providers or search for a workflow. Those responsibilities remain outside Core.
+It generically verifies Surface declarations in the same immutable module closure, but never
+parses or dispatches one. It does not parse source, resolve package locations, execute
+implementations, access artifact bytes, select providers or search for a workflow. Those
+responsibilities remain outside Core.
diff --git a/packages/core/src/link.ts b/packages/core/src/link.ts
index 7301b2d1..ad5d819c 100644
--- a/packages/core/src/link.ts
+++ b/packages/core/src/link.ts
@@ -88,6 +88,8 @@ export function verifyClosure(closure: ResolvedModuleClosure): void {
key,
);
ensureUniqueNames(module.manifest.types.map((item) => item.name), "type", key);
+ ensureUniqueNames(module.manifest.surfaces.map((item) => item.name), "surface", key);
+ ensureUniqueNames(module.manifest.surfaces.map((item) => item.tag), "surface tag", key);
ensureUniqueNames(module.manifest.producers.map((item) => item.name), "producer", key);
ensureUniqueNames(
module.manifest.dependencies.map((item) => moduleKey(item.module)),
@@ -104,6 +106,24 @@ export function verifyClosure(closure: ResolvedModuleClosure): void {
`${key}#${producer.name} implementation digest is invalid`,
);
}
+ for (const surface of module.manifest.surfaces) {
+ invariant(surface.tag.length > 0, "EMPTY_NAME", `${key} surface tag is empty`);
+ ensureUniqueNames(
+ surface.outputs.map((output) => typeKey(output)),
+ "Surface output type",
+ `${key}#${surface.name}`,
+ );
+ invariant(
+ surface.mode === "raw" || surface.mode === "structured",
+ "UNSUPPORTED_SURFACE_MODE",
+ `${key}#${surface.name} has unsupported Surface mode ${surface.mode}`,
+ );
+ invariant(
+ isDigest(surface.implementation.digest),
+ "INVALID_DIGEST",
+ `${key}#${surface.name} Surface implementation digest is invalid`,
+ );
+ }
modules.set(key, module);
}
@@ -134,6 +154,21 @@ export function verifyClosure(closure: ResolvedModuleClosure): void {
moduleKey(module.ref),
...module.manifest.dependencies.map((dependency) => moduleKey(dependency.module)),
]);
+ for (const surface of module.manifest.surfaces) {
+ for (const output of surface.outputs) {
+ invariant(
+ allowed.has(moduleKey(output.module)),
+ "UNDECLARED_TYPE_DEPENDENCY",
+ `${moduleKey(module.ref)}#${surface.name} Surface outputs ${typeKey(output)} without a dependency`,
+ );
+ const target = modules.get(moduleKey(output.module));
+ invariant(
+ target?.manifest.types.some((type) => type.name === output.name),
+ "UNKNOWN_TYPE",
+ `${moduleKey(module.ref)}#${surface.name} Surface outputs unknown type ${typeKey(output)}`,
+ );
+ }
+ }
for (const producer of module.manifest.producers) {
for (const port of [...producer.inputs, ...producer.outputs]) {
invariant(
diff --git a/packages/core/test/greeting-fixture.ts b/packages/core/test/greeting-fixture.ts
index c5283861..4162ea76 100644
--- a/packages/core/test/greeting-fixture.ts
+++ b/packages/core/test/greeting-fixture.ts
@@ -62,6 +62,7 @@ export const manifest: ModuleManifest = {
},
},
],
+ surfaces: [],
producers: [
{
name: producers.makePrompt.name,
diff --git a/packages/driver-node/src/manifest.ts b/packages/driver-node/src/manifest.ts
index 481c3c32..7c374e4b 100644
--- a/packages/driver-node/src/manifest.ts
+++ b/packages/driver-node/src/manifest.ts
@@ -12,6 +12,7 @@ import type {
PortDeclaration,
ProducerDeclaration,
ResolvedModuleClosure,
+ SurfaceDeclaration,
TypeDeclaration,
TypeRef,
ValueSchema,
@@ -216,6 +217,23 @@ function producer(value: unknown, path: string): ProducerDeclaration {
};
}
+function surface(value: unknown, path: string): SurfaceDeclaration {
+ const parsed = object(value, path);
+ const mode = string(parsed.mode, `${path}.mode`);
+ if (mode !== "raw" && mode !== "structured") {
+ throw new Error(`${path}.mode must be raw or structured`);
+ }
+ return {
+ name: string(parsed.name, `${path}.name`),
+ tag: string(parsed.tag, `${path}.tag`),
+ mode,
+ outputs: array(parsed.outputs, `${path}.outputs`).map((item, index) =>
+ typeRef(item, `${path}.outputs[${index}]`),
+ ),
+ implementation: implementation(parsed.implementation, `${path}.implementation`),
+ };
+}
+
export function parseModuleManifest(value: unknown): ModuleManifest {
const parsed = object(canonicalize(value), "$manifest");
if (parsed.format !== "svml.module@0") throw new Error("$manifest.format must be svml.module@0");
@@ -229,6 +247,9 @@ export function parseModuleManifest(value: unknown): ModuleManifest {
types: array(parsed.types, "$manifest.types").map((item, index) =>
typeDeclaration(item, `$manifest.types[${index}]`),
),
+ surfaces: array(parsed.surfaces, "$manifest.surfaces").map((item, index) =>
+ surface(item, `$manifest.surfaces[${index}]`),
+ ),
producers: array(parsed.producers, "$manifest.producers").map((item, index) =>
producer(item, `$manifest.producers[${index}]`),
),
diff --git a/packages/driver-node/test/driver.test.ts b/packages/driver-node/test/driver.test.ts
index dadde5d6..8c66c4ab 100644
--- a/packages/driver-node/test/driver.test.ts
+++ b/packages/driver-node/test/driver.test.ts
@@ -4,7 +4,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import test from "node:test";
-import { reduce } from "@svml/core";
+import { digestOf, reduce } from "@svml/core";
import {
HostRegistry,
MemoryArtifactStore,
@@ -159,10 +159,27 @@ test("Driver reads static manifests without executing package code", async () =>
const directory = await mkdtemp(join(tmpdir(), "svml-driver-"));
const path = join(directory, "svml.module.json");
try {
- await writeFile(path, JSON.stringify(manifest), "utf8");
+ const withSurface = {
+ ...manifest,
+ surfaces: [
+ {
+ name: "greeting",
+ tag: "greeting",
+ mode: "structured" as const,
+ outputs: [types.intent],
+ implementation: {
+ kind: "trusted-frontend-surface",
+ locator: "example.greeting/surface",
+ digest: digestOf("example.greeting/surface@0"),
+ },
+ },
+ ],
+ };
+ await writeFile(path, JSON.stringify(withSurface), "utf8");
const closure = await loadResolvedClosure([path]);
assert.equal(closure.modules.length, 1);
assert.equal(closure.modules[0]?.manifest.name, "example.greeting");
+ assert.equal(closure.modules[0]?.manifest.surfaces[0]?.mode, "structured");
} finally {
await rm(directory, { recursive: true, force: true });
}
diff --git a/packages/protocol/README.md b/packages/protocol/README.md
index 4cd083b4..3d873991 100644
--- a/packages/protocol/README.md
+++ b/packages/protocol/README.md
@@ -1,5 +1,8 @@
# `@svml/protocol`
Data-only v2 bootstrap contracts shared by Core, Drivers and future modules. The package defines
-canonical values, module and type references, records and origins, explicit plans, Needs,
-Receipts, Derivations, events and commands. It contains no parser, runtime or video vocabulary.
+canonical values, module/type references, static raw/structured Surface declarations, records and
+origins, explicit plans, Needs, Receipts, Derivations, events and commands. It contains no parser,
+runtime or video vocabulary. A Surface declaration identifies and locks a parser capability; the
+protocol never executes it. Every Surface also declares the exact set of record types it may
+author, so a parser cannot use its early execution position to impersonate another module.
diff --git a/packages/protocol/src/module.ts b/packages/protocol/src/module.ts
index b3b0c63d..02495f6e 100644
--- a/packages/protocol/src/module.ts
+++ b/packages/protocol/src/module.ts
@@ -28,6 +28,21 @@ export type ImplementationRef = {
readonly digest: Digest;
};
+/**
+ * A source Surface is an author-facing declaration exported by a module.
+ * `raw` delegates the complete region after its opening tag to the registered
+ * Surface implementation. `structured` first uses the selected Frontend's
+ * generic element parser and delegates only the resulting tree. `outputs`
+ * bounds which authored record types that parser is allowed to introduce.
+ */
+export type SurfaceDeclaration = {
+ readonly name: string;
+ readonly tag: string;
+ readonly mode: "raw" | "structured";
+ readonly outputs: readonly TypeRef[];
+ readonly implementation: ImplementationRef;
+};
+
export type ProducerDeclaration = {
readonly name: string;
readonly inputs: readonly PortDeclaration[];
@@ -42,6 +57,7 @@ export type ModuleManifest = {
readonly version: string;
readonly dependencies: readonly ModuleDependency[];
readonly types: readonly TypeDeclaration[];
+ readonly surfaces: readonly SurfaceDeclaration[];
readonly producers: readonly ProducerDeclaration[];
};
diff --git a/packages/script/README.md b/packages/script/README.md
new file mode 100644
index 00000000..ad4d4705
--- /dev/null
+++ b/packages/script/README.md
@@ -0,0 +1,32 @@
+# `@svml/script`
+
+Official raw Script Surface for the v2 text Frontend. It parses prose-first named Segment blocks,
+newline-independent Role Cues, Dual Text, Selection, Moment and Slot syntax, and lowers them to a
+canonical authored Narrative value with exactly `2M + 2N` semantic anchor identities.
+
+The package is an ordinary statically declared Surface module. Core does not import it and does not
+know that Script, Segment or Narrative exist.
+
+```svml
+<script id="story">
+ @answer
+
+ <opening>
+ <ALICE> 我先说第一句话。
+ <BOB> 然后我来回答。
+ </opening>
+
+ <pause/>
+ @/answer
+</script>
+```
+
+`<opening>` opens a Segment named `opening`; `</opening>` closes that exact Segment. While the
+parser is inside a Segment, a valid bare tag such as `<ALICE>` is a Role Cue. This is parser state,
+not indentation: the compact spelling
+`<opening><ALICE>我先说。<BOB>我回答。</opening>` has the same semantic value.
+
+The package exports its Manifest, `parseScript`, semantic/source-map projection helpers, a
+semantic-preserving formatter and the raw `decodeScriptSurface` handler. The first bootstrap keeps
+the Narrative contract in this package; it can be split into a domain-contract package later
+without changing Text or Core.
diff --git a/packages/script/package.json b/packages/script/package.json
new file mode 100644
index 00000000..2fcf19cf
--- /dev/null
+++ b/packages/script/package.json
@@ -0,0 +1,13 @@
+{
+ "name": "@svml/script",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "dependencies": {
+ "@svml/core": "workspace:*",
+ "@svml/protocol": "workspace:*"
+ }
+}
diff --git a/packages/script/src/error.ts b/packages/script/src/error.ts
new file mode 100644
index 00000000..ab63e0f1
--- /dev/null
+++ b/packages/script/src/error.ts
@@ -0,0 +1,13 @@
+export class ScriptSyntaxError extends Error {
+ readonly code: string;
+ readonly sourceName: string;
+ readonly offset: number | undefined;
+
+ constructor(code: string, message: string, sourceName: string, offset?: number) {
+ super(`${code}: ${message}${offset === undefined ? "" : ` (${sourceName}:${offset})`}`);
+ this.name = "ScriptSyntaxError";
+ this.code = code;
+ this.sourceName = sourceName;
+ this.offset = offset;
+ }
+}
diff --git a/packages/script/src/format.ts b/packages/script/src/format.ts
new file mode 100644
index 00000000..3029c1fc
--- /dev/null
+++ b/packages/script/src/format.ts
@@ -0,0 +1,83 @@
+import { digestOf } from "@svml/core";
+
+import { ScriptSyntaxError } from "./error.js";
+import { narrativeValue } from "./narrative.js";
+import { parseScript } from "./parser.js";
+
+const ROLE = /^[\p{L}\p{M}\p{N}_](?:[\p{L}\p{M}\p{N}_. -]{0,30}[\p{L}\p{M}\p{N}_.-])?$/u;
+
+function compact(value: string): string {
+ return value.replace(/\s+/gu, " ").trim();
+}
+
+function formatSegment(raw: string, id: string, selfClosing: boolean): string[] {
+ if (selfClosing) return [`<${id}/>`];
+ const openEnd = raw.indexOf(">") + 1;
+ const closeStart = raw.lastIndexOf(`</${id}>`);
+ const body = raw.slice(openEnd, closeStart);
+ const lines: string[] = [`<${id}>`];
+ const turns: string[] = [];
+ let chunkStart = 0;
+ let cursor = 0;
+ while (cursor < body.length) {
+ if (body[cursor] !== "<" || body.startsWith("<!--", cursor)) {
+ cursor += 1;
+ continue;
+ }
+ const end = body.indexOf(">", cursor + 1);
+ if (end < 0) break;
+ const inside = body.slice(cursor + 1, end);
+ if (!inside.includes("|") && ROLE.test(inside)) {
+ const before = compact(body.slice(chunkStart, cursor));
+ if (before) turns.push(before);
+ chunkStart = cursor;
+ }
+ cursor = end + 1;
+ }
+ const tail = compact(body.slice(chunkStart));
+ if (tail) turns.push(tail);
+ lines.push(...turns.map((turn) => ` ${turn}`));
+ lines.push(`</${id}>`);
+ return lines;
+}
+
+function formatOutside(raw: string): string[] {
+ return raw
+ .replace(/\r\n?/gu, "\n")
+ .split("\n")
+ .map((line) => line.trim())
+ .filter(Boolean);
+}
+
+/** Formats a Script body, not the outer `<script>` element. */
+export function formatScript(sourceName: string, source: string): string {
+ const parsed = parseScript(sourceName, source);
+ const output: string[] = [];
+ let cursor = 0;
+ for (const segment of parsed.segments) {
+ const start = segment.range.start;
+ const end = segment.range.end;
+ const outside = formatOutside(source.slice(cursor, start));
+ if (outside.length) {
+ if (output.length) output.push("");
+ output.push(...outside, "");
+ } else if (output.length) {
+ output.push("");
+ }
+ output.push(...formatSegment(source.slice(start, end), segment.id, segment.selfClosing));
+ cursor = end;
+ }
+ const tail = formatOutside(source.slice(cursor));
+ if (tail.length) output.push("", ...tail);
+ while (output.at(-1) === "") output.pop();
+ const formatted = `${output.join("\n")}\n`;
+ const reparsed = parseScript(sourceName, formatted);
+ if (digestOf(narrativeValue(parsed)) !== digestOf(narrativeValue(reparsed))) {
+ throw new ScriptSyntaxError(
+ "SCRIPT_FORMAT_SEMANTICS",
+ "Formatter refused to change Script semantics.",
+ sourceName,
+ );
+ }
+ return formatted;
+}
diff --git a/packages/script/src/index.ts b/packages/script/src/index.ts
new file mode 100644
index 00000000..66af6162
--- /dev/null
+++ b/packages/script/src/index.ts
@@ -0,0 +1,13 @@
+export { ScriptSyntaxError } from "./error.js";
+export { formatScript } from "./format.js";
+export {
+ narrativeSchema,
+ narrativeType,
+ scriptManifest,
+ scriptModuleRef,
+ scriptSurfaceImplementationDigest,
+} from "./manifest.js";
+export { narrativeSourceMap, narrativeValue } from "./narrative.js";
+export { parseScript } from "./parser.js";
+export { decodeScriptSurface } from "./surface.js";
+export type * from "./types.js";
diff --git a/packages/script/src/manifest.ts b/packages/script/src/manifest.ts
new file mode 100644
index 00000000..cdb9564a
--- /dev/null
+++ b/packages/script/src/manifest.ts
@@ -0,0 +1,219 @@
+import { digestOf } from "@svml/core";
+import type { ModuleManifest, TypeRef, ValueSchema } from "@svml/protocol";
+
+export const scriptModuleRef = { name: "@svml/script", version: "0.0.0-dev" } as const;
+
+export const narrativeType: TypeRef = {
+ module: scriptModuleRef,
+ name: "Narrative",
+};
+
+export const scriptSurfaceImplementationDigest = digestOf("@svml/script/surface@0");
+
+const stringSchema = { kind: "string", minLength: 1 } as const;
+const integerSchema = { kind: "number", integer: true, minimum: 0 } as const;
+const boundarySchema: ValueSchema = {
+ kind: "object",
+ fields: {
+ tokenIndex: { schema: integerSchema },
+ structuralPosition: { schema: integerSchema },
+ segmentId: { schema: stringSchema, optional: true },
+ },
+};
+const edgeSchema: ValueSchema = {
+ kind: "object",
+ fields: {
+ affinity: { schema: { kind: "string", enum: ["left", "right"] } },
+ boundary: { schema: boundarySchema },
+ },
+};
+
+export const narrativeSchema: ValueSchema = {
+ kind: "object",
+ fields: {
+ contract: { schema: { kind: "literal", value: "svml.narrative@0" } },
+ segments: {
+ schema: {
+ kind: "array",
+ minItems: 1,
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ index: { schema: integerSchema },
+ startAnchorId: { schema: stringSchema },
+ endAnchorId: { schema: stringSchema },
+ tokenStart: { schema: integerSchema },
+ tokenEndExclusive: { schema: integerSchema },
+ },
+ },
+ },
+ },
+ tokens: {
+ schema: {
+ kind: "array",
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ index: { schema: integerSchema },
+ segmentId: { schema: stringSchema },
+ segmentTokenIndex: { schema: integerSchema },
+ startAnchorId: { schema: stringSchema },
+ endAnchorId: { schema: stringSchema },
+ text: { schema: stringSchema },
+ normalized: { schema: stringSchema },
+ },
+ },
+ },
+ },
+ turns: {
+ schema: {
+ kind: "array",
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ segmentId: { schema: stringSchema },
+ role: { schema: stringSchema, optional: true },
+ tokenStart: { schema: integerSchema },
+ tokenEndExclusive: { schema: integerSchema },
+ },
+ },
+ },
+ },
+ selections: {
+ schema: {
+ kind: "array",
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ occurrences: {
+ schema: {
+ kind: "array",
+ minItems: 1,
+ items: {
+ kind: "object",
+ fields: {
+ occurrence: { schema: integerSchema },
+ open: { schema: edgeSchema },
+ close: { schema: edgeSchema },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ moments: {
+ schema: {
+ kind: "array",
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ occurrences: {
+ schema: {
+ kind: "array",
+ minItems: 1,
+ items: {
+ kind: "object",
+ fields: {
+ occurrence: { schema: integerSchema },
+ affinity: { schema: { kind: "string", enum: ["left", "right"] } },
+ boundary: { schema: boundarySchema },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ },
+ captionAtoms: {
+ schema: {
+ kind: "array",
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ display: { schema: { kind: "string" } },
+ segmentId: { schema: stringSchema },
+ startToken: { schema: integerSchema },
+ endTokenExclusive: { schema: integerSchema },
+ },
+ },
+ },
+ },
+ semanticIndex: {
+ schema: {
+ kind: "object",
+ fields: {
+ contract: { schema: { kind: "literal", value: "svml.semantic-index@0" } },
+ anchors: {
+ schema: {
+ kind: "array",
+ minItems: 2,
+ items: {
+ kind: "object",
+ fields: {
+ id: { schema: stringSchema },
+ kind: {
+ schema: {
+ kind: "string",
+ enum: ["segment-start", "token-start", "token-end", "segment-end"],
+ },
+ },
+ segmentId: { schema: stringSchema },
+ tokenId: { schema: stringSchema, optional: true },
+ segmentTokenIndex: { schema: integerSchema, optional: true },
+ },
+ },
+ },
+ },
+ digest: { schema: stringSchema },
+ },
+ },
+ },
+ projections: {
+ schema: {
+ kind: "object",
+ fields: {
+ dialogue: { schema: { kind: "string" } },
+ speech: { schema: { kind: "string" } },
+ caption: { schema: { kind: "string" } },
+ },
+ },
+ },
+ },
+};
+
+export const scriptManifest: ModuleManifest = {
+ format: "svml.module@0",
+ name: scriptModuleRef.name,
+ version: scriptModuleRef.version,
+ dependencies: [],
+ types: [
+ {
+ name: narrativeType.name,
+ schema: narrativeSchema,
+ description: "Authoritative narrative, projections and 2M + 2N semantic identities.",
+ },
+ ],
+ surfaces: [
+ {
+ name: "script",
+ tag: "script",
+ mode: "raw",
+ outputs: [narrativeType],
+ implementation: {
+ kind: "trusted-frontend-surface",
+ locator: "@svml/script/surface",
+ digest: scriptSurfaceImplementationDigest,
+ },
+ },
+ ],
+ producers: [],
+};
diff --git a/packages/script/src/narrative.ts b/packages/script/src/narrative.ts
new file mode 100644
index 00000000..3491171b
--- /dev/null
+++ b/packages/script/src/narrative.ts
@@ -0,0 +1,99 @@
+import { canonicalize } from "@svml/core";
+import type { CanonicalValue } from "@svml/protocol";
+
+import type { ParsedNarrative } from "./types.js";
+
+function semanticBoundary(boundary: {
+ readonly tokenIndex: number;
+ readonly structuralPosition: number;
+ readonly segmentId?: string;
+}): CanonicalValue {
+ return canonicalize(boundary);
+}
+
+export function narrativeValue(parsed: ParsedNarrative): CanonicalValue {
+ return canonicalize({
+ contract: "svml.narrative@0",
+ segments: parsed.segments.map((segment) => ({
+ id: segment.id,
+ index: segment.index,
+ startAnchorId: segment.startAnchorId,
+ endAnchorId: segment.endAnchorId,
+ tokenStart: segment.tokenStart,
+ tokenEndExclusive: segment.tokenEndExclusive,
+ })),
+ tokens: parsed.tokens.map((token) => ({
+ id: token.id,
+ index: token.index,
+ segmentId: token.segmentId,
+ segmentTokenIndex: token.segmentTokenIndex,
+ startAnchorId: token.startAnchorId,
+ endAnchorId: token.endAnchorId,
+ text: token.text,
+ normalized: token.normalized,
+ })),
+ turns: parsed.turns.map((turn) => ({
+ id: turn.id,
+ segmentId: turn.segmentId,
+ ...(turn.role === undefined ? {} : { role: turn.role }),
+ tokenStart: turn.tokenStart,
+ tokenEndExclusive: turn.tokenEndExclusive,
+ })),
+ selections: parsed.selections.map((selection) => ({
+ id: selection.id,
+ occurrences: selection.occurrences.map((occurrence) => ({
+ occurrence: occurrence.occurrence,
+ open: {
+ affinity: occurrence.open.affinity,
+ boundary: semanticBoundary(occurrence.open.boundary),
+ },
+ close: {
+ affinity: occurrence.close.affinity,
+ boundary: semanticBoundary(occurrence.close.boundary),
+ },
+ })),
+ })),
+ moments: parsed.moments.map((moment) => ({
+ id: moment.id,
+ occurrences: moment.occurrences.map((occurrence) => ({
+ occurrence: occurrence.occurrence,
+ affinity: occurrence.affinity,
+ boundary: semanticBoundary(occurrence.boundary),
+ })),
+ })),
+ captionAtoms: parsed.captionAtoms.map((atom) => ({
+ id: atom.id,
+ display: atom.display,
+ segmentId: atom.segmentId,
+ startToken: atom.startToken,
+ endTokenExclusive: atom.endTokenExclusive,
+ })),
+ semanticIndex: parsed.semanticIndex,
+ projections: parsed.projections,
+ });
+}
+
+export function narrativeSourceMap(recordId: string, parsed: ParsedNarrative): CanonicalValue {
+ return canonicalize({
+ format: "svml.script-source-map@0",
+ record: recordId,
+ segments: parsed.segments.map((segment) => ({ id: segment.id, range: segment.range })),
+ tokens: parsed.tokens.map((token) => ({ id: token.id, range: token.range })),
+ turns: parsed.turns.map((turn) => ({ id: turn.id, range: turn.range })),
+ selections: parsed.selections.map((selection) => ({
+ id: selection.id,
+ occurrences: selection.occurrences.map((occurrence) => ({
+ occurrence: occurrence.occurrence,
+ open: occurrence.open.range,
+ close: occurrence.close.range,
+ })),
+ })),
+ moments: parsed.moments.map((moment) => ({
+ id: moment.id,
+ occurrences: moment.occurrences.map((occurrence) => ({
+ occurrence: occurrence.occurrence,
+ range: occurrence.range,
+ })),
+ })),
+ });
+}
diff --git a/packages/script/src/parser.ts b/packages/script/src/parser.ts
new file mode 100644
index 00000000..7b385e7b
--- /dev/null
+++ b/packages/script/src/parser.ts
@@ -0,0 +1,605 @@
+import { digestOf } from "@svml/core";
+
+import { ScriptSyntaxError } from "./error.js";
+import type {
+ Affinity,
+ MarkerBoundary,
+ ParsedAtom,
+ ParsedCaptionAtom,
+ ParsedMoment,
+ ParsedNarrative,
+ ParsedSegment,
+ ParsedSelection,
+ ParsedToken,
+ ParsedTurn,
+ SemanticAnchor,
+} from "./types.js";
+
+type Marker = {
+ readonly id: string;
+ readonly kind: "open" | "close" | "moment";
+ readonly affinity: Affinity;
+ readonly length: number;
+};
+
+type MutableSegment = {
+ readonly id: string;
+ readonly index: number;
+ readonly atoms: ParsedAtom[];
+ readonly tokenStart: number;
+ readonly sourceStart: number;
+ lexicalRun: string;
+ readonly lexicalMarkers: Array<{ readonly position: number; readonly offset: number }>;
+};
+
+const WORD =
+ /[\p{Script=Han}\p{Script=Hiragana}\p{Script=Katakana}]\p{M}*|[\p{L}\p{M}\p{N}]+(?:['’.-][\p{L}\p{M}\p{N}]+)*/gu;
+const SEGMENT_ID = /^[a-z][a-z0-9_-]{0,63}$/u;
+const TEMPORAL_ID = /^[a-z][a-z0-9_-]{0,63}$/u;
+const ROLE_LABEL = /^[\p{L}\p{M}\p{N}_](?:[\p{L}\p{M}\p{N}_. -]{0,30}[\p{L}\p{M}\p{N}_.-])?$/u;
+const RESERVED_SEGMENT_IDS = new Set(["script"]);
+
+function normalizeWord(value: string): string {
+ return value
+ .normalize("NFKC")
+ .toLocaleLowerCase("en")
+ .replace(/[^\p{L}\p{M}\p{N}]+/gu, "");
+}
+
+function cleanProjection(value: string): string {
+ return value
+ .replace(/\s+/gu, " ")
+ .replace(/\s+([,.;:!?])/gu, "$1")
+ .replace(/([([{])\s+/gu, "$1")
+ .replace(/\s+([)\]}])/gu, "$1")
+ .trim();
+}
+
+function joinProjection(parts: readonly string[]): string {
+ return cleanProjection(parts.filter((part) => part.trim()).join(" "));
+}
+
+function parseMarker(source: string, offset: number): Marker | undefined {
+ const markerName = String.raw`[a-z][a-z0-9_-]{0,63}`;
+ const rest = source.slice(offset);
+ const closeRight = new RegExp(String.raw`^@\/(${markerName})~`, "u").exec(rest);
+ if (closeRight) {
+ return { id: closeRight[1]!, kind: "close", affinity: "right", length: closeRight[0].length };
+ }
+ const closeLeft = new RegExp(String.raw`^@\/(${markerName})(?![A-Za-z0-9_-])`, "u").exec(rest);
+ if (closeLeft) {
+ return { id: closeLeft[1]!, kind: "close", affinity: "left", length: closeLeft[0].length };
+ }
+ const moment = new RegExp(String.raw`^(~)?@(${markerName})!`, "u").exec(rest);
+ if (moment) {
+ return {
+ id: moment[2]!,
+ kind: "moment",
+ affinity: moment[1] === "~" ? "left" : "right",
+ length: moment[0].length,
+ };
+ }
+ const open = new RegExp(String.raw`^(~)?@(${markerName})(?![A-Za-z0-9_-])`, "u").exec(rest);
+ if (!open) return undefined;
+ return {
+ id: open[2]!,
+ kind: "open",
+ affinity: open[1] === "~" ? "left" : "right",
+ length: open[0].length,
+ };
+}
+
+function segmentAnchorId(segmentId: string, edge: "start" | "end"): string {
+ return `segment:${segmentId}:${edge}`;
+}
+
+function tokenId(segmentId: string, segmentTokenIndex: number): string {
+ return `segment:${segmentId}:token:${segmentTokenIndex + 1}`;
+}
+
+function tokenAnchorId(segmentId: string, segmentTokenIndex: number, edge: "start" | "end"): string {
+ return `${tokenId(segmentId, segmentTokenIndex)}:${edge}`;
+}
+
+function findUnescaped(value: string, character: string, from = 0): number {
+ for (let index = from; index < value.length; index += 1) {
+ if (value[index] !== character) continue;
+ let slashes = 0;
+ for (let before = index - 1; before >= 0 && value[before] === "\\"; before -= 1) slashes += 1;
+ if (slashes % 2 === 0) return index;
+ }
+ return -1;
+}
+
+export function parseScript(
+ sourceName: string,
+ input: string,
+ sourceOffset = 0,
+ bindings: Readonly<Record<string, string>> = {},
+): ParsedNarrative {
+ const source = input.replace(/\r\n?/gu, "\n").normalize("NFC");
+ const fail = (code: string, message: string, offset?: number): never => {
+ throw new ScriptSyntaxError(code, message, sourceName, offset === undefined ? undefined : sourceOffset + offset);
+ };
+
+ const segments: ParsedSegment[] = [];
+ const tokens: ParsedToken[] = [];
+ const captionAtoms: ParsedCaptionAtom[] = [];
+ const selections = new Map<string, ParsedSelection["occurrences"] extends readonly (infer T)[] ? T[] : never>();
+ const moments = new Map<string, ParsedMoment["occurrences"] extends readonly (infer T)[] ? T[] : never>();
+ const openSelections = new Map<string, {
+ readonly affinity: Affinity;
+ readonly boundary: MarkerBoundary;
+ readonly start: number;
+ readonly end: number;
+ }>();
+ let current: MutableSegment | undefined;
+ let offset = 0;
+ let structuralPosition = 0;
+
+ const boundary = (): MarkerBoundary => ({
+ tokenIndex: tokens.length,
+ structuralPosition,
+ ...(current ? { segmentId: current.id } : {}),
+ });
+
+ const finishLexicalRun = (): void => {
+ if (!current) return;
+ WORD.lastIndex = 0;
+ let match: RegExpExecArray | null;
+ while ((match = WORD.exec(current.lexicalRun))) {
+ const start = match.index;
+ const end = start + match[0].length;
+ const split = current.lexicalMarkers.find((marker) => start < marker.position && marker.position < end);
+ if (split) {
+ fail(
+ "SCRIPT_MARKER_TOKEN_BOUNDARY",
+ "A Selection or Moment marker cannot split a speech token.",
+ split.offset,
+ );
+ }
+ }
+ current.lexicalRun = "";
+ current.lexicalMarkers.length = 0;
+ };
+
+ const addMarker = (marker: Marker, start: number): void => {
+ if (!TEMPORAL_ID.test(marker.id)) fail("SCRIPT_TEMPORAL_ID", `Invalid temporal id "${marker.id}".`, start);
+ if (current) current.lexicalMarkers.push({ position: current.lexicalRun.length, offset: start });
+ if (marker.kind === "moment") {
+ if (openSelections.has(marker.id) || selections.has(marker.id)) {
+ fail("SCRIPT_TEMPORAL_TYPE", `Temporal name "${marker.id}" cannot be both Selection and Moment.`, start);
+ }
+ const values = moments.get(marker.id) ?? [];
+ values.push({
+ occurrence: values.length,
+ affinity: marker.affinity,
+ boundary: boundary(),
+ range: { start: sourceOffset + start, end: sourceOffset + start + marker.length },
+ });
+ moments.set(marker.id, values);
+ return;
+ }
+ if (marker.kind === "open") {
+ if (moments.has(marker.id)) {
+ fail("SCRIPT_TEMPORAL_TYPE", `Temporal name "${marker.id}" cannot be both Selection and Moment.`, start);
+ }
+ if (openSelections.has(marker.id)) {
+ fail("SCRIPT_SELECTION_REOPENED", `Selection "${marker.id}" is reopened before it closes.`, start);
+ }
+ openSelections.set(marker.id, {
+ affinity: marker.affinity,
+ boundary: boundary(),
+ start: sourceOffset + start,
+ end: sourceOffset + start + marker.length,
+ });
+ return;
+ }
+ const open = openSelections.get(marker.id)
+ ?? fail("SCRIPT_SELECTION_CLOSE", `Selection "${marker.id}" closes without an open.`, start);
+ openSelections.delete(marker.id);
+ const values = selections.get(marker.id) ?? [];
+ values.push({
+ occurrence: values.length,
+ open: {
+ affinity: open.affinity,
+ boundary: open.boundary,
+ range: { start: open.start, end: open.end },
+ },
+ close: {
+ affinity: marker.affinity,
+ boundary: boundary(),
+ range: { start: sourceOffset + start, end: sourceOffset + start + marker.length },
+ },
+ });
+ selections.set(marker.id, values);
+ };
+
+ const literalPieces = (
+ raw: string,
+ absoluteStart: number,
+ dual = false,
+ ): Array<{ readonly value: string; readonly start: number; readonly end: number }> => {
+ const pieces: Array<{ value: string; start: number; end: number }> = [];
+ let buffer = "";
+ let bufferStart = 0;
+ const flush = (end: number): void => {
+ if (buffer) pieces.push({ value: buffer, start: absoluteStart + bufferStart, end: absoluteStart + end });
+ buffer = "";
+ bufferStart = end;
+ };
+ for (let index = 0; index < raw.length;) {
+ if (raw.startsWith("${", index)) {
+ flush(index);
+ const close = raw.indexOf("}", index + 2);
+ if (close < 0) fail("SCRIPT_SLOT_UNCLOSED", "Unclosed Script Slot.", absoluteStart + index);
+ const id = raw.slice(index + 2, close);
+ if (!/^[\p{L}_][\p{L}\p{M}\p{N}_-]{0,63}$/u.test(id)) {
+ fail("SCRIPT_SLOT_ID", `Invalid Script Slot "${id}".`, absoluteStart + index);
+ }
+ const value = bindings[id]
+ ?? fail("SCRIPT_SLOT_UNBOUND", `Script Slot "${id}" is not bound.`, absoluteStart + index);
+ if (/[\r\n\p{Cc}]/u.test(value)) fail("SCRIPT_SLOT_VALUE", `Script Slot "${id}" must be single-line text.`);
+ pieces.push({ value, start: absoluteStart + index, end: absoluteStart + close + 1 });
+ index = close + 1;
+ bufferStart = index;
+ continue;
+ }
+ if (raw[index] === "\\") {
+ const known = raw.startsWith("\\${", index)
+ ? { value: "${", length: 3 }
+ : ["@", "<", "\\", ...(dual ? ["|", ">"] : [])].includes(raw[index + 1] ?? "")
+ ? { value: raw[index + 1]!, length: 2 }
+ : undefined;
+ const escape = known
+ ?? fail("SCRIPT_ESCAPE", `Unknown Script escape "${raw.slice(index, index + 2)}".`, absoluteStart + index);
+ buffer += escape.value;
+ index += escape.length;
+ continue;
+ }
+ buffer += raw[index];
+ index += 1;
+ }
+ flush(raw.length);
+ return pieces;
+ };
+
+ const literalString = (raw: string, start: number, dual = false): string =>
+ literalPieces(raw, start, dual).map((piece) => piece.value).join("");
+
+ const addText = (speech: string, caption: string, start: number, end: number): void => {
+ if (!current) {
+ if (speech.trim() || caption.trim()) {
+ fail("SCRIPT_TEXT_OUTSIDE_SEGMENT", "Natural-language text is only allowed inside a named Segment.", start);
+ }
+ return;
+ }
+ current.lexicalRun += speech;
+ const tokenStart = tokens.length;
+ WORD.lastIndex = 0;
+ let match: RegExpExecArray | null;
+ while ((match = WORD.exec(speech))) {
+ const normalized = normalizeWord(match[0]);
+ if (!normalized) continue;
+ const index = tokens.length;
+ const segmentTokenIndex = index - current.tokenStart;
+ const id = tokenId(current.id, segmentTokenIndex);
+ tokens.push({
+ id,
+ index,
+ segmentId: current.id,
+ segmentTokenIndex,
+ startAnchorId: tokenAnchorId(current.id, segmentTokenIndex, "start"),
+ endAnchorId: tokenAnchorId(current.id, segmentTokenIndex, "end"),
+ text: match[0],
+ normalized,
+ range: {
+ start: sourceOffset + start + match.index,
+ end: sourceOffset + start + match.index + match[0].length,
+ },
+ });
+ }
+ current.atoms.push({
+ kind: "text",
+ speech,
+ caption,
+ tokenStart,
+ tokenEndExclusive: tokens.length,
+ range: { start: sourceOffset + start, end: sourceOffset + end },
+ });
+ };
+
+ const consumeSpeechSide = (raw: string, absoluteStart: number, caption: string): void => {
+ const startToken = tokens.length;
+ let partStart = 0;
+ let index = 0;
+ let emittedCaption = false;
+ const addLiteral = (part: string, start: number): void => {
+ for (const piece of literalPieces(part, start, true)) {
+ addText(piece.value, emittedCaption ? "" : caption, piece.start, piece.end);
+ emittedCaption = true;
+ }
+ };
+ while (index < raw.length) {
+ if (raw.startsWith("${", index)) {
+ const close = raw.indexOf("}", index + 2);
+ if (close < 0) fail("SCRIPT_SLOT_UNCLOSED", "Unclosed Script Slot.", absoluteStart + index);
+ index = close + 1;
+ continue;
+ }
+ if (raw[index] === "\\") {
+ index += raw.startsWith("\\${", index) ? 3 : 2;
+ continue;
+ }
+ const marker = parseMarker(raw, index);
+ if (!marker) {
+ if (raw[index] === "@") fail("SCRIPT_MARKER", "Unescaped @ must begin a valid marker.", absoluteStart + index);
+ index += 1;
+ continue;
+ }
+ addLiteral(raw.slice(partStart, index), absoluteStart + partStart);
+ addMarker(marker, absoluteStart + index);
+ index += marker.length;
+ partStart = index;
+ }
+ addLiteral(raw.slice(partStart), absoluteStart + partStart);
+ if (tokens.length > startToken) {
+ captionAtoms.push({
+ id: `caption-atom:${captionAtoms.length + 1}`,
+ display: cleanProjection(caption),
+ segmentId: current!.id,
+ startToken,
+ endTokenExclusive: tokens.length,
+ range: { start: sourceOffset + absoluteStart, end: sourceOffset + absoluteStart + raw.length },
+ });
+ }
+ };
+
+ const closeCurrent = (end: number, selfClosing: boolean): void => {
+ const segment = current
+ ?? fail("SCRIPT_SEGMENT_CLOSE", "Segment close has no matching open.", end);
+ finishLexicalRun();
+ segments.push({
+ id: segment.id,
+ index: segment.index,
+ startAnchorId: segmentAnchorId(segment.id, "start"),
+ endAnchorId: segmentAnchorId(segment.id, "end"),
+ tokenStart: segment.tokenStart,
+ tokenEndExclusive: tokens.length,
+ atoms: segment.atoms,
+ range: { start: segment.sourceStart, end: sourceOffset + end },
+ selfClosing,
+ });
+ current = undefined;
+ structuralPosition += 1;
+ };
+
+ while (offset < source.length) {
+ if (source.startsWith("<!--", offset)) {
+ finishLexicalRun();
+ const end = source.indexOf("-->", offset + 4);
+ if (end < 0) fail("SCRIPT_COMMENT", "Unclosed Script comment.", offset);
+ offset = end + 3;
+ continue;
+ }
+
+ if (!current && source[offset] === "<") {
+ const self = /^<([a-z][a-z0-9_-]{0,63})\s*\/>/u.exec(source.slice(offset));
+ const open = self ?? /^<([a-z][a-z0-9_-]{0,63})\s*>/u.exec(source.slice(offset));
+ if (open) {
+ const id = open[1]!;
+ if (!SEGMENT_ID.test(id) || RESERVED_SEGMENT_IDS.has(id)) {
+ fail("SCRIPT_SEGMENT_ID", `Invalid or reserved Segment id "${id}".`, offset);
+ }
+ if (segments.some((segment) => segment.id === id)) {
+ fail("SCRIPT_SEGMENT_DUPLICATE", `Duplicate Segment id "${id}".`, offset);
+ }
+ current = {
+ id,
+ index: segments.length,
+ atoms: [],
+ tokenStart: tokens.length,
+ sourceStart: sourceOffset + offset,
+ lexicalRun: "",
+ lexicalMarkers: [],
+ };
+ offset += open[0].length;
+ if (self) closeCurrent(offset, true);
+ continue;
+ }
+ const close = /^<\/([^>]+)>/u.exec(source.slice(offset));
+ if (close) fail("SCRIPT_SEGMENT_CLOSE", `Unexpected Segment close </${close[1]}>.`, offset);
+ fail(
+ "SCRIPT_SEGMENT_OPEN",
+ "Script body expects a lower-case named Segment without attributes.",
+ offset,
+ );
+ }
+
+ if (current && source.startsWith("</", offset)) {
+ const close = /^<\/([a-z][a-z0-9_-]{0,63})\s*>/u.exec(source.slice(offset))
+ ?? fail("SCRIPT_SEGMENT_CLOSE", "Malformed Segment close.", offset);
+ if (close[1] !== current.id) {
+ fail("SCRIPT_SEGMENT_MISMATCH", `Segment "${current.id}" was closed by "${close[1]}".`, offset);
+ }
+ offset += close[0].length;
+ closeCurrent(offset, false);
+ continue;
+ }
+
+ const marker = parseMarker(source, offset);
+ if (marker) {
+ addMarker(marker, offset);
+ offset += marker.length;
+ continue;
+ }
+ if (source[offset] === "@") fail("SCRIPT_MARKER", "Unescaped @ must begin a valid marker.", offset);
+
+ if (source[offset] === "<" && current) {
+ const end = findUnescaped(source, ">", offset + 1);
+ if (end < 0) fail("SCRIPT_ANGLE", "Unclosed Script angle construct.", offset);
+ const inside = source.slice(offset + 1, end);
+ const pipe = findUnescaped(inside, "|");
+ if (pipe >= 0) {
+ finishLexicalRun();
+ const display = literalString(inside.slice(0, pipe), offset + 1, true);
+ const speech = inside.slice(pipe + 1);
+ if (!speech.replace(/~?@\/?[A-Za-z_][A-Za-z0-9_.-]*!?~?/gu, "").trim()) {
+ fail("SCRIPT_DUAL_EMPTY", "Dual Text speech side must not be empty.", offset);
+ }
+ consumeSpeechSide(speech, offset + pipe + 2, display);
+ finishLexicalRun();
+ offset = end + 1;
+ continue;
+ }
+ if (!ROLE_LABEL.test(inside)) fail("SCRIPT_ANGLE", `Unknown Script construct <${inside}>.`, offset);
+ finishLexicalRun();
+ current.atoms.push({
+ kind: "role",
+ label: inside,
+ range: { start: sourceOffset + offset, end: sourceOffset + end + 1 },
+ });
+ offset = end + 1;
+ continue;
+ }
+
+ const textStart = offset;
+ while (offset < source.length) {
+ if (
+ source.startsWith("<!--", offset)
+ || parseMarker(source, offset)
+ || source[offset] === "<"
+ || source[offset] === "@"
+ ) break;
+ if (source[offset] === "\\") {
+ offset += source.startsWith("\\${", offset) ? 3 : Math.min(2, source.length - offset);
+ continue;
+ }
+ offset += 1;
+ }
+ const raw = source.slice(textStart, offset);
+ for (const piece of literalPieces(raw, textStart)) addText(piece.value, piece.value, piece.start, piece.end);
+ }
+
+ if (current) fail("SCRIPT_SEGMENT_UNCLOSED", `Segment "${current.id}" is not closed.`);
+ const dangling = openSelections.keys().next().value as string | undefined;
+ if (dangling) fail("SCRIPT_SELECTION_UNCLOSED", `Selection "${dangling}" is not closed.`);
+ if (!segments.length) fail("SCRIPT_SEGMENT_CARDINALITY", "Script requires at least one Segment.");
+
+ const speechSegments: string[] = [];
+ const captionSegments: string[] = [];
+ const dialogueTurns: string[] = [];
+ const turns: ParsedTurn[] = [];
+ for (const segment of segments) {
+ const speechParts: string[] = [];
+ const captionParts: string[] = [];
+ let activeRole: { readonly label: string; readonly start: number } | undefined;
+ let sawRole = false;
+ let unownedSpeech = "";
+ let turnParts: string[] = [];
+ let turnTokenStart: number | undefined;
+ let turnTokenEndExclusive: number | undefined;
+ let turnEnd = segment.range.end;
+ const flushTurn = (): void => {
+ const body = joinProjection(turnParts);
+ if (activeRole && !body) {
+ fail("SCRIPT_ROLE_EMPTY", `Role Cue <${activeRole.label}> is not followed by spoken content.`, activeRole.start - sourceOffset);
+ }
+ if (body) dialogueTurns.push(activeRole ? `${activeRole.label}: ${body}` : body);
+ if (turnTokenStart !== undefined && turnTokenEndExclusive !== undefined) {
+ const ordinal = turns.filter((turn) => turn.segmentId === segment.id).length + 1;
+ turns.push({
+ id: `segment:${segment.id}:turn:${ordinal}`,
+ segmentId: segment.id,
+ ...(activeRole ? { role: activeRole.label } : {}),
+ tokenStart: turnTokenStart,
+ tokenEndExclusive: turnTokenEndExclusive,
+ range: { start: activeRole?.start ?? segment.range.start, end: turnEnd },
+ });
+ }
+ turnParts = [];
+ turnTokenStart = undefined;
+ turnTokenEndExclusive = undefined;
+ };
+ for (const atom of segment.atoms) {
+ if (atom.kind === "role") {
+ if (!sawRole && cleanProjection(unownedSpeech)) {
+ fail("SCRIPT_ROLE_AFTER_TEXT", `Segment "${segment.id}" introduces a Role Cue after unowned speech.`, atom.range.start - sourceOffset);
+ }
+ flushTurn();
+ activeRole = { label: atom.label, start: atom.range.start };
+ sawRole = true;
+ continue;
+ }
+ speechParts.push(atom.speech);
+ captionParts.push(atom.caption);
+ if (!sawRole) unownedSpeech += atom.speech;
+ turnParts.push(atom.speech);
+ if (atom.tokenEndExclusive > atom.tokenStart) {
+ turnTokenStart ??= atom.tokenStart;
+ turnTokenEndExclusive = atom.tokenEndExclusive;
+ turnEnd = atom.range.end;
+ }
+ }
+ flushTurn();
+ const speech = joinProjection(speechParts);
+ const caption = joinProjection(captionParts);
+ if (speech) speechSegments.push(speech);
+ if (caption) captionSegments.push(caption);
+ }
+
+ const anchors: SemanticAnchor[] = segments.flatMap((segment) => [
+ { id: segment.startAnchorId, kind: "segment-start" as const, segmentId: segment.id },
+ ...tokens.slice(segment.tokenStart, segment.tokenEndExclusive).flatMap((token) => [
+ {
+ id: token.startAnchorId,
+ kind: "token-start" as const,
+ segmentId: segment.id,
+ tokenId: token.id,
+ segmentTokenIndex: token.segmentTokenIndex,
+ },
+ {
+ id: token.endAnchorId,
+ kind: "token-end" as const,
+ segmentId: segment.id,
+ tokenId: token.id,
+ segmentTokenIndex: token.segmentTokenIndex,
+ },
+ ]),
+ { id: segment.endAnchorId, kind: "segment-end" as const, segmentId: segment.id },
+ ]);
+ if (anchors.length !== 2 * tokens.length + 2 * segments.length) {
+ fail("SCRIPT_ANCHOR_CARDINALITY", "Semantic anchor cardinality is not 2M + 2N.");
+ }
+ const semanticPayload = {
+ contract: "svml.semantic-index@0" as const,
+ anchors,
+ tokens: tokens.map((token) => ({
+ id: token.id,
+ segmentId: token.segmentId,
+ segmentTokenIndex: token.segmentTokenIndex,
+ normalized: token.normalized,
+ startAnchorId: token.startAnchorId,
+ endAnchorId: token.endAnchorId,
+ })),
+ };
+
+ return {
+ segments,
+ tokens,
+ turns,
+ selections: [...selections].sort(([left], [right]) => left.localeCompare(right)).map(([id, occurrences]) => ({ id, occurrences })),
+ moments: [...moments].sort(([left], [right]) => left.localeCompare(right)).map(([id, occurrences]) => ({ id, occurrences })),
+ captionAtoms,
+ semanticIndex: {
+ contract: "svml.semantic-index@0",
+ anchors,
+ digest: digestOf(semanticPayload),
+ },
+ projections: {
+ dialogue: dialogueTurns.join("\n"),
+ speech: speechSegments.join("\n"),
+ caption: captionSegments.join("\n"),
+ },
+ };
+}
diff --git a/packages/script/src/surface.ts b/packages/script/src/surface.ts
new file mode 100644
index 00000000..d8af31e2
--- /dev/null
+++ b/packages/script/src/surface.ts
@@ -0,0 +1,75 @@
+import { canonicalize } from "@svml/core";
+
+import { ScriptSyntaxError } from "./error.js";
+import { narrativeSourceMap, narrativeValue } from "./narrative.js";
+import { narrativeType } from "./manifest.js";
+import { parseScript } from "./parser.js";
+import type { ScriptSurfaceInput, ScriptSurfaceOutput } from "./types.js";
+
+const RECORD_ID = /^[a-z][a-z0-9_-]{0,63}$/u;
+
+function findClose(input: ScriptSurfaceInput): { readonly start: number; readonly end: number } {
+ const close = `</${input.tag}>`;
+ let cursor = input.contentStart;
+ while (cursor < input.source.length) {
+ if (input.source.startsWith("<!--", cursor)) {
+ const commentEnd = input.source.indexOf("-->", cursor + 4);
+ if (commentEnd < 0) {
+ throw new ScriptSyntaxError("SCRIPT_COMMENT", "Unclosed Script comment.", input.sourceName, cursor);
+ }
+ cursor = commentEnd + 3;
+ continue;
+ }
+ if (input.source.startsWith(close, cursor)) {
+ let slashes = 0;
+ for (let before = cursor - 1; before >= 0 && input.source[before] === "\\"; before -= 1) slashes += 1;
+ if (slashes % 2 === 0) return { start: cursor, end: cursor + close.length };
+ }
+ cursor += 1;
+ }
+ throw new ScriptSyntaxError(
+ "SCRIPT_SURFACE_UNCLOSED",
+ `Raw Surface <${input.tag}> is not closed.`,
+ input.sourceName,
+ input.openingStart,
+ );
+}
+
+export function decodeScriptSurface(input: ScriptSurfaceInput): ScriptSurfaceOutput {
+ const unknown = Object.keys(input.attributes).filter((name) => name !== "id");
+ if (unknown.length) {
+ throw new ScriptSyntaxError(
+ "SCRIPT_SURFACE_ATTRIBUTE",
+ `<${input.tag}> does not declare attribute "${unknown[0]}".`,
+ input.sourceName,
+ input.openingStart,
+ );
+ }
+ const rawId = input.attributes.id ?? "script";
+ if (typeof rawId !== "string" || !RECORD_ID.test(rawId)) {
+ throw new ScriptSyntaxError(
+ "SCRIPT_SURFACE_ID",
+ `<${input.tag}> id must be a canonical lower-case identifier.`,
+ input.sourceName,
+ input.openingStart,
+ );
+ }
+ const close = findClose(input);
+ const parsed = parseScript(
+ input.sourceName,
+ input.source.slice(input.contentStart, close.start),
+ input.contentStart,
+ );
+ return {
+ nextOffset: close.end,
+ records: [
+ {
+ id: rawId,
+ type: narrativeType,
+ value: { kind: "inline", value: narrativeValue(parsed) },
+ range: { start: input.openingStart, end: close.end },
+ },
+ ],
+ sourceMaps: [canonicalize(narrativeSourceMap(rawId, parsed))],
+ };
+}
diff --git a/packages/script/src/types.ts b/packages/script/src/types.ts
new file mode 100644
index 00000000..3bbd10ff
--- /dev/null
+++ b/packages/script/src/types.ts
@@ -0,0 +1,148 @@
+import type { CanonicalValue, SourceRange, StoredValue, TypeRef } from "@svml/protocol";
+
+export type Affinity = "left" | "right";
+
+export type MarkerBoundary = {
+ readonly tokenIndex: number;
+ readonly structuralPosition: number;
+ readonly segmentId?: string;
+};
+
+export type ParsedTextAtom = {
+ readonly kind: "text";
+ readonly speech: string;
+ readonly caption: string;
+ readonly tokenStart: number;
+ readonly tokenEndExclusive: number;
+ readonly range: SourceRange;
+};
+
+export type ParsedRoleAtom = {
+ readonly kind: "role";
+ readonly label: string;
+ readonly range: SourceRange;
+};
+
+export type ParsedAtom = ParsedTextAtom | ParsedRoleAtom;
+
+export type ParsedSegment = {
+ readonly id: string;
+ readonly index: number;
+ readonly startAnchorId: string;
+ readonly endAnchorId: string;
+ readonly tokenStart: number;
+ readonly tokenEndExclusive: number;
+ readonly atoms: readonly ParsedAtom[];
+ readonly range: SourceRange;
+ readonly selfClosing: boolean;
+};
+
+export type ParsedToken = {
+ readonly id: string;
+ readonly index: number;
+ readonly segmentId: string;
+ readonly segmentTokenIndex: number;
+ readonly startAnchorId: string;
+ readonly endAnchorId: string;
+ readonly text: string;
+ readonly normalized: string;
+ readonly range: SourceRange;
+};
+
+export type ParsedTurn = {
+ readonly id: string;
+ readonly segmentId: string;
+ readonly role?: string;
+ readonly tokenStart: number;
+ readonly tokenEndExclusive: number;
+ readonly range: SourceRange;
+};
+
+export type ParsedSelectionOccurrence = {
+ readonly occurrence: number;
+ readonly open: {
+ readonly affinity: Affinity;
+ readonly boundary: MarkerBoundary;
+ readonly range: SourceRange;
+ };
+ readonly close: {
+ readonly affinity: Affinity;
+ readonly boundary: MarkerBoundary;
+ readonly range: SourceRange;
+ };
+};
+
+export type ParsedSelection = {
+ readonly id: string;
+ readonly occurrences: readonly ParsedSelectionOccurrence[];
+};
+
+export type ParsedMomentOccurrence = {
+ readonly occurrence: number;
+ readonly affinity: Affinity;
+ readonly boundary: MarkerBoundary;
+ readonly range: SourceRange;
+};
+
+export type ParsedMoment = {
+ readonly id: string;
+ readonly occurrences: readonly ParsedMomentOccurrence[];
+};
+
+export type ParsedCaptionAtom = {
+ readonly id: string;
+ readonly display: string;
+ readonly segmentId: string;
+ readonly startToken: number;
+ readonly endTokenExclusive: number;
+ readonly range: SourceRange;
+};
+
+export type SemanticAnchor = {
+ readonly id: string;
+ readonly kind: "segment-start" | "token-start" | "token-end" | "segment-end";
+ readonly segmentId: string;
+ readonly tokenId?: string;
+ readonly segmentTokenIndex?: number;
+};
+
+export type ParsedNarrative = {
+ readonly segments: readonly ParsedSegment[];
+ readonly tokens: readonly ParsedToken[];
+ readonly turns: readonly ParsedTurn[];
+ readonly selections: readonly ParsedSelection[];
+ readonly moments: readonly ParsedMoment[];
+ readonly captionAtoms: readonly ParsedCaptionAtom[];
+ readonly semanticIndex: {
+ readonly contract: "svml.semantic-index@0";
+ readonly anchors: readonly SemanticAnchor[];
+ readonly digest: string;
+ };
+ readonly projections: {
+ readonly dialogue: string;
+ readonly speech: string;
+ readonly caption: string;
+ };
+};
+
+export type ScriptSurfaceRecordDraft = {
+ readonly id: string;
+ readonly type: TypeRef;
+ readonly value: StoredValue;
+ readonly range: SourceRange;
+};
+
+export type ScriptSurfaceInput = {
+ readonly sourceName: string;
+ readonly source: string;
+ readonly tag: string;
+ readonly openingStart: number;
+ readonly contentStart: number;
+ readonly attributes: Readonly<Record<string, string | CanonicalValue>>;
+};
+
+export type ScriptSurfaceOutput = {
+ readonly nextOffset: number;
+ readonly records: readonly ScriptSurfaceRecordDraft[];
+ readonly sourceMaps: readonly CanonicalValue[];
+};
diff --git a/packages/script/test/script.test.ts b/packages/script/test/script.test.ts
new file mode 100644
index 00000000..b5151f68
--- /dev/null
+++ b/packages/script/test/script.test.ts
@@ -0,0 +1,101 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { digestOf } from "@svml/core";
+import {
+ ScriptSyntaxError,
+ formatScript,
+ narrativeValue,
+ parseScript,
+} from "@svml/script";
+
+test("named blocks are Segments and Role Cues do not depend on line breaks", () => {
+ const compact = parseScript(
+ "compact.svml",
+ "<opening><ALICE>Hello there.<BOB>Good morning.</opening><pause/>",
+ );
+ const multiline = parseScript(
+ "multiline.svml",
+ `
+ <opening>
+ <ALICE>Hello there.
+ <BOB>Good morning.
+ </opening>
+
+ <pause/>
+ `,
+ );
+
+ assert.equal(digestOf(narrativeValue(compact)), digestOf(narrativeValue(multiline)));
+ assert.deepEqual(compact.segments.map((segment) => segment.id), ["opening", "pause"]);
+ assert.deepEqual(compact.turns.map((turn) => turn.role), ["ALICE", "BOB"]);
+ assert.equal(compact.projections.dialogue, "ALICE: Hello there.\nBOB: Good morning.");
+});
+
+test("Script produces exactly 2M + 2N independent semantic anchors", () => {
+ const parsed = parseScript("anchors.svml", "<one>One two.</one><silence/><two>Three.</two>");
+ assert.equal(
+ parsed.semanticIndex.anchors.length,
+ 2 * parsed.tokens.length + 2 * parsed.segments.length,
+ );
+ assert.equal(new Set(parsed.semanticIndex.anchors.map((anchor) => anchor.id)).size, parsed.semanticIndex.anchors.length);
+ assert.deepEqual(
+ parsed.segments.map((segment) => [segment.startAnchorId, segment.endAnchorId]),
+ [
+ ["segment:one:start", "segment:one:end"],
+ ["segment:silence:start", "segment:silence:end"],
+ ["segment:two:start", "segment:two:end"],
+ ],
+ );
+});
+
+test("selections, moments and Dual Text preserve separate semantic projections", () => {
+ const parsed = parseScript(
+ "rich.svml",
+ `@whole
+ <answer>
+ <BOB>I @beat!really <laughed | laughed my ass off> there.</answer>
+ @/whole~`,
+ );
+
+ assert.equal(parsed.projections.speech, "I really laughed my ass off there.");
+ assert.equal(parsed.projections.caption, "I really laughed there.");
+ assert.equal(parsed.projections.dialogue, "BOB: I really laughed my ass off there.");
+ assert.deepEqual(parsed.selections.map((selection) => selection.id), ["whole"]);
+ assert.deepEqual(parsed.moments.map((moment) => moment.id), ["beat"]);
+ assert.equal(parsed.captionAtoms[0]?.display, "laughed");
+});
+
+test("mismatched named Segment closes are rejected", () => {
+ assert.throws(
+ () => parseScript("bad.svml", "<opening>Hello.</ending>"),
+ (error: unknown) => error instanceof ScriptSyntaxError && error.code === "SCRIPT_SEGMENT_MISMATCH",
+ );
+ assert.throws(
+ () => parseScript("attribute.svml", "<opening id=\"old\">Hello.</opening>"),
+ (error: unknown) => error instanceof ScriptSyntaxError && error.code === "SCRIPT_SEGMENT_OPEN",
+ );
+});
+
+test("zero-width temporal markers may touch a token edge but cannot split a token", () => {
+ const parsed = parseScript("edge.svml", "<line>@beat!really good</line>");
+ assert.deepEqual(parsed.tokens.map((token) => token.text), ["really", "good"]);
+ assert.deepEqual(parsed.moments.map((moment) => moment.id), ["beat"]);
+
+ assert.throws(
+ () => parseScript("split.svml", "<line>re@beat!ally good</line>"),
+ (error: unknown) =>
+ error instanceof ScriptSyntaxError && error.code === "SCRIPT_MARKER_TOKEN_BOUNDARY",
+ );
+});
+
+test("the formatter is semantic-preserving and idempotent", () => {
+ const input = "<opening><ALICE>Hello there.<BOB>Good morning.</opening><pause/>";
+ const once = formatScript("format.svml", input);
+ const twice = formatScript("format.svml", once);
+ assert.equal(twice, once);
+ assert.equal(
+ digestOf(narrativeValue(parseScript("before.svml", input))),
+ digestOf(narrativeValue(parseScript("after.svml", once))),
+ );
+});
diff --git a/packages/text/README.md b/packages/text/README.md
new file mode 100644
index 00000000..c8a452c8
--- /dev/null
+++ b/packages/text/README.md
@@ -0,0 +1,30 @@
+# `@svml/text`
+
+Official XML-like Author Frontend for `.svml` source units. It owns only the `<svml>` envelope,
+the leading Import Prologue, namespace binding, generic structured elements and dispatch to
+statically declared module Surfaces.
+
+The package has no built-in Script, media or video vocabulary. `<script>` is accepted only when an
+imported module declares that tag and the Host explicitly registers the locked Surface
+implementation. The current registry is for trusted official/in-process use; it is not a security
+sandbox for third-party parser code.
+
+```svml
+<svml>
+ <import from="@svml/[email protected]"/>
+
+ <script id="story">
+ <opening><ALICE>Hello.</opening>
+ </script>
+</svml>
+```
+
+Decoding has two passes. `discoverText` reads only the root and complete leading Import Prologue.
+After Driver supplies one immutable resolved closure, `decodeText` freezes the visible Surface
+scope and parses the body. A raw Surface receives the source cursor immediately after its opening
+tag and must return the cursor after its own close; a structured Surface receives Text's generic
+element tree. Text seals each returned draft as an authored typed Record with separate source,
+frontend-closure and semantic identities.
+
+Source imports using another codec are discovered by the syntax but intentionally rejected by the
+current decoder until Driver owns recursive SourceUnit resolution.
diff --git a/packages/text/package.json b/packages/text/package.json
new file mode 100644
index 00000000..5df0f6ae
--- /dev/null
+++ b/packages/text/package.json
@@ -0,0 +1,16 @@
+{
+ "name": "@svml/text",
+ "version": "0.0.0-dev",
+ "private": true,
+ "type": "module",
+ "exports": {
+ ".": "./src/index.ts"
+ },
+ "dependencies": {
+ "@svml/core": "workspace:*",
+ "@svml/protocol": "workspace:*"
+ },
+ "devDependencies": {
+ "@svml/script": "workspace:*"
+ }
+}
diff --git a/packages/text/src/error.ts b/packages/text/src/error.ts
new file mode 100644
index 00000000..2110131a
--- /dev/null
+++ b/packages/text/src/error.ts
@@ -0,0 +1,13 @@
+export class TextFrontendError extends Error {
+ readonly code: string;
+ readonly sourceName: string;
+ readonly offset: number | undefined;
+
+ constructor(code: string, message: string, sourceName: string, offset?: number) {
+ super(`${code}: ${message}${offset === undefined ? "" : ` (${sourceName}:${offset})`}`);
+ this.name = "TextFrontendError";
+ this.code = code;
+ this.sourceName = sourceName;
+ this.offset = offset;
+ }
+}
diff --git a/packages/text/src/frontend.ts b/packages/text/src/frontend.ts
new file mode 100644
index 00000000..fdf6723c
--- /dev/null
+++ b/packages/text/src/frontend.ts
@@ -0,0 +1,232 @@
+import {
+ digestOf,
+ sealRecord,
+ sealTypedModule,
+ verifyClosure,
+ verifyRecord,
+} from "@svml/core";
+import type { CanonicalValue, ModuleRef, ResolvedModule, SurfaceDeclaration } from "@svml/protocol";
+
+import { TextFrontendError } from "./error.js";
+import {
+ closeDocument,
+ discoverText,
+ parseOpeningTag,
+ parseStructuredElement,
+ skipTextTrivia,
+} from "./syntax.js";
+import type {
+ RawSurfaceHandler,
+ SourceUnit,
+ StructuredSurfaceHandler,
+ SurfaceRecordDraft,
+ TextDecodeContext,
+ TextDecodeResult,
+ TextImportRequest,
+} from "./types.js";
+
+export const textFrontendRef = { module: "@svml/text", version: "0.0.0-dev", name: "text" } as const;
+export const textFrontendImplementationDigest = digestOf("@svml/text/frontend@0");
+
+type BoundSurface = {
+ readonly tag: string;
+ readonly module: ResolvedModule;
+ readonly declaration: SurfaceDeclaration;
+};
+
+function moduleKey(ref: ModuleRef): string {
+ return `${ref.name}@${ref.version}`;
+}
+
+function sameModule(left: ModuleRef, right: ModuleRef): boolean {
+ return left.name === right.name && left.version === right.version;
+}
+
+function fail(source: SourceUnit, code: string, message: string, offset?: number): never {
+ throw new TextFrontendError(code, message, source.name, offset);
+}
+
+function moduleForImport(
+ source: SourceUnit,
+ request: TextImportRequest,
+ context: TextDecodeContext,
+): ResolvedModule {
+ const ref = context.resolveModule(request);
+ const module = context.closure.modules.find((item) => sameModule(item.ref, ref));
+ if (!module) {
+ fail(source, "TEXT_IMPORT_CLOSURE", `${moduleKey(ref)} is not present in the resolved closure.`, request.range.start);
+ }
+ return module;
+}
+
+function surfaceScope(
+ source: SourceUnit,
+ imports: readonly TextImportRequest[],
+ context: TextDecodeContext,
+): Map<string, BoundSurface> {
+ const scope = new Map<string, BoundSurface>();
+ const aliases = new Set<string>();
+ for (const request of imports) {
+ if (request.alias !== undefined) {
+ if (aliases.has(request.alias)) fail(source, "TEXT_ALIAS_DUPLICATE", `Duplicate import alias "${request.alias}".`, request.range.start);
+ aliases.add(request.alias);
+ }
+ if (request.using !== undefined) continue;
+ const module = moduleForImport(source, request, context);
+ for (const declaration of module.manifest.surfaces) {
+ const tag = request.alias === undefined ? declaration.tag : `${request.alias}:${declaration.tag}`;
+ if (scope.has(tag)) fail(source, "TEXT_SURFACE_COLLISION", `Surface tag <${tag}> is imported more than once.`, request.range.start);
+ scope.set(tag, { tag, module, declaration });
+ }
+ }
+ return scope;
+}
+
+export function decodeText(source: SourceUnit, context: TextDecodeContext): TextDecodeResult {
+ verifyClosure(context.closure);
+ const discovery = discoverText(source);
+ if (discovery.imports.some((request) => request.using !== undefined)) {
+ const request = discovery.imports.find((item) => item.using !== undefined)!;
+ fail(
+ source,
+ "TEXT_SOURCE_IMPORT_UNRESOLVED",
+ `Source import "${request.from}" must be decoded by Driver before this SourceUnit.`,
+ request.range.start,
+ );
+ }
+ const scope = surfaceScope(source, discovery.imports, context);
+ const sourceDigest = digestOf(source.text);
+ const frontendClosureDigest = digestOf({
+ frontend: textFrontendImplementationDigest,
+ surfaces: [...scope.values()]
+ .sort((left, right) => left.tag.localeCompare(right.tag))
+ .map((surface) => ({
+ tag: surface.tag,
+ module: surface.module.ref,
+ moduleDigest: surface.module.digest,
+ name: surface.declaration.name,
+ implementationDigest: surface.declaration.implementation.digest,
+ })),
+ });
+ const records = [];
+ const sourceMaps: CanonicalValue[] = [];
+ const recordIds = new Set<string>();
+ let cursor = discovery.bodyStart;
+ let closed = false;
+ while (cursor < source.text.length) {
+ cursor = skipTextTrivia(source, cursor);
+ if (source.text.startsWith("</", cursor)) {
+ cursor = closeDocument(source, cursor);
+ closed = true;
+ break;
+ }
+ if (source.text[cursor] !== "<") {
+ fail(source, "TEXT_BODY_TEXT", "Natural-language text is not allowed directly under <svml>.", cursor);
+ }
+ const opening = parseOpeningTag(source, cursor);
+ if (opening.name === "import") {
+ fail(source, "TEXT_IMPORT_AFTER_BODY", "All imports must appear in the leading Import Prologue.", cursor);
+ }
+ const bound = scope.get(opening.name);
+ if (!bound) fail(source, "TEXT_UNKNOWN_SURFACE", `No imported module declares <${opening.name}>.`, cursor);
+ const registered = context.registry.resolve(bound.module.ref, bound.declaration.name);
+ if (!registered) {
+ fail(source, "TEXT_SURFACE_UNREGISTERED", `Surface ${moduleKey(bound.module.ref)}#${bound.declaration.name} is not registered.`, cursor);
+ }
+ if (
+ registered.mode !== bound.declaration.mode
+ || registered.implementationDigest !== bound.declaration.implementation.digest
+ ) {
+ fail(source, "TEXT_SURFACE_MISMATCH", `Registered Surface ${bound.declaration.name} does not match the locked Manifest.`, cursor);
+ }
+ let drafts: readonly SurfaceRecordDraft[];
+ let maps: readonly CanonicalValue[] | undefined;
+ if (bound.declaration.mode === "raw") {
+ if (opening.selfClosing) fail(source, "TEXT_RAW_SELF_CLOSING", `Raw Surface <${opening.name}> cannot be self-closing.`, cursor);
+ const output = (registered.handler as RawSurfaceHandler)({
+ sourceName: source.name,
+ source: source.text,
+ tag: opening.name,
+ openingStart: opening.start,
+ contentStart: opening.end,
+ attributes: opening.attributes,
+ });
+ if (!Number.isInteger(output.nextOffset) || output.nextOffset <= opening.end || output.nextOffset > source.text.length) {
+ fail(source, "TEXT_SURFACE_CURSOR", `Raw Surface <${opening.name}> returned an invalid cursor.`, cursor);
+ }
+ cursor = output.nextOffset;
+ drafts = output.records;
+ maps = output.sourceMaps;
+ } else {
+ const parsed = parseStructuredElement(source, cursor);
+ const output = (registered.handler as StructuredSurfaceHandler)({
+ sourceName: source.name,
+ element: parsed.element,
+ });
+ cursor = parsed.nextOffset;
+ drafts = output.records;
+ maps = output.sourceMaps;
+ }
+ for (const draft of drafts) {
+ if (
+ !Number.isInteger(draft.range.start)
+ || !Number.isInteger(draft.range.end)
+ || draft.range.start < opening.start
+ || draft.range.end < draft.range.start
+ || draft.range.end > cursor
+ ) {
+ fail(
+ source,
+ "TEXT_SURFACE_RANGE",
+ `Surface ${moduleKey(bound.module.ref)}#${bound.declaration.name} returned an invalid source range.`,
+ opening.start,
+ );
+ }
+ if (recordIds.has(draft.id)) fail(source, "TEXT_RECORD_DUPLICATE", `Duplicate authored record "${draft.id}".`, draft.range.start);
+ if (!bound.declaration.outputs.some((output) => sameModule(output.module, draft.type.module) && output.name === draft.type.name)) {
+ fail(
+ source,
+ "TEXT_SURFACE_OUTPUT",
+ `Surface ${moduleKey(bound.module.ref)}#${bound.declaration.name} did not declare output type ${moduleKey(draft.type.module)}#${draft.type.name}.`,
+ draft.range.start,
+ );
+ }
+ recordIds.add(draft.id);
+ const record = sealRecord({
+ id: draft.id,
+ type: draft.type,
+ value: draft.value,
+ conformance: "exact",
+ origin: {
+ kind: "authored",
+ sourceDigest,
+ frontendClosureDigest,
+ sourceName: source.name,
+ range: draft.range,
+ },
+ });
+ verifyRecord(context.closure, record);
+ records.push(record);
+ }
+ sourceMaps.push(...(maps ?? []));
+ }
+ if (!closed) {
+ fail(source, "TEXT_ROOT_UNCLOSED", "Document is missing </svml>.", source.text.length);
+ }
+ return {
+ module: sealTypedModule({
+ id: `source:${source.name}`,
+ closureDigest: context.closure.digest,
+ records,
+ }),
+ imports: discovery.imports,
+ frontendClosureDigest,
+ sourceMaps,
+ };
+}
+
+export const textFrontend = {
+ id: textFrontendRef,
+ discover: discoverText,
+ decode: decodeText,
+} as const;
diff --git a/packages/text/src/index.ts b/packages/text/src/index.ts
new file mode 100644
index 00000000..5ddabc9c
--- /dev/null
+++ b/packages/text/src/index.ts
@@ -0,0 +1,16 @@
+export { TextFrontendError } from "./error.js";
+export {
+ decodeText,
+ textFrontend,
+ textFrontendImplementationDigest,
+ textFrontendRef,
+} from "./frontend.js";
+export { TextSurfaceRegistry } from "./registry.js";
+export {
+ closeDocument,
+ discoverText,
+ parseOpeningTag,
+ parseStructuredElement,
+ skipTextTrivia,
+} from "./syntax.js";
+export type * from "./types.js";
diff --git a/packages/text/src/registry.ts b/packages/text/src/registry.ts
new file mode 100644
index 00000000..5fdf9008
--- /dev/null
+++ b/packages/text/src/registry.ts
@@ -0,0 +1,46 @@
+import { isDigest } from "@svml/core";
+import type { Digest, ModuleRef } from "@svml/protocol";
+
+import type {
+ RawSurfaceHandler,
+ RegisteredSurface,
+ StructuredSurfaceHandler,
+ TextSurfaceRegistryLike,
+} from "./types.js";
+
+function key(module: ModuleRef, surface: string): string {
+ return `${module.name}@${module.version}#${surface}`;
+}
+
+export class TextSurfaceRegistry implements TextSurfaceRegistryLike {
+ readonly #values = new Map<string, RegisteredSurface>();
+
+ registerRaw(
+ module: ModuleRef,
+ surface: string,
+ implementationDigest: Digest,
+ handler: RawSurfaceHandler,
+ ): void {
+ this.register({ module, surface, implementationDigest, mode: "raw", handler });
+ }
+
+ registerStructured(
+ module: ModuleRef,
+ surface: string,
+ implementationDigest: Digest,
+ handler: StructuredSurfaceHandler,
+ ): void {
+ this.register({ module, surface, implementationDigest, mode: "structured", handler });
+ }
+
+ private register(value: RegisteredSurface): void {
+ if (!isDigest(value.implementationDigest)) throw new Error("Surface implementation digest is invalid");
+ const identity = key(value.module, value.surface);
+ if (this.#values.has(identity)) throw new Error(`Surface ${identity} is already registered`);
+ this.#values.set(identity, value);
+ }
+
+ resolve(module: ModuleRef, surface: string): RegisteredSurface | undefined {
+ return this.#values.get(key(module, surface));
+ }
+}
diff --git a/packages/text/src/syntax.ts b/packages/text/src/syntax.ts
new file mode 100644
index 00000000..767fb74a
--- /dev/null
+++ b/packages/text/src/syntax.ts
@@ -0,0 +1,257 @@
+import type { SourceRange } from "@svml/protocol";
+
+import { TextFrontendError } from "./error.js";
+import type {
+ SourceUnit,
+ StructuredElement,
+ StructuredNode,
+ TextAttributeValue,
+ TextDiscovery,
+ TextImportRequest,
+} from "./types.js";
+
+const NAME = /^[A-Za-z_][A-Za-z0-9_.:-]*/u;
+const ALIAS = /^[a-z][a-z0-9_-]{0,63}$/u;
+const REFERENCE = /^[A-Za-z_][A-Za-z0-9_.:-]*(?:\.[A-Za-z_][A-Za-z0-9_.:-]*)*$/u;
+
+type OpeningTag = {
+ readonly name: string;
+ readonly attributes: Readonly<Record<string, TextAttributeValue>>;
+ readonly start: number;
+ readonly end: number;
+ readonly selfClosing: boolean;
+};
+
+function fail(source: SourceUnit, code: string, message: string, offset?: number): never {
+ throw new TextFrontendError(code, message, source.name, offset);
+}
+
+function decodeEntities(value: string, source: SourceUnit, offset: number): string {
+ return value.replace(/&(?:lt|gt|amp|quot|apos);/gu, (entity) => {
+ switch (entity) {
+ case "&lt;": return "<";
+ case "&gt;": return ">";
+ case "&amp;": return "&";
+ case "&quot;": return "\"";
+ case "&apos;": return "'";
+ default: fail(source, "TEXT_ENTITY", `Unsupported entity ${entity}.`, offset);
+ }
+ });
+}
+
+function skipSpace(text: string, start: number): number {
+ let cursor = start;
+ while (/\s/u.test(text[cursor] ?? "")) cursor += 1;
+ return cursor;
+}
+
+function skipTrivia(source: SourceUnit, start: number): number {
+ let cursor = start;
+ while (cursor < source.text.length) {
+ const next = skipSpace(source.text, cursor);
+ if (source.text.startsWith("<!--", next)) {
+ const end = source.text.indexOf("-->", next + 4);
+ if (end < 0) fail(source, "TEXT_COMMENT", "Unclosed comment.", next);
+ cursor = end + 3;
+ continue;
+ }
+ return next;
+ }
+ return cursor;
+}
+
+function parseName(source: SourceUnit, start: number): { readonly value: string; readonly end: number } {
+ const match = NAME.exec(source.text.slice(start));
+ if (!match) fail(source, "TEXT_NAME", "Expected a name.", start);
+ return { value: match[0], end: start + match[0].length };
+}
+
+function parseAttributeValue(
+ source: SourceUnit,
+ start: number,
+): { readonly value: TextAttributeValue; readonly end: number } {
+ const quote = source.text[start];
+ if (quote === "\"" || quote === "'") {
+ const close = source.text.indexOf(quote, start + 1);
+ if (close < 0) fail(source, "TEXT_ATTRIBUTE", "Unclosed quoted attribute.", start);
+ return {
+ value: decodeEntities(source.text.slice(start + 1, close), source, start),
+ end: close + 1,
+ };
+ }
+ if (quote === "{") {
+ const close = source.text.indexOf("}", start + 1);
+ if (close < 0) fail(source, "TEXT_REFERENCE", "Unclosed reference attribute.", start);
+ const path = source.text.slice(start + 1, close).trim();
+ if (!REFERENCE.test(path)) fail(source, "TEXT_REFERENCE", `Invalid reference "${path}".`, start);
+ return { value: { kind: "reference", path }, end: close + 1 };
+ }
+ fail(source, "TEXT_ATTRIBUTE", "Attribute values must be quoted strings or whole-value references.", start);
+}
+
+export function parseOpeningTag(source: SourceUnit, start: number): OpeningTag {
+ if (source.text[start] !== "<" || source.text[start + 1] === "/") {
+ fail(source, "TEXT_OPEN", "Expected an opening tag.", start);
+ }
+ const parsedName = parseName(source, start + 1);
+ let cursor = parsedName.end;
+ const attributes: Record<string, TextAttributeValue> = {};
+ while (cursor < source.text.length) {
+ cursor = skipSpace(source.text, cursor);
+ if (source.text.startsWith("/>", cursor)) {
+ return { name: parsedName.value, attributes, start, end: cursor + 2, selfClosing: true };
+ }
+ if (source.text[cursor] === ">") {
+ return { name: parsedName.value, attributes, start, end: cursor + 1, selfClosing: false };
+ }
+ const attribute = parseName(source, cursor);
+ if (Object.hasOwn(attributes, attribute.value)) {
+ fail(source, "TEXT_ATTRIBUTE_DUPLICATE", `Duplicate attribute "${attribute.value}".`, cursor);
+ }
+ cursor = skipSpace(source.text, attribute.end);
+ if (source.text[cursor] !== "=") fail(source, "TEXT_ATTRIBUTE", `Attribute "${attribute.value}" requires =.`, cursor);
+ cursor = skipSpace(source.text, cursor + 1);
+ const value = parseAttributeValue(source, cursor);
+ Object.defineProperty(attributes, attribute.value, {
+ value: value.value,
+ enumerable: true,
+ configurable: true,
+ writable: true,
+ });
+ cursor = value.end;
+ }
+ fail(source, "TEXT_OPEN", `Opening tag <${parsedName.value}> is not closed.`, start);
+}
+
+function parseClose(source: SourceUnit, start: number): { readonly name: string; readonly end: number } {
+ if (!source.text.startsWith("</", start)) fail(source, "TEXT_CLOSE", "Expected a closing tag.", start);
+ const name = parseName(source, start + 2);
+ const cursor = skipSpace(source.text, name.end);
+ if (source.text[cursor] !== ">") fail(source, "TEXT_CLOSE", "Malformed closing tag.", start);
+ return { name: name.value, end: cursor + 1 };
+}
+
+export function parseStructuredElement(
+ source: SourceUnit,
+ start: number,
+): { readonly element: StructuredElement; readonly nextOffset: number } {
+ const opening = parseOpeningTag(source, start);
+ if (opening.selfClosing) {
+ return {
+ element: {
+ kind: "element",
+ name: opening.name,
+ attributes: opening.attributes,
+ children: [],
+ range: { start, end: opening.end },
+ },
+ nextOffset: opening.end,
+ };
+ }
+ const children: StructuredNode[] = [];
+ let cursor = opening.end;
+ while (cursor < source.text.length) {
+ if (source.text.startsWith("<!--", cursor)) {
+ const end = source.text.indexOf("-->", cursor + 4);
+ if (end < 0) fail(source, "TEXT_COMMENT", "Unclosed comment.", cursor);
+ cursor = end + 3;
+ continue;
+ }
+ if (source.text.startsWith("</", cursor)) {
+ const close = parseClose(source, cursor);
+ if (close.name !== opening.name) {
+ fail(source, "TEXT_CLOSE_MISMATCH", `<${opening.name}> was closed by </${close.name}>.`, cursor);
+ }
+ return {
+ element: {
+ kind: "element",
+ name: opening.name,
+ attributes: opening.attributes,
+ children,
+ range: { start, end: close.end },
+ },
+ nextOffset: close.end,
+ };
+ }
+ if (source.text[cursor] === "<") {
+ const child = parseStructuredElement(source, cursor);
+ children.push(child.element);
+ cursor = child.nextOffset;
+ continue;
+ }
+ const next = source.text.indexOf("<", cursor);
+ const end = next < 0 ? source.text.length : next;
+ children.push({
+ kind: "text",
+ value: decodeEntities(source.text.slice(cursor, end), source, cursor),
+ range: { start: cursor, end },
+ });
+ cursor = end;
+ }
+ fail(source, "TEXT_ELEMENT_UNCLOSED", `<${opening.name}> is not closed.`, start);
+}
+
+function stringAttribute(
+ source: SourceUnit,
+ opening: OpeningTag,
+ name: string,
+ required = false,
+): string | undefined {
+ const value = opening.attributes[name];
+ if (value === undefined) {
+ if (required) fail(source, "TEXT_IMPORT", `<import> requires ${name}.`, opening.start);
+ return undefined;
+ }
+ if (typeof value !== "string") fail(source, "TEXT_IMPORT", `<import> ${name} must be a string.`, opening.start);
+ return value;
+}
+
+function importRequest(source: SourceUnit, opening: OpeningTag): TextImportRequest {
+ if (opening.name !== "import" || !opening.selfClosing) {
+ fail(source, "TEXT_IMPORT", "Imports must use <import .../>.", opening.start);
+ }
+ const unknown = Object.keys(opening.attributes).filter((name) => !["from", "as", "using"].includes(name));
+ if (unknown.length) fail(source, "TEXT_IMPORT", `Unknown import attribute "${unknown[0]}".`, opening.start);
+ const alias = stringAttribute(source, opening, "as");
+ if (alias !== undefined && !ALIAS.test(alias)) fail(source, "TEXT_IMPORT_ALIAS", `Invalid alias "${alias}".`, opening.start);
+ const using = stringAttribute(source, opening, "using");
+ if (using !== undefined && alias === undefined) {
+ fail(source, "TEXT_IMPORT_USING", "A source import using a codec requires an alias.", opening.start);
+ }
+ return {
+ from: stringAttribute(source, opening, "from", true)!,
+ ...(alias === undefined ? {} : { alias }),
+ ...(using === undefined ? {} : { using }),
+ range: { start: opening.start, end: opening.end },
+ };
+}
+
+export function discoverText(source: SourceUnit): TextDiscovery {
+ let cursor = skipTrivia(source, 0);
+ const root = parseOpeningTag(source, cursor);
+ if (root.name !== "svml" || root.selfClosing) fail(source, "TEXT_ROOT", "Document must open with <svml>.", cursor);
+ if (Object.keys(root.attributes).length) fail(source, "TEXT_ROOT_ATTRIBUTE", "<svml> declares no attributes.", cursor);
+ cursor = root.end;
+ const imports: TextImportRequest[] = [];
+ while (true) {
+ cursor = skipTrivia(source, cursor);
+ if (source.text.startsWith("</", cursor) || source.text[cursor] !== "<") break;
+ const opening = parseOpeningTag(source, cursor);
+ if (opening.name !== "import") break;
+ imports.push(importRequest(source, opening));
+ cursor = imports.at(-1)!.range.end;
+ }
+ return { imports, bodyStart: cursor };
+}
+
+export function closeDocument(source: SourceUnit, start: number): number {
+ const close = parseClose(source, start);
+ if (close.name !== "svml") fail(source, "TEXT_ROOT_CLOSE", `Expected </svml>, received </${close.name}>.`, start);
+ const end = skipTrivia(source, close.end);
+ if (end !== source.text.length) fail(source, "TEXT_TRAILING", "Only trivia may follow </svml>.", end);
+ return end;
+}
+
+export function skipTextTrivia(source: SourceUnit, start: number): number {
+ return skipTrivia(source, start);
+}
diff --git a/packages/text/src/types.ts b/packages/text/src/types.ts
new file mode 100644
index 00000000..5d4c1b71
--- /dev/null
+++ b/packages/text/src/types.ts
@@ -0,0 +1,109 @@
+import type {
+ CanonicalValue,
+ Digest,
+ ModuleRef,
+ ResolvedModuleClosure,
+ SourceRange,
+ StoredValue,
+ SurfaceDeclaration,
+ TypeRef,
+ TypedModule,
+} from "@svml/protocol";
+
+export type SourceUnit = {
+ readonly name: string;
+ readonly text: string;
+};
+
+export type TextImportRequest = {
+ readonly from: string;
+ readonly alias?: string;
+ readonly using?: string;
+ readonly range: SourceRange;
+};
+
+export type TextDiscovery = {
+ readonly imports: readonly TextImportRequest[];
+ readonly bodyStart: number;
+};
+
+export type TextReference = {
+ readonly kind: "reference";
+ readonly path: string;
+};
+
+export type TextAttributeValue = string | TextReference;
+
+export type StructuredText = {
+ readonly kind: "text";
+ readonly value: string;
+ readonly range: SourceRange;
+};
+
+export type StructuredElement = {
+ readonly kind: "element";
+ readonly name: string;
+ readonly attributes: Readonly<Record<string, TextAttributeValue>>;
+ readonly children: readonly StructuredNode[];
+ readonly range: SourceRange;
+};
+
+export type StructuredNode = StructuredText | StructuredElement;
+
+export type SurfaceRecordDraft = {
+ readonly id: string;
+ readonly type: TypeRef;
+ readonly value: StoredValue;
+ readonly range: SourceRange;
+};
+
+export type SurfaceDecodeOutput = {
+ readonly records: readonly SurfaceRecordDraft[];
+ readonly sourceMaps?: readonly CanonicalValue[];
+};
+
+export type RawSurfaceInput = {
+ readonly sourceName: string;
+ readonly source: string;
+ readonly tag: string;
+ readonly openingStart: number;
+ readonly contentStart: number;
+ readonly attributes: Readonly<Record<string, TextAttributeValue>>;
+};
+
+export type RawSurfaceOutput = SurfaceDecodeOutput & {
+ readonly nextOffset: number;
+};
+
+export type StructuredSurfaceInput = {
+ readonly sourceName: string;
+ readonly element: StructuredElement;
+};
+
+export type RawSurfaceHandler = (input: RawSurfaceInput) => RawSurfaceOutput;
+export type StructuredSurfaceHandler = (input: StructuredSurfaceInput) => SurfaceDecodeOutput;
+
+export type RegisteredSurface = {
+ readonly module: ModuleRef;
+ readonly surface: string;
+ readonly implementationDigest: Digest;
+ readonly mode: SurfaceDeclaration["mode"];
+ readonly handler: RawSurfaceHandler | StructuredSurfaceHandler;
+};
+
+export type TextDecodeContext = {
+ readonly closure: ResolvedModuleClosure;
+ readonly registry: TextSurfaceRegistryLike;
+ readonly resolveModule: (request: TextImportRequest) => ModuleRef;
+};
+
+export type TextDecodeResult = {
+ readonly module: TypedModule;
+ readonly imports: readonly TextImportRequest[];
+ readonly frontendClosureDigest: Digest;
+ readonly sourceMaps: readonly CanonicalValue[];
+};
+
+export interface TextSurfaceRegistryLike {
+ resolve(module: ModuleRef, surface: string): RegisteredSurface | undefined;
+}
diff --git a/packages/text/test/text.test.ts b/packages/text/test/text.test.ts
new file mode 100644
index 00000000..abf4526e
--- /dev/null
+++ b/packages/text/test/text.test.ts
@@ -0,0 +1,235 @@
+import assert from "node:assert/strict";
+import test from "node:test";
+
+import { createResolvedClosure, digestOf, isDigest, link } from "@svml/core";
+import type { ModuleManifest } from "@svml/protocol";
+import {
+ decodeScriptSurface,
+ scriptManifest,
+ scriptModuleRef,
+ scriptSurfaceImplementationDigest,
+} from "@svml/script";
+import {
+ TextFrontendError,
+ TextSurfaceRegistry,
+ decodeText,
+ parseStructuredElement,
+} from "@svml/text";
+
+function scriptContext() {
+ const closure = createResolvedClosure([scriptManifest]);
+ const registry = new TextSurfaceRegistry();
+ registry.registerRaw(
+ scriptModuleRef,
+ "script",
+ scriptSurfaceImplementationDigest,
+ decodeScriptSurface,
+ );
+ return {
+ closure,
+ registry,
+ resolveModule: () => scriptModuleRef,
+ } as const;
+}
+
+test("Text learns <script> only from an imported Script Manifest", () => {
+ const result = decodeText(
+ {
+ name: "talk.svml",
+ text: `<svml>
+ <import from="@svml/[email protected]"/>
+
+ <script id="story">
+ <opening>
+ <ALICE>Hello.
+ <BOB>Hi.
+ </opening>
+ </script>
+ </svml>`,
+ },
+ scriptContext(),
+ );
+
+ assert.equal(result.module.records.length, 1);
+ assert.equal(result.module.records[0]?.id, "story");
+ assert.equal(result.module.records[0]?.type.name, "Narrative");
+ assert.equal(result.sourceMaps.length, 1);
+ assert.doesNotThrow(() => link(scriptContext().closure, [result.module]));
+});
+
+test("the same Script meaning has the same authored Record digest across reflow", () => {
+ const compact = decodeText(
+ {
+ name: "compact.svml",
+ text: `<svml><import from="@svml/script"/><script id="story"><opening><ALICE>Hello.<BOB>Hi.</opening></script></svml>`,
+ },
+ scriptContext(),
+ );
+ const multiline = decodeText(
+ {
+ name: "multiline.svml",
+ text: `<svml>
+ <import from="@svml/script"/>
+ <script id="story">
+ <opening>
+ <ALICE>Hello.
+ <BOB>Hi.
+ </opening>
+ </script>
+ </svml>`,
+ },
+ scriptContext(),
+ );
+
+ assert.equal(compact.module.records[0]?.digest, multiline.module.records[0]?.digest);
+ assert.notEqual(
+ compact.module.records[0]?.origin.kind === "authored" ? compact.module.records[0].origin.sourceDigest : undefined,
+ multiline.module.records[0]?.origin.kind === "authored" ? multiline.module.records[0].origin.sourceDigest : undefined,
+ );
+});
+
+test("without the import, Text has no hard-coded knowledge of Script", () => {
+ assert.throws(
+ () => decodeText(
+ { name: "unknown.svml", text: "<svml><script><opening>Hello.</opening></script></svml>" },
+ scriptContext(),
+ ),
+ (error: unknown) => error instanceof TextFrontendError && error.code === "TEXT_UNKNOWN_SURFACE",
+ );
+});
+
+test("a raw Surface cannot consume the Text document close", () => {
+ const closure = createResolvedClosure([scriptManifest]);
+ const registry = new TextSurfaceRegistry();
+ registry.registerRaw(
+ scriptModuleRef,
+ "script",
+ scriptSurfaceImplementationDigest,
+ (input) => ({ nextOffset: input.source.length, records: [] }),
+ );
+ assert.throws(
+ () => decodeText(
+ {
+ name: "swallowed.svml",
+ text: `<svml><import from="@svml/script"/><script><opening>Hello.</opening></script></svml>`,
+ },
+ { closure, registry, resolveModule: () => scriptModuleRef },
+ ),
+ (error: unknown) => error instanceof TextFrontendError && error.code === "TEXT_ROOT_UNCLOSED",
+ );
+});
+
+test("imports are frozen before body decoding", () => {
+ assert.throws(
+ () => decodeText(
+ {
+ name: "late.svml",
+ text: `<svml>
+ <import from="@svml/script"/>
+ <script><opening>Hello.</opening></script>
+ <import from="@svml/script"/>
+ </svml>`,
+ },
+ scriptContext(),
+ ),
+ (error: unknown) => error instanceof TextFrontendError && error.code === "TEXT_IMPORT_AFTER_BODY",
+ );
+});
+
+test("a body tag whose name merely starts with import is not treated as an import", () => {
+ assert.throws(
+ () => decodeText(
+ { name: "important.svml", text: "<svml><important/></svml>" },
+ scriptContext(),
+ ),
+ (error: unknown) => error instanceof TextFrontendError && error.code === "TEXT_UNKNOWN_SURFACE",
+ );
+});
+
+test("Text also exposes a generic structured Surface tree", () => {
+ const source = {
+ name: "structured.svml",
+ text: `<card title="Hello" target={story}><line>World</line></card>`,
+ };
+ const parsed = parseStructuredElement(source, 0);
+ assert.equal(parsed.nextOffset, source.text.length);
+ assert.equal(parsed.element.name, "card");
+ assert.deepEqual(parsed.element.attributes.target, { kind: "reference", path: "story" });
+ assert.equal(parsed.element.children[0]?.kind, "element");
+});
+
+test("a module can use Text's generic structured parser without adding another parser", () => {
+ const module = { name: "example.card", version: "1" } as const;
+ const type = { module, name: "Card" } as const;
+ const implementationDigest = digestOf("example.card/surface@1");
+ const manifest: ModuleManifest = {
+ format: "svml.module@0",
+ name: module.name,
+ version: module.version,
+ dependencies: [],
+ types: [
+ {
+ name: type.name,
+ schema: {
+ kind: "object",
+ fields: { tag: { schema: { kind: "literal", value: "card" } } },
+ },
+ },
+ ],
+ surfaces: [
+ {
+ name: "card",
+ tag: "card",
+ mode: "structured",
+ outputs: [type],
+ implementation: {
+ kind: "trusted-frontend-surface",
+ locator: "example.card/surface",
+ digest: implementationDigest,
+ },
+ },
+ ],
+ producers: [],
+ };
+ const closure = createResolvedClosure([manifest]);
+ const registry = new TextSurfaceRegistry();
+ registry.registerStructured(module, "card", implementationDigest, ({ element }) => ({
+ records: [
+ {
+ id: "card",
+ type,
+ value: { kind: "inline", value: { tag: element.name } },
+ range: element.range,
+ },
+ ],
+ }));
+ const result = decodeText(
+ {
+ name: "card.svml",
+ text: `<svml><import from="example.card@1"/><card title="Hello"><line>World</line></card></svml>`,
+ },
+ { closure, registry, resolveModule: () => module },
+ );
+
+ assert.equal(result.module.records[0]?.id, "card");
+ assert.equal(isDigest(result.module.records[0]?.digest ?? ""), true);
+
+ const overreachingRegistry = new TextSurfaceRegistry();
+ overreachingRegistry.registerStructured(module, "card", implementationDigest, ({ element }) => ({
+ records: [
+ {
+ id: "other",
+ type: { module, name: "Other" },
+ value: { kind: "inline", value: { tag: element.name } },
+ range: element.range,
+ },
+ ],
+ }));
+ assert.throws(
+ () => decodeText(
+ { name: "overreach.svml", text: `<svml><import from="example.card@1"/><card/></svml>` },
+ { closure, registry: overreachingRegistry, resolveModule: () => module },
+ ),
+ (error: unknown) => error instanceof TextFrontendError && error.code === "TEXT_SURFACE_OUTPUT",
+ );
+});
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index e63a4cce..8cd1e969 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -39,6 +39,28 @@ importers:
packages/protocol: {}
+ packages/script:
+ dependencies:
+ '@svml/core':
+ specifier: workspace:*
+ version: link:../core
+ '@svml/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+
+ packages/text:
+ dependencies:
+ '@svml/core':
+ specifier: workspace:*
+ version: link:../core
+ '@svml/protocol':
+ specifier: workspace:*
+ version: link:../protocol
+ devDependencies:
+ '@svml/script':
+ specifier: workspace:*
+ version: link:../script
+
packages:
'@emnapi/[email protected]':
diff --git a/spec/core-kernel-v0.md b/spec/core-kernel-v0.md
index aaae0682..9769ff90 100644
--- a/spec/core-kernel-v0.md
+++ b/spec/core-kernel-v0.md
@@ -29,5 +29,7 @@ The Node Driver owns manifest I/O, registered implementation execution, requirem
artifact bytes, journaling and pause/resume orchestration. Core emits commands; it never executes
them.
-Frontend selection, `.svml`, `.svs`, Import Prologue and Surface Parser behavior are explicitly
-outside Core v0 and Driver v0.
+Frontend selection, `.svml`, `.svs`, Import Prologue and Surface Parser execution are explicitly
+outside Core v0 and Driver v0. The data-only Protocol may declare a Surface in a Module Manifest,
+and Core verifies its identity/mode/digest as ordinary closure data; this does not give Core any
+source syntax or parser dispatch behavior.
diff --git a/spec/script-surface-v1.md b/spec/script-surface-v1.md
index c150c64c..b6f03047 100644
--- a/spec/script-surface-v1.md
+++ b/spec/script-surface-v1.md
@@ -46,17 +46,17 @@ Script 编译必须先生成保留源码映射的 Narrative IR,再从同一份
<script>
@whole
- <segment id="hook">
+ <hook>
<A> I just @laugh @pop! <lmao | laughed my @punch ass out @/punch> @/laugh.
- </segment>
+ </hook>
@silence
- <segment id="pause"/>
+ <pause/>
@/silence
- <segment id="close">
+ <close>
<B> Meet <${product} | ${product_pronunciation}>.
- </segment>
+ </close>
@/whole~
</script>
@@ -86,20 +86,24 @@ Segment 的两个独立结构端点;真正的静音或素材行为由外部 Pr
## 1. 文档与 Segment
-一个 Script 有且仅有一个 `<script>` 根。v1 的 Segment 形式只有:
+在官方 Text Frontend 中,一个 `<script>` Surface 实例产生一份 Narrative;
+`<script>` 外壳由 Text 根据导入后的 Surface Registry 分派,Script Parser 接收的
+只是它的 raw body。Script body 的 Segment 形式只有具名块和具名空块:
```svml
-<segment id="intro">
+<intro>
Hello.
-</segment>
+</intro>
-<segment id="pause"/>
+<pause/>
```
规范规则:
- Script 包含一个或多个按源码顺序排列的 Segment。
-- Segment id 在文档内唯一,Segment 不允许嵌套。
+- Segment 的标签名就是 id;id 在当前 Script 内唯一,Segment 不允许嵌套。
+- Segment opening 不接受 attribute;`script` 是外层 raw Surface 的保留名,不能
+ 作为 Segment id。
- 自然语言只能出现在 Segment 内;Segment 之间只允许空白、Selection/Moment
标记和注释。
- 空 Segment 使用自闭合形式。它有结构开始/结束切点,但没有词法锚点。
@@ -115,27 +119,33 @@ Segment 的两个独立结构端点;真正的静音或素材行为由外部 Pr
```svml
@chapter
-<segment id="one">
+<one>
One.
-</segment>
-<segment id="pause"/>
-<segment id="two">
+</one>
+<pause/>
+<two>
Two.
-</segment>
+</two>
@/chapter
```
## 2. Role Cue
-Role Cue 只在 Segment 内的逻辑行首(忽略规范缩进)出现,并开启一个
-spoken turn:
+Role Cue 只在 Parser 已进入某个 Segment body 后出现,并开启一个 spoken turn。
+它的识别由解析状态决定,与逻辑行首和物理换行无关:
```svml
-<segment id="dialogue">
+<dialogue>
<A> What time
is it?
<B> It’s 8:30.
-</segment>
+</dialogue>
+```
+
+下列紧凑写法具有完全相同的 Narrative 语义:
+
+```svml
+<dialogue><A> What time is it?<B> It’s 8:30.</dialogue>
```
它在 `dialogue` 投影中由规范 serializer 输出为:
@@ -153,8 +163,9 @@ Compiler 将结果降低为一个可非连通的派生 SelectionSet。仅有 `<A
为消除歧义:
-- Role Cue 只由“逻辑行首、合法尖括号内容、且不含未转义 `|`”这一位置识别,
- 并且同一行必须跟随非空 spoken content。
+- 在 Segment body 状态中,不含未转义 `|`、没有属性或 `/`、且满足 Role label
+ 字符约束的裸尖括号构造被识别为 Role Cue;它必须在下一个 Role Cue 或 Segment
+ 结束前跟随非空 spoken content。
- 正文里的冒号永远是正文;解析器不通过 `A:` 猜说话人。
- 同一个非空 Segment 要么完全无 Role Cue,要么第一个 spoken atom 必须由
Role Cue 开启。时间标记、注释和布局空白不算 spoken atom;已经以无 Cue
@@ -253,7 +264,7 @@ before ~@x hello @/x~ after
```svml
@pause
-<segment id="empty"/>
+<empty/>
@/pause
```
@@ -483,10 +494,11 @@ Dual Text 内另有:
未知反斜杠转义必须报错。解析优先级固定为:
-1. 已知结构标签与注释;
-2. Segment 内逻辑行首、不含未转义 `|` 的 `<Role>`;
-3. 含一个未转义分隔 `|` 的 inline `<display | speech>`;
-4. 其他尖括号结构报错。
+1. Script body 状态中的注释、temporal marker 与具名 Segment open;
+2. Segment body 状态中的当前 Segment close;
+3. Segment body 中含一个未转义分隔 `|` 的 inline `<display | speech>`;
+4. Segment body 中满足 Role label 合同的裸 `<Role>`;
+5. 其他尖括号结构报错。
Segment 与 temporal name 使用独立命名空间;Selection 和 Moment 共享
temporal name namespace。三者的 id 均满足:
@@ -495,8 +507,8 @@ temporal name namespace。三者的 id 均满足:
[a-z][a-z0-9_-]{0,63}
```
-除“逻辑行首可识别 Role Cue”这一词法作用外,源码缩进、换行和连续空白都
-是 authoring layout,不是 turn、字幕换行或停顿指令。编译投影在 atom
+源码缩进、换行和连续空白全部是 authoring layout,不参与 Segment、Role Cue
+或 turn 的识别,也不是字幕换行或停顿指令。编译投影在 atom
边界保留必要分隔并规范化布局空白;字幕 cue 切分与硬换行由外部 Caption
Program 决定。
diff --git a/tsconfig.v2.json b/tsconfig.v2.json
index b45c9222..3368a676 100644
--- a/tsconfig.v2.json
+++ b/tsconfig.v2.json
@@ -7,7 +7,9 @@
"paths": {
"@svml/protocol": ["packages/protocol/src/index.ts"],
"@svml/core": ["packages/core/src/index.ts"],
- "@svml/driver-node": ["packages/driver-node/src/index.ts"]
+ "@svml/driver-node": ["packages/driver-node/src/index.ts"],
+ "@svml/script": ["packages/script/src/index.ts"],
+ "@svml/text": ["packages/text/src/index.ts"]
}
},
"include": ["packages/**/*.ts"],