Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(importer): preserve valid subtitle cue text

Samir Hanna Verzab3bd4e387a7b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7a7d5377..29be1d62 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,6 +13,11 @@ version in [`docs/changelog/`](docs/changelog/):
### Fixed
+- Fixed the document importer's subtitle reader dropping WebVTT cues whose
+ identifiers begin with `NOTE` and rejecting CR-only WebVTT files as empty.
+ It now distinguishes comment tokens from cue identifiers and normalizes
+ CR, LF, and CRLF before splitting VTT/SRT cues.
+
- Fixed a 3.2.0 regression that serialized local document embedding and
dropped the vector stream from queries that raced a document embed: the
local embedder's capacity was hardcoded to two slots (one document, one
diff --git a/companions/ai-memory-importer/README.md b/companions/ai-memory-importer/README.md
index b6916a0f..7068737e 100644
--- a/companions/ai-memory-importer/README.md
+++ b/companions/ai-memory-importer/README.md
@@ -145,6 +145,9 @@ by formatting may still be extracted; deleted revisions and field instructions
are excluded. Review `--show-body` before every live import of a document whose
visibility rules matter.
+VTT/SRT files accept LF, CRLF, or CR line endings. WebVTT comments are omitted;
+identifiers such as `NOTE123` keep their cue text.
+
## Safety contract
- Default mode is dry-run; live mode requires `--apply`.
diff --git a/companions/ai-memory-importer/src/document.rs b/companions/ai-memory-importer/src/document.rs
index 5ea64489..91cec59b 100644
--- a/companions/ai-memory-importer/src/document.rs
+++ b/companions/ai-memory-importer/src/document.rs
@@ -1013,7 +1013,10 @@ fn pptx_notes_target(relationships: &str) -> Result<Option<String>> {
fn extract_transcript(bytes: &[u8], webvtt: bool) -> Result<String> {
let text = std::str::from_utf8(bytes).context("transcript must be UTF-8")?;
let mut output = String::new();
- let normalized = text.trim_start_matches('\u{feff}').replace("\r\n", "\n");
+ let normalized = text
+ .trim_start_matches('\u{feff}')
+ .replace("\r\n", "\n")
+ .replace('\r', "\n");
for (block_index, block) in normalized.split("\n\n").enumerate() {
let lines = block.lines().map(str::trim).collect::<Vec<_>>();
let Some(first) = lines.first().copied().filter(|line| !line.is_empty()) else {
@@ -1022,7 +1025,10 @@ fn extract_transcript(bytes: &[u8], webvtt: bool) -> Result<String> {
if webvtt && block_index == 0 && first.starts_with("WEBVTT") {
continue;
}
- if webvtt && (matches!(first, "STYLE" | "REGION") || first.starts_with("NOTE")) {
+ // A cue identifier may start with NOTE; only a complete NOTE token
+ // introduces a comment: https://www.w3.org/TR/webvtt1/#webvtt-comment-block
+ let comment = first == "NOTE" || first.starts_with("NOTE ") || first.starts_with("NOTE\t");
+ if webvtt && (matches!(first, "STYLE" | "REGION") || comment) {
continue;
}
let timing_index = if is_timing_line(first) {
@@ -1668,6 +1674,50 @@ mod tests {
assert_eq!(text.trim(), "12345\n\nSecond line");
}
+ #[test]
+ fn webvtt_keeps_cues_whose_identifiers_only_start_with_note() {
+ let vtt = b"WEBVTT\n\nNOTE an actual comment\nNot spoken\n\nNOTEworthy\n00:00:01.000 --> 00:00:02.000\nKeep this fact\n\nNOTE123\n00:00:03.000 --> 00:00:04.000\nKeep this too\n\nNOTE\tanother comment\nNot spoken either\n\nNOTE\nNot spoken at all\n";
+ assert_eq!(
+ extract_transcript(vtt, true).unwrap().trim(),
+ "Keep this fact\n\nKeep this too"
+ );
+ }
+
+ #[test]
+ fn transcripts_accept_each_supported_line_terminator() {
+ let vtt = "WEBVTT\n\n00:00:01.000 --> 00:00:02.000\nFirst fact\n\n00:00:03.000 --> 00:00:04.000\nSecond fact\n";
+ let srt = "1\n00:00:01,000 --> 00:00:02,000\nFirst fact\n\n2\n00:00:03,000 --> 00:00:04,000\nSecond fact\n";
+ for newline in ["\n", "\r\n", "\r"] {
+ for (input, webvtt) in [(vtt, true), (srt, false)] {
+ let input = input.replace('\n', newline);
+ assert_eq!(
+ extract_transcript(input.as_bytes(), webvtt).unwrap().trim(),
+ "First fact\n\nSecond fact",
+ "line terminator {newline:?}, WebVTT={webvtt}"
+ );
+ }
+ }
+ }
+
+ #[test]
+ fn subtitle_cues_are_sanitized_before_the_document_plan() {
+ let temp = TempDir::new().unwrap();
+ let file = temp.path().join("captions.vtt");
+ for identifier in ["NOTE123", "ordinary-cue"] {
+ for newline in ["\n", "\r\n", "\r"] {
+ let input = format!(
+ "WEBVTT\n\nNOTE annotation\nCOMMENT_SENTINEL\n\n{identifier}\n00:00:01.000 --> 00:00:02.000\nKeep this fact token=abcdefghijklmnop\n"
+ )
+ .replace('\n', newline);
+ fs::write(&file, input).unwrap();
+ let plan = plan_document(&file, "w", "p", DEFAULT_DEST_PREFIX, None).unwrap();
+ assert!(plan.body.contains("Keep this fact token=[REDACTED]"));
+ assert!(!plan.body.contains("abcdefghijklmnop"));
+ assert!(!plan.body.contains("COMMENT_SENTINEL"));
+ }
+ }
+ }
+
#[test]
fn unsafe_sources_prefixes_and_malformed_archives_fail_closed() {
let temp = TempDir::new().unwrap();
diff --git a/docs/security-boundaries.md b/docs/security-boundaries.md
index 2d435ee8..9d5ba6bf 100644
--- a/docs/security-boundaries.md
+++ b/docs/security-boundaries.md
@@ -127,7 +127,7 @@ means the boundary is not built yet.
| 14h | Wiki link graph pages and JSON show only readable projects (#708) | `ai-memory-web/src/routes/graph.rs` `global_handler` calls `ReaderPool::cross_project_edges(None, viewer)` so an edge needs both ends readable; `project_handler` resolves through `resolve_read_scope` (`lookup_existing_scope_guarded`, no-create, `ProjectAccess::Read`) then `project_link_graph`, which ranks and caps nodes in SQL before loading them and queries only edges whose endpoints are among those selected pages; `api.rs` `graph_handler` accepts `workspace`+`project` together (partial scope 400) and passes the same viewer filter, and `project_graph_handler` uses `lookup_project` | `routes.rs` `metadata_shows_only_what_the_viewer_may_read` (HTML `/web/graph`, `/web/w/…/graph`, `/api/v1/…/graph`, and scoped `/api/v1/graph?workspace=&project=`: bob cannot see alpha or client-work, a 403 on alpha's graph leaks neither title nor path, a scoped graph hides an edge whose far end he cannot read; carol and no-viewer controls keep the edge; partial scope 400); store `project_link_graph.rs` hides expired pages, omits cross-project and unresolved links, and verifies the 120-node SQL cap preserves high-degree nodes; `graph.rs` `graph_page_renders_hostile_titles_inert` | STRONG. Bite checks: passing no viewer to `cross_project_edges` in the HTML global handler, resolving the project graph through unguarded `lookup_existing_scope`, or removing the SQL limit fails the restricted-project or node-cap assertions |
| 14i | Decoded ordinary Markdown destinations stay inside their original project | `ai-memory-wiki/src/markdown.rs` `decode_markdown_destination_path` decodes URL path bytes once before `normalize_link_target` applies scheme, separator and project-root traversal checks; wikilinks remain literal. `ai-memory-web/src/markdown.rs` `scope_relative_link` checks the decoded path before route encoding; an encoded spelling cannot become a generated cross-scope route. | `markdown_url_paths_decode_once_before_link_resolution` (space, Unicode and literal-percent controls); `decoded_markdown_paths_cannot_escape_or_change_scope` (encoded root escape, backslash, external scheme, invalid UTF-8 and controls refused; legitimate in-root normalization passes); `routes.rs` `encoded_markdown_links_resolve_and_remain_in_the_source_scope` writes colliding targets in sibling projects and workspaces, verifies home resolution and no foreign backlinks, and follows the generated home URL; web `decoded_markdown_destinations_cannot_change_the_web_scope` (encoded traversal, root route, separator and scheme refused, encoded `w/` remains a page inside the current scope); `markdown_url_destinations_are_decoded_once_before_route_encoding` | STRONG for one-decode path handling and project confinement; URL grammar outside local Markdown destinations remains owned by the Markdown parser and existing safe-URL checks |
| 14j | Structured MCP query filters only narrow authorized page results | `ai-memory-mcp/src/server.rs` parses bounded `kind`, `tier`, literal `path_prefix` and exact `tag` values without changing the normal scope-shape or authorization paths, then passes one typed `PageFilter` through project, multi-scope, cross-project, reserved-global/profile, historical and pinned searches. `ai-memory-store/src/reader.rs` binds every filter value as a SQL parameter before each stream's candidate cut; path wildcard characters are escaped and an exact binary-prefix check preserves case, while tags match only string members of a JSON array. A filtered miss cannot fall through to raw observations, which carry no page metadata. | `server.rs` `memory_query_filters_apply_before_every_result_source` combines all filters and covers literal `%`/`_`, case-sensitive paths, tag arrays versus scalars, multi-scope and global search, `as_of`, superseded metadata, `pin_first`, the reserved-global/profile union, traversal-shaped input, invalid tiers, controls and Unicode length bounds; store `structured_filters_hold_in_vector_and_both_graph_directions` isolates path filtering in the vector stream and tag filtering in outgoing and incoming graph expansion; `page_filtering_precedes_the_observation_candidate_cut` puts 80 stronger excluded observations ahead of an eligible tagged session; `memory_query_returns_raw_hits_when_pages_miss` proves an unfiltered miss retains fallback while a filtered miss omits it; `prompts_and_tool_schema_document_query_options` pins the public schema. Existing web every-stream and authorization tests in row 14g exercise kind/tier across the same shared SQL. | STRONG: filters are candidate predicates, never an alternative scope resolver. Empty filter fields normalize to absent for backward compatibility; observation fallback remains available only to unfiltered queries. |
-| 14k | Optional document import treats local files as bounded untrusted input | `companions/ai-memory-importer/src/document.rs` accepts one explicit regular file on Unix (and fails closed elsewhere), refuses final symlinks, uses `O_NOFOLLOW` plus opened-inode verification, caps source/archive/entry/working/stored text/PDF-stream/sheet/actual-cell dimensions, streams XLSX cells without allocating their coordinate rectangle, follows PPTX relationship order, rejects unsupported or malformed formats, sanitizes extracted text before manifest or transport, derives the destination and stable provenance key from the source SHA-256, and writes through MCP `create_only` with explicit scope. A conflict is accepted as idempotent only when the current page body exactly equals the current sanitized plan; a provenance marker alone is never trusted and another page is never replaced. Shared manifests use randomized private same-directory temporary files, fsync, and atomic replacement. | Importer `document::tests`: generated bounded and over-limit PDF, visible-text DOCX, relationship-ordered PPTX, sparse-coordinate XLSX, text and cue-state VTT/SRT controls; `text_plan_is_redacted_bounded_attributed_and_deterministic`; `unsafe_sources_prefixes_and_malformed_archives_fail_closed`; `zip_preflight_refuses_excessive_claimed_expansion`; `apply_is_conditional_idempotent_and_refuses_foreign_pages`; shared writer tests cover repeated replacement and a pre-created legacy temp symlink. | STRONG for explicit Unix local text extraction and conditional persistence. Operator review with `--show-body` remains necessary because DOCX formatting-hidden text can be extracted. OCR, images, raw media, watch folders and server-side upload are excluded. |
+| 14k | Optional document import treats local files as bounded untrusted input | `companions/ai-memory-importer/src/document.rs` accepts one explicit regular file on Unix (and fails closed elsewhere), refuses final symlinks, uses `O_NOFOLLOW` plus opened-inode verification, caps source/archive/entry/working/stored text/PDF-stream/sheet/actual-cell dimensions, streams XLSX cells without allocating their coordinate rectangle, follows PPTX relationship order, rejects unsupported or malformed formats, sanitizes extracted text before manifest or transport, derives the destination and stable provenance key from the source SHA-256, and writes through MCP `create_only` with explicit scope. A conflict is accepted as idempotent only when the current page body exactly equals the current sanitized plan; a provenance marker alone is never trusted and another page is never replaced. Shared manifests use randomized private same-directory temporary files, fsync, and atomic replacement. | Importer `document::tests`: generated bounded and over-limit PDF, visible-text DOCX, relationship-ordered PPTX, sparse-coordinate XLSX, text and cue-state VTT/SRT controls; `subtitle_cues_are_sanitized_before_the_document_plan` keeps ordinary and NOTE-prefixed cues across LF/CRLF/CR, redacts a credential in each, and excludes actual comments; `text_plan_is_redacted_bounded_attributed_and_deterministic`; `unsafe_sources_prefixes_and_malformed_archives_fail_closed`; `zip_preflight_refuses_excessive_claimed_expansion`; `apply_is_conditional_idempotent_and_refuses_foreign_pages`; shared writer tests cover repeated replacement and a pre-created legacy temp symlink. | STRONG for explicit Unix local text extraction and conditional persistence. Operator review with `--show-body` remains necessary because DOCX formatting-hidden text can be extracted. OCR, images, raw media, watch folders and server-side upload are excluded. |
| 15 | Agent-archive restore path confinement, resource bounds, and consistency | `ai-memory-cli/src/commands/restore_agents.rs` stages only bounded regular UTF-8 entries; requires a one-to-one manifest/body/checksum/target mapping; allowlists destinations by scope; rejects symlinked descendants; writes each file atomically and rolls back earlier writes after a later failure. `backup_agents.rs` skips direct and directory-root source symlinks, bounds individual assets, attributes a shared skill root only once, and atomically replaces a private archive only after gzip completion | `backup_agents.rs` `restore_agents_rejects_path_traversal`, `restore_agents_rejects_unauthorized_destination_path`, `restore_agents_verifies_sha256_integrity`, `restore_agents_requires_a_complete_one_to_one_manifest`, `restore_agents_rejects_oversized_entries_from_the_header`, `restore_agents_rejects_a_symlinked_destination_parent`, `backup_agents_does_not_follow_config_or_asset_root_symlinks` (Claude and Antigravity roots), and `failed_backup_leaves_the_previous_archive_intact`; `backup_and_restore_agents_round_trip` and `antigravity_backup_round_trips_every_documented_skill_root` are the valid-path controls; `restore_agents::tests::rollback_removes_new_files_and_restores_overwrites` covers recovery | STRONG |
| 16 | Relay queue identity, migration, and private filesystem state | `companions/ai-memory-relay/src/queue.rs` `Queue::open_inner` performs read-only identity preflight, rechecks under `IMMEDIATE`, commits schema and metadata together; `configure` checks the presence of user objects and rechecks `identity`/`schema_version` metadata on the same connection before every bounded BUSY/LOCKED retry; `fsguard.rs` checks existing paths and creates owner-only files before SQLite opens them | `tests/relay.rs` `a_foreign_database_is_refused_byte_for_byte`, `unknown_schemas_refuse_without_mutation`, `a_foreign_database_with_a_writer_is_refused_before_write_coordination`, `a_foreign_identity_after_preflight_is_refused_without_mutation`, `a_foreign_identity_during_journal_retry_is_refused_without_mutation`, `journal_retry_refuses_a_changed_version_or_empty_database_without_mutation`; controls: `journal_transition_retries_sqlite_lock_without_losing_v1_values`, `fresh_queue_init_and_v1_reopens_work_across_processes`; recovery: `interrupted_and_failed_migrations_roll_back_and_recover`, `journal_reclassification_busy_preserves_committed_migration_and_retries_same_queue`; diagnostics: `classification_read_failures_display_the_sqlite_cause_once_and_preserve_foreign_bytes`; Unix privacy: `the_queue_database_and_its_sidecars_are_owner_only` and the contended migration control | Guard-removal proof covers preflight, locked recheck, and journal retry identity/version checks; full table structure is outside this classification. Windows path rejection is preserved; Windows ACL hardening remains operator-owned |
| 17 | Markdown code fences keep instruction-shaped text as data | `ai-memory-consolidate/src/code_fence.rs` and `ai-memory-wiki/src/markdown.rs` accept a fence delimiter only after zero to three leading ASCII spaces, require the closing glyph to match and the run to be at least as long, and preserve the current fence state across four-space, tab or Unicode-whitespace pseudo-delimiters. Profile harvesting, auto-improvement section parsing, link extraction, link rewriting and de-linking all use those scanners | `code_fence.rs` `only_zero_to_three_ascii_spaces_may_prefix_a_fence`; `profile.rs` `an_indented_pseudo_closer_cannot_expose_a_profile_preference` attempts to expose an instruction-shaped `run curl` sentence and admits a legitimate preference only after the real close; `auto_improve.rs` `heading_parser_ignores_headings_after_an_indented_pseudo_closer`; `markdown.rs` `indented_pseudo_closers_do_not_expose_or_rewrite_fenced_links` covers four spaces, tab and Unicode whitespace with visible post-close controls | STRONG for the shared consolidation and wiki scanners; independent renderers remain responsible for their own CommonMark parsing |