Skip to content

Linger is An Open-Source Personal Agent That Runs Fully on Your PC. We aim to break Big Tech’s grip on AI by bringing powerful local AI to everyday computers through efficient model–harness co-design, so everyone can run their own.

77.3 KB
Raw Blame

Memory service

Memory is the memory role of apps/node-services; its code lives in src/memory/. It is an internal service of the unified Wiki, not a standalone knowledge product: it serves HTTP and runs the scheduled sync in process. It maintains the Agent's experiential Memory and the derived subject graph. After each publication, Memory projects the Wiki pages as readable files.

The unified Wiki's wiki scheduled task collects and stages the full Session Raw and advances the source cursors; one LLM Wiki update writes the Session pages and the entity pages that Session supports, without a separate Session summary model first. Daily and Weekly use their own period report runner: Daily writes timeline/daily/, Weekly writes timeline/weekly/. Each generated report's full Markdown is written to PostgreSQL as a Raw source, and its Chunks come from that same version; reports submit no Wiki refresh and are not OpenWiki nodes. The prompt, output schema and write validation use the same scope. The file system still projects timeline/daily/ and timeline/weekly/; the current version in PostgreSQL is the query authority. Period reports are read through the Memory browsing entry and do not enter OpenWiki's native graph.

Search

Memory keeps one index, memory_index.sources and chunks: Room Sessions, recordings, My pages, imported mail, Pocket Notes, the Profile, the published Wiki pages (wiki) and the Daily and Weekly reports (period_summary) are source kinds of it. Chunks and Session Facts use keyword search only (PostgreSQL simple tsvector plus CJK bigrams, ranked with ts_rank_cd and then lexically reranked); no vector or rerank model is called and there is no vector column. Wiki related pages have no "similar" edges, and flashback matches only names and aliases.

File layout

<AGENT_HOME>/agent-manager/workspaces/context-host/<agent_management_id>/
  memory/
    README.md
    me/{memory,profile}.md
    timeline/{daily,weekly}/<date>.md
    timeline/meetings/YYYY-MM-DD HH-MM-SS.md
    knowledge/{people,clients,projects,topics}/<title>.md
  memory-state/
    evidence/<date>/sessions/            # raw material and collection checkpoints
    raw/                                # index snapshots

AnotherMe mounts the personal directory read-only. Meeting and knowledge pages are projected as readable files after the Wiki publishes; links follow the projected paths, and formal Document IDs and Wiki page keys stay stable. Each Scheduler run and each LLM Wiki generation writes only structured logs (memory_schedule_run, wiki.generation_phase); no run list is kept in memory or on disk. The source trigger's pending targets and its discovery cursor live in Postgres (memory_index.session_trigger_targets and memory_index.scheduler_state). Session checkpoints and pending Wiki downstream markers are persisted too; they are not a run history.

The Memory role applies file retention at start and then once a day: the dated run records of sessionly, daily, weekly and wiki-sync under updates/ are deleted after 7 days without changes, except that a day's Sessionly records stay while that day's Wiki sync is still pending; each Agent keeps only its latest memory.pre-rebuild-*. Apply writes no audit log and there is no manual refresh; old logs, indexes and manual refresh receipts under a subject's memory/updates/ are deleted as well. The same pass deletes the retired scheduler files (updates/wiki-calendar.json* and updates/task-state/session-triggers/), whose state now lives in Postgres. updates/task-state/wiki-pending/ and the lock directories are not pruned.

memory/me/profile.md projects only the current Revision of TeamHub's unified private Profile. The page Profile and AnotherMe use the same user_profile.md; it is generated by Settings → Add source or AnotherMe's personal-profile Skill, merging the old body with new sources when a body exists. Generation uses TeamHub's Profile model configuration with no second Soul call; the weekly step does not rewrite the profile.

The Profile does not enter AnotherMe's prompt; memory/me/memory.md does: built from the Profile and kept current by the Daily and Weekly steps, it is the read-only projection of TeamHub user_memory_revisions.

Memory does no profile inference and does not rewrite identical content; when the authoritative profile is missing it restores the draft projection and clears the old stable body. This file is not a Wiki source and does not enter the Subject Graph. The Chunk sync separately reads the current Profile Revision through memory_source.another_me_subjects and writes it into memory_index as a profile Raw (teamhub://personal-profile/revisions/<id>) for knowledge search. Each AnotherMe keeps one Profile source, replaced when a new Revision is generated, and the Profile takes no part in Session fact extraction.

Both Markdown knowledge trees are read as OKF v0.2. The root index.md declares okf_version: "0.2"; an ordinary concept page must start with parseable YAML frontmatter and a non-empty type. The optional generated, verified, sources, status and stale_after must satisfy OKF v0.2 when present. Top-level status uses only draft, stable and deprecated; business states of Sessions, Projects and Clients use extension fields such as session_status, project_status and relationship_status. When reading or graphing existing Session Memory whose top-level status still holds a lifecycle value (such as summarized, in_progress or waiting_for_user), Memory records the missing session_status in memory and treats the document status as stable; it neither rewrites the source file nor relaxes new writes.

Before writing a batch to disk, Memory checks OKF and the required fields of each content type; the content validator owns these checks and loads no graph schema. If any file fails, nothing in the batch is written.

Daily and Weekly models return one JSON object of report fields (title, description, source_refs, one array per section, Weekly work_intensity) under a strict JSON schema; they never write Markdown, paths or headings. The application renders the whole file: YAML frontmatter, the fixed English H2 headings (DAILY_REPORT_SECTIONS / WEEKLY_REPORT_SECTIONS in @linger/knowledge-format/sections), one bullet per item, None observed. for an empty section, the fixed no-baseline sentence when there is no yesterday Daily or previous Weekly, and a final ## Sources section. Daily writes only the day's report and Weekly only the week's user dynamic; neither deletes files. A candidate the schema or the renderer rejects is corrected within the memory kind's three attempts; an invalid candidate never reaches Memory. A manual apply validates the saved model JSON against the same schema and renders it without calling the model.

Models select evidence only through source_refs, the short refs of this collection, and cite inside item text as [[source:<source_ref>]]. The application renders source titles, YAML sources and Markdown links. Daily cites the day's raw conversations, one source_ref per Room; Weekly cites the week's Daily reports and marks the previous week only as a comparison baseline. Raw internal paths or URIs in model text, unselected or unknown refs, duplicates and malformed markers are rejected. Weekly runs only when a current-week Daily has a list item in an activity section (Daily Work, Resolved Items, Unresolved Items, Tomorrow Focus or Pattern), and its output must select at least one Daily. Daily reads the day's raw conversations, yesterday's Daily and the Pocket Note; it never waits for the Wiki, since Room chats have no Session pages. Tracking Items read the same raw text and Pocket Note. Client, Contact, Project and Topic pages are maintained only by the Wiki generator; Memory keeps no entity files.

The shared OKF layer keeps valid YAML and fixes only a zero-indented root mapping field that the parser locates exactly. When the nesting cannot be confirmed, or there is another syntax error or a conflict, the candidate is still rejected.

TeamHub data access

Memory connects to TeamHub's database through TEAMHUB_DATABASE_URL, but reads only the read-only memory_source views and functions that TeamHub migrations define, and writes only memory_index. The role has one connection pool (src/memory/database.ts, at most 10 connections): indexing, collection, queries and command lines all use it, and it closes when the role stops or the command line ends. tests/memory/database-boundary.test.ts ensures that Memory's SQL references only these two schemas and that the pool is created only there.

  • another_me_subjects: the one definition of an AnotherMe subject, an active AnotherMe and the Human it represents.
  • subject_rooms, conversation_messages, message_attachments: the Rooms a subject may remember (the AnotherMe or the Human is a listed member and the Room is not deleted), with their conversation messages and attachments (the message_attachments view expands the messages.attachments list).
  • subject_recordings: the finished text transcripts in those Rooms. Private questions and task requests go only to the recorder's own AnotherMe.
  • finalized_meetings and human_can_read_document(): a meeting is listed while both its recording and transcript Documents are active; hidden_meetings holds the meetings the Space's Human removed (hid) from Meetings.
  • profile_revision_dates: when each Profile Revision was written, used as the Profile Raw's date.
  • pocket_notes and personal_documents(): Pocket Notes (without deleted dates), and the current personal documents in My Pages and in the mail-import Wiki sources; a native page gives the Markdown Export Artifact of its current Revision.

When TeamHub changes a table these views depend on, update the views in the same migration; memory_source_views_test.go checks the authorization rules in the views.

Starting

NODE_SERVICES_ROLES=memory pnpm --filter @linger/node-services dev

The root pnpm dev already starts Memory in the node-services process.

Core configuration:

  • AGENT_HOME: the Agent state directory, ~/.mirror by default. Memory writes to the Agent Context Host root <AGENT_HOME>/agent-manager/workspaces/context-host, the same directory the Agent role mounts; both come from the same function in src/paths.ts.
  • LINGER_SECRET_SEED: the service credential seed; Memory refuses to start without it. The credential derived from it with the node-services-internal label authenticates internal clients calling Memory (shared with TeamHub's other calls to Node Services): today MCP Server queries, development LLM Wiki generation log reports, TeamHub's Wiki graph update commands and the rebuild CLI. The Agent Service's flashback calls Memory directly in the same process, not over HTTP. The credential derived with the teamhub-internal label authenticates Memory's calls to TeamHub (shared with Node Services' other calls to TeamHub).
  • TEAMHUB_BASE_URL: reads the latest published Wiki and requests a Wiki refresh after Sessions are indexed.
  • Business dates use the process's local time zone (TZ): the collection window, Session dates, meeting directories, Pocket Note reads and calendar scheduling all use it, and there is no separate time zone variable. Compose containers default to UTC; set TZ in .env when needed.
  • These features are always on, with no switch: the in-process scheduled sync (hourly Wiki, daily Daily and weekly Weekly), the targeted refresh after a Room is quiet for 10 minutes, the separate Session refresh of formal recording transcripts, Session fact extraction and fact recall, the nightly tracking-item pass, and the nightly revision of a person's reminder guidelines from the day's proactive reminder feedback. Session facts need TEAMHUB_DATABASE_URL.
  • Built-in constants (no longer environment variables): calendar tasks and their retries run at most 4 AnotherMes at once (a deployment has one account, usually one AnotherMe), and each person's Room Sessions also run 4 at a time; weekly always runs on Monday; Memory model requests share a 10-minute business cancellation, response header and body read timeout, with no streaming; the Chunk reconciliation runs hourly, with at most 4 Sources indexed at once per Agent.
  • TEAMHUB_DATABASE_URL is required: the Memory role keeps its scheduler state in that database and does not start without it. It also enables the Raw Chunk persistent index and search.
  • The full Raw Chunk reconciliation runs once at service start and then at the built-in interval.
  • Session facts: after Raw Chunk sync finishes an Agent (without blocking indexing), Memory extracts Session facts into memory_index.session_facts, following Hindsight retain. Facts come from Room conversations, recording transcripts, Pocket Notes and imported mail; My pages, the Wiki, period reports and the Profile stay chunk-only. Mail is chosen by the - Class: line TeamHub writes into it: self_sent, direct_inbound and system_notification are extracted, and so is every message of a thread with a self_sent message; marketing, subscriptions and drafts are not. Each window's Context line names its origin, so the one prompt knows a transcript may not label speakers, mail must tell the sender from the user, and a notification may hold no facts. A window's Event Date is its first Room message time, else its source date at local midnight; it is shown in the process time zone, so a message sent just after midnight belongs to the Human's own day rather than the UTC one. Consecutive Chunks form windows of at most 3,000 characters, extracted with apps/node-services/prompts/memory/facts/fact-extraction.md (the Hindsight concise prompt) at low reasoning effort. Up to 16 windows form one batch, and each batch runs as one ModelCall chain of kind memory, keyed memory.facts:<sha256 of the batch's sorted window hashes> so a restart asks the same key. Its calls run in order. Each model attempt queues separately, so chat can start before the next step, retry or correction. An answer still invalid after them is split into two halves, and windows whose facts drift into another language are extracted once more; both are steps of the same chain (first, split-<n>, <step>-drift). Each sync keeps at most two batches on the queue at once. Backfill write transactions are short and share Memory's connection pool with recall. Windows are recorded by content hash in memory_index.session_fact_windows, so only new or changed windows are extracted and failed windows retry on the next sync; facts of unchanged windows are rebound to the Chunk with the same content in the new Raw version.
  • Fact recall: a Raw Chunk query opens a separate fact channel that recalls facts by keyword (the query's words merged with alternates); with an event time window, facts inside the window that match the keywords get a second vote, fused by RRF, and the response returns at most 30 facts at the top level. Without keyword matches it returns none. Each fact carries its Raw's evidenceId and sectionId for a follow-up read. Facts take no part in the Chunk ranking and take at most a quarter of the response's output budget (below); with or without the fact channel, both lists are ranked the same. A summaries-only search omits facts; a failed fact recall only omits facts.

Memory's endpoints mount on the shared Node Services listener (127.0.0.1:4319 by default), all under the /internal/v1/ prefix and protected by the workload token. The process answers GET /healthz, GET /startupz and GET /readyz.

  • POST /internal/v1/published-wiki-graph-updates: called after TeamHub atomically publishes a new Wiki Revision; updates only the AnotherMe graph of that owner and skips an identical Revision.
  • POST /internal/v1/pocket-note-tracking-refreshes: TeamHub calls it about five minutes after today's Pocket Note changes. It reads the note as it is then, by tenant_id + owner_actor_id + date, and updates that Human's Tracking Items from this page alone: open items from the page that it no longer mentions are deleted, the rest is the nightly comparison (add, update in place), and expired items are not cleaned up. Without an AnotherMe it answers 204; a model or TeamHub failure answers 503 and TeamHub retries.
  • A meeting follow-up needs no endpoint: once the hourly Wiki or a recording's Session collection stages a formal transcript under evidence/<date>/sessions/recordings/, Tracking Items are updated in the background from that transcript alone (no cleanup, never holding the Wiki run); a failure is logged as ERROR and skipped, and that night's Daily catches up.
  • POST /internal/v1/personal-knowledge-graph-visualization: protected by the workload token; reads the existing AnotherMe graph for tenant_id + owner_actor_id and returns embedded HTML without internal subject identity. The request is read-only and triggers no graph build; it returns 404 when there is no graph.
  • POST /internal/v1/query: protected by the workload token; the one knowledge search over every source kind, returning two separately ranked lists (below) for the question, optional alternates, optional absolute dates, optional scope (originals or summaries, one list with the whole budget) and optional page.
  • POST /internal/v1/personal-knowledge/evidence: protected by the workload token; reads at most 8 selected Chunks with an opaque raw_ Evidence ID returned by a query. The service revalidates the ID against the database's current Raw version and content hash and accepts no file paths.
  • GET /internal/v1/health/failures, POST /internal/v1/health/failures:retry: protected by the workload token; a read-only failure summary and manual recovery, see "Failure visibility and manual recovery" below.
  • POST /internal/v1/rebuild, GET /internal/v1/rebuild: protected by the workload token; rebuild one Another Me's Wiki, Daily and Weekly over its actual Session dates, see "Production history rebuild" below.

The MCP Server calls these endpoints after TeamHub authorizes the Context and the specific Tool. AnotherMe, General and Builder Agents of one Human share that Human's knowledge in meta / alter_ego Contexts. TeamHub resolves the hidden personal_knowledge_agent_id from the current registration's Human owner and Tenant; MCP passes it to Memory as agent_id, while agent_actor_id keeps the real caller. The model cannot choose sources and cannot treat Room members or recent speakers as knowledge owners; when unauthorized or a source is unavailable, Memory does not fall back to another Agent's knowledge. Memory's Subject, Tenant, Human and Raw snapshot checks stay unchanged, as do the graph and Raw generation flows. There is no cross-user Agent transfer flow, and the general MCP credential and session revocation policy is unchanged. The model has one search, teamhub knowledge search (query + alternates? + dateFrom/dateTo? + eventFrom/eventTo? + scope? + page?), and one read, teamhub knowledge read (evidenceId + sectionRequests[{sectionId, offset?}]). Actor, Tenant, Human owner, sources and file paths are set by trusted services, never by the model. For overviews the Agent answers from the summaries list when it states what it needs; for details, exact words and counts it answers from the items list of original sources. The search does not build the graph; the graph only serves the Workspace visualization and flashback. Memory builds OpenWiki's native WikiGraph from the indexed pages of the published Wiki when it is read: one node per page with a relative path ID, source/target edges only from parseable Markdown links, plus the inferred mention edges in inferredEdges. Nothing is stored; flashback keeps the graph of the indexed revision in memory. Without a published Wiki there is no graph, and Memory files are never scanned as a fallback. No shared-source or root edges are built. Old Sessions are not dropped from the graph after a seven- or fourteen-day window; date filtering is stated explicitly by the query. The visualization is rendered per request; no HTML is written to disk. Raw takes the currently authorized version in PostgreSQL as the authority; the file system is only staging and projection. Raw is checked by Tenant, Human, Agent and content hash; publishing Raw does not change the graph, and original-text queries do not depend on the graph. memory-state/raw/ is the current original-text storage path, not old graph data; cleaning old graphs must keep it and all Memory and Wiki bodies.

A Raw Chunk query deduplicates by immutable Raw source and returns at most 20 items. Three kinds of time can come with a query, and none filters a source. The two windows are each one more recall channel (following Hindsight's temporal window) that also requires the query keywords; as_of only reorders the fused list:

  • event_from/event_to, when the asked event happened, for an event that may have been told later: a fact votes when its event interval overlaps the window, or, with no interval (something said rather than an event), when its source date falls inside it; a mention in the window does not count for an event elsewhere, unlike Hindsight. A period report or a Session page is about its own day and votes by that date. A conversation, transcript, note or mail Chunk only knows when it was recorded, so it takes no event vote: its events reach the Agent through the facts list, as in Hindsight, where time is judged on facts only. In a benchmark of events told later, a vote through facts gave original Chunks no better rank than none.
  • date_from/date_to, when the asked records were made (mail received in January, Monday's meeting): a Chunk or a fact votes when its source's local date falls inside the window.
  • as_of, the day a changing value or state is asked for (today for the current one): Hindsight's recency boost without a cross-encoder. Each fused candidate's rank maps to a base score from 1.0 down to 0.1, multiplied by 1 + 0.2 * (recency - 0.5); recency is 1.0 on that day and falls linearly to 0.1 sixty days before it (Hindsight uses a year; a changing value is restated within weeks), a source after that day takes the 0.1 floor, and an undated source is neutral (0.5). Close matches move toward the record nearest before the day; a weak match is never lifted over a strong one.

A window channel takes at most 60 by ts_rank_cd, then picks 10 round-robin over 8 time buckets so no single day fills it. Each wording (the query and each alternate) runs its own time channels. A Wiki page other than a Session page is dated by the Wiki's publication, not by what it is about, so no time channel takes it; a Session page (sessions/<date>/<session>.md) is dated by its Session. TeamHub forwards the Agent's dateFrom/dateTo, eventFrom/eventTo and asOf unchanged. None of them excludes a source: in LongMemEval the model writes the current date into a time parameter almost every time, and a filter would exclude the earlier conversation that holds the answer. For "what was done in some month" search by entity and action and verify the event time from the original text, which lets later records recount past events. Relative event times resolve against the date of the source they appear in, and an explicit historical cutoff still applies.

Raw and Wiki queries may also pass alternates: at most 6 synonymous wordings or translations, each at most 200 characters. Each wording runs its own keyword search in parallel with the query and votes as its own RRF list, so a synonym match is not buried under documents that only repeat the query's original words; inside a time window each wording votes again with the window's matches, ranked by distinct-term coverage like its keyword route. Raw's BM25 fallback scan and Session Fact keywords use the query's words merged with all alternates. It adds no model call and no network round trip while the user waits.

One search returns two separately ranked lists, because keyword scores cannot compare a long Daily report with a short chat message fairly: items holds original sources (room, recording, mail, my_pages, pocket_note, profile, at most 20) and summaries holds the Wiki and Daily/Weekly reports (wiki, period_summary, at most 10). Each list runs its own keyword search (60 Chunks), fusion and coverage rerank, then keeps one entry per source. An entry has only citeAs (S<n> for items, W<n> for summaries), evidenceId, kind, title, timeHint, sections (at most 4 matched and neighboring sections to read), roomId for a Room conversation, and either content (text around its best match, keeping the time and speaker heading of its message) or snippet (about 200 characters around the best match).

Memory keeps every search and read result within an output budget of JSON characters (UTF-16 units); the CLI passes results through unchanged. The Daemon limits one tool result to 15% of the model's context window, between 4,000 and 25,000 tokens, and to 12,000 tokens when the window is unknown. Codex counts four bytes per token and cuts the middle out of a longer result, so it keeps limit×4/3 characters of three-byte text such as CJK; Claude's Bash keeps the same number of characters and saves a longer result to a file. The budget is 90% of those characters for the model the owner's calls run on, leaving room for the CLI's labels; at no more than three bytes per character, a bounded result reaches the Agent whole on either Runtime. The window comes from the owner's Computer through the model call layer; an unknown window, or an offline Computer, gets the unknown-window limit.

The top entries of each list carry content: from a budget of 20,000 characters up, 5 items and 3 summaries of about 1,500 characters; below it, 3 items of about 1,200 and 2 summaries of about 800. Session facts take at most a quarter of the budget, the best-ranked first. Over budget, Memory drops in order: the lowest-ranked entries without content (alternating lists), facts from the tail, content beyond the floors (3 items, 2 summaries), then shortens floor content down to 600 characters; floor entries are never dropped. Room left over gives more entries content, items first. A scope search gives its one list the items' share.

  • When entries, content or facts were left out, or more entries exist, the result carries the gap results_trimmed; nextPage names the page that continues the same search, laid out again from the first page so it starts exactly after the last shown entry. Status follows the gaps: entries with no gap are grounded, entries with any gap (this one included) are partial, and no entries is empty. grounded still means only that the search matched, not that the evidence answers the question.

Raw query candidates, MCP read input and output, and Memory HTTP reads use one Evidence ID protocol: raw_ followed by the first 16 characters of the source ID, an underscore and the first 16 characters of the content hash, naming one original snapshot (^raw_[a-f0-9]{16}_[a-f0-9]{16}$); Wiki pages and reports use the same IDs. A read resolves the prefix within the currently authorized owner scope and checks the current version; an ambiguous prefix or a changed version returns not found. An entry's sections list only its matched chunks and their neighbors. The model must pass IDs back unchanged; it cannot build, convert or replace them with file paths. The format is defined by the Memory OpenAPI, and the public validation rule exported by the generated client is shared by MCP and Memory HTTP; a valid format does not grant access, and reads still verify the trusted identity and the database's current snapshot. While relevant candidates are unread, the Agent cannot conclude that information is missing; authorized cross-Room Summary text and Raw original text are both usable, and the Room file projection boundary does not narrow what the personal knowledge tools are authorized to read.

The Raw directory comes from deterministic splitting of the source text by apps/node-services/src/memory/search/chunks.ts, and the second step reads only the Chunks the Agent explicitly selects. The requested sections share the read budget left after the title, source and directory, in request order; a section's unused share carries over to the next. A section that does not fit returns a non-empty first page with truncated and nextOffset (a UTF-16 offset that never splits a surrogate pair), and reading on from nextOffset continues exactly. Unknown Chunks go into missingSections. A read confirms again that the Evidence ID still names the currently authorized source version in the database, so old IDs, retired sources and arbitrary caller paths cannot bypass the boundary.

With the persistent index enabled, Memory reconciles the current Raw source snapshots every hour by default; start-up itself triggers no scan. The Chunk flow only reads sources and writes to the database; it never creates, overwrites or restores original files or the memory-state/raw/ projection. The only deletion is the projection of a deleted Room (see below). The immutable normalized Raw text and manifest live in PostgreSQL memory_index.source_versions, one current version per source. Memory prepares the full chunks first, then in one transaction saves the Raw version, replaces all of the source's old chunks, switches sources.active_sha256 and deletes the old version; the Session Wiki update reads and checks the committed PostgreSQL version. Any failure in the transaction rolls it back and the old version and keyword index stay usable. For one Agent, the full sync, dated Raw sync and publication callback sync share a serial queue and a cross-process lock; triggers with the same key that arrive during a run merge into one follow-up, which rereads the current authoritative source, so an old trigger never writes back an old version. One Agent sync indexes at most 4 different Sources in parallel (a built-in constant); different Agents run their own queues in parallel. An identical content hash and full index profile skip the source, and queries always read the active version of the last complete committed transaction. memory-state/raw/ is maintained by the upstream collection flow for manual inspection and compatible tools; a missing local file is never a reason to delete original text from the database, and a file whose content hash differs cannot overwrite the current PostgreSQL version. The original Version bytes of ordinary Documents, Connected App Documents and recording transcripts live only in object storage; TeamHub's Wiki dispatch reads the object each time and checks its length and SHA-256, with no second copy in PostgreSQL. memory_index.source_versions holds the normalized canonical Raw shared by Chunks and the Session Wiki, with PostgreSQL as the primary store; memory-state/raw/ is only a deterministic mapping and takes no part in choosing the current version. The reconciliation covers Sessionly Room messages (text, author, time and the text attachments read), the transcripts of finished recordings, the current Document Version in My Pages, Wiki sources a Human uploaded or imported directly, and Connected App content already materialized as the same Document; one Connector Document is not indexed twice because it is also in My Pages. On the next reconciliation My Pages deletes sources no longer in TeamHub's source list, cascading their versions, chunks and Session facts; a new version replaces only that source's chunks. The Daily/Weekly reports are derived summaries, indexed as their own period_summary sources and never as Raw again. When a current safe Wiki page references an exact historical Version of a Document, that Version enters Raw/Chunk ahead of the Document's current head, so a Wiki reference never points at the uploaded original while the Chunk indexes a later export. My Pages and imported mail enter only when the Wiki whitelist would admit them (TeamHub's source_rejection_reason is empty): images, scanned PDFs, spreadsheets, data, log and code files, archives, and audio or video stay out of the index, because the Wiki never records them. Memory does not normalize Documents: it reads My pages and imported mail as the Markdown TeamHub stored in document_texts (one River job per version), through memory_source.personal_documents and, for Documents whose indexed hash differs, personal_document_markdown. A My pages Document is dated by its Version; an imported mail message by the send time in its - Timestamp: header and named by its # subject line, because TeamHub stores every message at the time of the import (the Version time and file name are used only when the header has no readable time or subject). The normalizer TeamHub's job calls is apps/node-services/src/document-normalizer, served by LLM Wiki in a restricted worker thread of this process. It converts PDF, DOC/DOCX, PPTX, XLSX, ODT/ODS/ODP, EPUB, EML/MHTML, Office templates and SVG into canonical Markdown. There is no OCR: a PDF gives only its text layer, so a scanned PDF has no text, and raster images are unsupported. Conversion needs no network, Docling, LibreOffice, extra API key or separate service. After installing the repository's locked dependencies, pnpm dev uses this path; CI and the production image carry the same parsers. The index excludes generic ZIP/gzip, audio/video and legacy XLS/PPT through document_wiki_source_heads, without mistaking ZIP-based Office/EPUB for archives. There are no backends for macro-enabled Office (PPTM/XLSM/XLTM), Box Notes, Outlook MSG, Apple Pages, AFP/EBCDIC or DocLang. AsciiDoc and LaTeX uploaded as UTF-8 text/* can be read as text but have no dedicated parser; .adoc with application/octet-stream and .tex with application/x-tex are refused. Complex PDF layouts need their own acceptance. A version the normalizer cannot read has no ready text in TeamHub and is simply not indexed; Memory keeps no failure record of its own, and the next version is indexed once its text is ready. Markdown is split into Chunks by heading level and paragraph boundaries; consecutive timestamped turns merge into a conversation Chunk with context, so one sentence never takes a whole result. Ordinary blocks do not overlap; only very long blocks use a small overlap. Each record keeps the Raw relative path, heading path, one-based line numbers and content hash. CJK bigrams and English or numeric whole words are generated in the application and written into a field-weighted search_tsv; the GIN path ranks with ts_rank_cd. This is PostgreSQL FTS, not strict Okapi BM25. At query time each channel (the query's lexical match, each alternates lexical match, the time window) takes its top 50, merged by RRF with k=60; adjacent results from the same source are deduplicated, preferring three per source before backfilling. The Chunk ranking may keep several matches from one source, but Personal Knowledge output deduplicates by immutable Raw source and returns one evidence ID per source: the top Chunk is the preview, and the full Chunk directory is offered for the second step. When neither requested list has a valid chunk match, Memory runs a BM25 fallback over the text of the current PostgreSQL Raw versions within the same Tenant, Human owner, Agent and date range (per list, at most 30 recent sources and 3 MB of text). It reads no local files, never scans all Raw, and never revives a retired My Pages source from a local Document projection. Fallback candidates can be verified by the same boundary's second-step Evidence read; a database query error returns partial with raw_chunk_search_unavailable rather than posing as an empty result.

The index lives in the separate memory_index.sources / memory_index.chunks tables of TeamHub's PostgreSQL. The source table also records JSON metadata such as source_kind (Room messages, recording transcript, Document, Pocket Note pocket_note), origin_kind (Room, Recording, My Pages, Wiki source, Pocket Note pocket_note), version and URI. Both candidate SQL queries always filter by the Tenant, Human owner and Personal Knowledge Agent the server resolved. Each source keeps one active_sha256; a CAS on the hash read earlier, in the same transaction, deletes all of the source's old Chunks and writes all new ones. A slow task therefore cannot overwrite another version committed while it prepared, and other sources' chunks are unaffected. Old and new chunks are never spliced incrementally. The skip condition checks the Raw, the index profile, the chunk count and the content version they belong to, so a changed active hash is never mistaken for complete old chunks. PostgreSQL sources are never cleaned up from a local directory listing. Sources are deleted only with their owner's lifecycle: Documents follow TeamHub's authoritative list, with the scan start time protecting concurrent new publications; Session and recording sources record room_id, and after a Room is deleted (memory_source.deleted_rooms) the reconciliation deletes its local Raw projection and index while keeping the Session summaries; after an AnotherMe is revoked (memory_source.revoked_another_me_agents) the full sync deletes its sources (failure records go with the source rows).

Only a Raw snapshot saved successfully during sync can become searchable original text; without a snapshot the read returns raw_source_not_found, and a summary page never stands in for the original. Raw manifests and snapshots carry the content hash, source identity, anchored file and source version checks; when these fail, old Raw Evidence IDs stop working. Raw evidence identity comes from the source and the content hash, is not written into native graph nodes, and search does not go through the graph, so deleting old graphs does not change these evidence IDs.

My Pages and Wiki source documents read their Chunk content directly from TeamHub, build the manifest in memory and write it to PostgreSQL without a local snapshot; the manifest anchors the exact file (an uploaded file's Document Version, a native page's Export Artifact) and its content hash. These sources become Raw Chunk candidates once normalized and stored; candidate SQL always filters by the server-resolved Tenant, Human owner and Personal Knowledge Agent and cannot bypass authorization.

Session original snapshots are laid out by the Session's local business date, not by write time or the messages' UTC date: memory-state/raw/<YYYY>/<MM>/<DD>/<roomId>/ holds the current manifest.json and Markdown named by content hash. The raw collection input is still written before the summary to updates/sessionly/<date>/<subjectKey>/<tenantId>/<roomId>/collection.json; the Raw search snapshot is saved after the Session summary is written and holds message times, authors, text and the text of attachments read successfully, without the Provider transcript. Each message heading is ## <local time with offset> · <author>, such as ## 2026-11-11T11:45:38+08:00 · Ada, in the process time zone, so Agents and fact extraction read the time the Web client shows. Snapshots of earlier days keep the UTC headings they were written with; both forms carry an offset. A standalone recording replaces the <roomId> subpath with recordings/<captureId>; its Raw is the full original transcript without uploader speaker labels. The formal audio is still managed by TeamHub's object storage.

Raw Evidence identity comes from the source ID and content hash, independent of graph nodes or local directories. The database holds the currently authorized sources and version text, and file system directories are only staging and projection; choosing the current version, identity checks and text reads all use the database. Without a database source, local manifests are never enumerated to restore candidates, and a database error cannot pose as a successful file fallback. When no Chunk matches, BM25 may run over the Raw text in the database, never over the disk as a second authority. A returned Evidence ID is rechecked against the database's current version and hash when read.

Writing a Raw projection still needs atomic replacement and the workspace lock, and old files of the same source are reclaimed only after the new projection is published; this produces or changes no graph. The memory-state/raw/ directory name refers only to the original-text projection, not to the deleted old graph format. Graph cache cleanup must keep original text and version data; a lost projection does not affect sources in the database, and reads do not rebuild projections.

The graph comes only from the indexed published Wiki; without one there is no graph. The visualization supports zoom, drag, search, native page-type filters and node details, and writes no HTML.

The first Raw Chunk step returns only candidates for the Agent to choose from; it produces no user answer and does not mean the text was read:

{
  "schemaVersion": 2,
  "status": "partial",
  "items": [
    {
      "citeAs": "S1",
      "evidenceId": "raw_3faaaaaaaaaaaaaa_7cbbbbbbbbbbbbbb",
      "kind": "room",
      "title": "2026-09-09 Hardware partnership",
      "timeHint": "2026-09-09",
      "content": "## 2026-09-09T02:10:00.000Z · Alice\n\nWe collect the hardware requirements; they confirm the delivery date.",
      "sections": [{ "sectionId": "chunk-1a2b3c4d-0", "title": "Messages" }],
      "roomId": "room_hw"
    }
  ],
  "summaries": [
    {
      "citeAs": "W1",
      "evidenceId": "raw_8dcccccccccccccc_2eaaaaaaaaaaaaaa",
      "kind": "wiki",
      "title": "Hardware partnership",
      "timeHint": "2026-09-10",
      "snippet": "Open work: delivery date pending the partner's confirmation.",
      "sections": [{ "sectionId": "chunk-5e6f7a8b-2", "title": "Open work" }]
    }
  ],
  "gaps": [{ "code": "results_trimmed" }],
  "indexedAt": "2026-09-10T00:00:00.000Z",
  "page": 1,
  "nextPage": 2
}

The Agent then selects the Chunks it needs with teamhub knowledge read; a successful response:

{
  "schemaVersion": 1,
  "evidenceId": "raw_3faaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_7cbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
  "title": "September 9 hardware partnership call",
  "source": {
    "kind": "room",
    "ref": "raw_3faaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_7cbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
    "label": "September 9 hardware partnership call"
  },
  "timeHint": "2026-09-09",
  "directory": [
    { "id": "chunk-1a2b3c4d-0", "title": "Action Items", "level": 2 }
  ],
  "sections": [
    {
      "id": "chunk-1a2b3c4d-0",
      "title": "Action Items",
      "level": 2,
      "content": "- Us: collect the hardware requirements.\n- Them: confirm the delivery date.",
      "offset": 0,
      "truncated": false
    }
  ],
  "missingSections": [],
  "complete": true
}

source.ref is an opaque source reference, not a display field, and cannot be used as a file path. Only sections[].content is text that has been read; the Agent answers the original question from it together with the current developer instructions, Profile and Personality, and never sends candidate objects or source references to the user as they are.

Scheduled tasks

Index timing

Both kinds of index are isolated by Tenant / Human owner, and Raw is also bound to the user's active AnotherMe:

  • Raw indexing does not wait for a Wiki: every active Another Me in memory_source.another_me_subjects is a subject, including a Human who has only My Pages, Pocket Notes or a Profile. A Wiki publication never triggers Raw indexing. Imported mail is not placed in My Pages but is indexed as the mail origin.
  • Raw independently checks the Human's latest My Pages bodies, mail import sources and authorized Session text every hour; Session updates and ordinary Wiki refreshes do not trigger Raw again.
  • A Wiki publication indexes only the Wiki pages.
  • TeamHub's durable publication job calls Memory and retries until indexing succeeds; Memory keeps no queue or receipt of its own, and a repeated call indexes only pages whose Document Version it does not hold. An indexing failure never rolls back the published Wiki.
  • Subjects come from the database, not from local directories. My pages are read from TeamHub's document_texts, so no snapshot is prepared and no Wiki refresh is needed first; an old Wiki reference never overrides a file's latest version.

The one index (memory_index.sources and chunks, for every source kind including wiki and period_summary) is created by TeamHub migrations; deploy TeamHub before Memory. Runtime switches stay explicit configuration, and the root .env is never changed automatically.

The calendar saves its planning cursor, unfinished tasks and success receipts in one memory_index.scheduler_state row in TeamHub's database, so the Memory role does not start without TEAMHUB_DATABASE_URL. The first start checks from midnight of the current business day; after downtime the saved cursor catches up the missed closed dates in Session → Daily → Weekly order. Saved successes never rerun and failed tasks survive restarts. Missed ordinary hours are not rescanned one by one; only closed dates and the current hour are collected. A calendar saved in another time zone is ignored and planning starts over in the current one.

Memory has one in-process scheduler and one timer; there is no cron or extra background service. Each pass plans calendar work, admits rebuild steps and runs the periodic work that is due: the source trigger scan and the report reconciliation every minute, the Raw Chunk sync at start and then hourly, and file retention at start and then daily. After a pass the timer sleeps until the next local hour, the next due calendar or rebuild retry, or the next periodic work, and never longer than one minute. A task that settles, a queued rebuild and a manual retry wake it at once; passes stay at least one second apart. The source check reads only database metadata; it never reads message text or transcripts or calls a model on each pass, and it does not block the hourly tasks from being queued. Tasks run in one lane per person (memory subject): one person's Session, Wiki, Daily and rebuild run strictly in order, different people run in parallel, and one person being stuck (for example a meeting request waiting for a running Wiki refresh) does not block others. The calendar tasks for everyone (hourly Wiki, Daily, Weekly) still run in order, and each person's part enters that person's lane. At most four source triggers run at once. The two entries below run with the in-process Scheduler; an explicit replay plan does not enable them.

  • Quiet Room trigger: checks the last activity by the conversation messages Memory already uses, not rooms.updated_at, and ignores agent_runtime and task_terminal_notice. After 10 minutes without a new message it collects only the target subject, Room and business date's Session; when the item is dequeued it checks the quiet state again, and a new message across midnight restarts the timer. Queueing may run it later than 10 minutes. It shares the (created_at, message_id) checkpoint with the hourly task; after a success with no new messages it does not summarize again, and a new message restarts the timer without restarting the Room process.
  • Recording-ready trigger: each Capture maps to its own file evidence/<date>/sessions/recordings/<captureId>.md; several recordings in one Room are never merged and do not wait for the Room to be quiet. Only finalized Captures with a formal transcript Document Version are handled. Late finished sources are found by the Capture update time and the version creation time, deduplicated by documentVersionId + sha256, and a regenerated transcript updates the same Session. The file date is fixed to the local date when the first formal transcript Document was created; the actual recording time is kept in metadata. The Room keeps only permissions and file delivery; recording participants are not inferred, and its chat and run logs are not read. The text is read through TeamHub's authorized content API, never from local Artifact storage, and in-progress HLS, ASR staging and audio bytes are never read. A subject with standalone recording handling no longer puts the Capture transcript attachment into the Room Session as well. When the recorder is the subject's own Human, the same query also reads voice_capture_questions and renders them by recording time (the answer cutoff minus requested_transcript_start_offset_ms) as two sections, "Questions asked during the meeting" and "Requests the recorder gave Another Me during the meeting" (Ask requests recognized as tasks go into the second, marked with whether they were handed to Another Me). They follow the transcript into the Session Evidence, Raw and Chunks; empty sections are omitted, and other members' subjects never read this private content. Recording notes are retired and are no longer a meeting summary source. The deduplication version is then documentVersionId:sha256:<first 16 characters of the private sections' SHA-256>, so a new question or a change in a task request's sending state restages the Session; new question creation times also count for discovery. The meeting page only requires the checkpoint to start with the current transcript's documentVersionId:sha256.

One UTF-8 transcript is capped at 4,000,000 bytes and keeps the full original text and timecodes. The version checkpoint advances only after the Session Evidence and Raw are saved and the downstream pending marker is on disk. Exceeding the cap, a permission failure, a byte count or hash mismatch, or invalid UTF-8 never advances the checkpoint. One Raw is capped at 5,000,000 bytes; formal collection no longer calls the old summary model segment by segment.

The first enablement discovers only sources after the enablement time and never backfills history. Later runs use a persisted discovery watermark with a 48-hour overlap window; after downtime longer than the window they look back from the last watermark, and pending items already found are never lost when they leave the window across days. A pending target and its backoff are saved in Postgres before it is queued; failures back off from one minute to at most one hour, and a success deletes the target. After five failed runs the same version is skipped with a memory_session_trigger_skipped ERROR log, so it no longer holds the subject's lane; a newer message or recording version triggers it again, and a skipped target is dropped once it leaves the discovery window. Because the scan runs once a minute, a Room is summarized between 10 and 11 minutes after its last message (plus queueing), and a ready recording within a minute. A restart restores pending items; after a Session is written, the subject's Wiki downstream recovery marker is saved before the checkpoint is committed. When a targeted source finishes later than the midnight Daily of its date, the same subject's Daily → Wiki → Graph for that date runs again, and the recovery marker is cleared only after the whole chain succeeds, without updating other subjects. Per-person lanes hold only within one process and are no distributed multi-process lock; each data root still needs exactly one Scheduler process.

The calendar tasks run each AnotherMe isolated and in parallel, at most 4 at once. A slot is taken only when the subject's lane is free, so a subject still busy with its own earlier work does not hold one:

  • Hourly wiki: lists every authorized AnotherMe (a deployment usually has one) and runs each in isolation. The chain is Session Raw staging → indexing that Raw. Only the 00:00 run that closes the previous day, and a meeting's recording, then ask TeamHub for a Wiki refresh; the other hourly runs and a quiet Room only index (--index-only). At midnight each subject checks all changed sources, even if its conversations were already indexed or only My Pages changed. The nightly refresh is not awaited, so the Daily does not wait for the Wiki. A failure of one person leaves only that person's Wiki downstream pending and logs memory_subject_refresh_failed; it does not mark the whole memory_schedule_run failed or interrupt others. Only when every AnotherMe fails is the run failed, retried hourly. Each (memory subject, Room) saves the last successfully handled Message in evidence/<date>/sessions/.checkpoints/<roomId>.json. The midnight run handles the previous day, catching up the messages after the last read and the pending Wiki updates; from 01:00 runs handle the new day. Daytime runs without new Room messages skip; whether the night generates is decided by TeamHub from changes across all sources. An explicitly named Room that does not match, a database failure and an attachment read failure still report errors. Raw is restaged only when new messages appear after that position; the input always holds the Session's full Room message history, and OpenWiki keeps old facts from the existing safe Wiki pages. Staged files live at evidence/<date>/sessions/<roomId>.md; the checkpoint advances only after the Raw is saved and the downstream pending marker is on disk. Imported messages are converted to business time and reordered by instant and Message ID, using the same comparison as the checkpoint, so several messages in the same second are not generated twice because of their physical import order. The checkpoint file format is unchanged and history need not be cleared. The Wiki generator (src/llm-wiki/agent-wiki-backend.ts) writes Client, Contact, Project and Topic pages from the Raw; only meetings get Session pages. The Scheduler and the source triggers share one Raw collection. The Wiki downstream pending marker lives at updates/task-state/wiki-pending/<date>/<subjectKey>.json. Graph building checks historical Memory/Wiki concept pages against OKF: a page missing type or a path's required fields, or with broken YAML, is skipped and recorded in memory_graph_skip / memory_schedule_run.graphSkips. Historical Client/Contact/Project pages with only invalid_status or invalid_stale_after still enter the graph. Source files are never deleted. The hourly task can still succeed, but memory_graph_skip, memory_subject_refresh_failed and a memory_schedule_run with skips or failures all log "level":"ERROR" so log collection raises an alert; other AnotherMes in the same calendar refresh are not marked failed because of it. New writes still fail closed as a whole batch, and unqualified model output never reaches disk.
  • Daily at 00:00 (daily): per AnotherMe, in parallel, after the closing Raw staging and without waiting for the Wiki. The day read (day-read.ts, one memory Agent task) reads the day's raw conversations once and writes the memory.md edits, then timeline/daily/<date>.md, which is indexed beside the Wiki pages as a period_summary source, never as Raw. Daily keeps no long-lived entities and builds no graph. After the day read, the same person's prompts/tracking/tracking-items.md updates that Human's Tracking Items (memory_index.tracking_items). The input is the full text of every Session that day (Pocket Notes included) and the existing items; the model returns only creations and deletions, and items are never edited in place. It keeps only dated arrangements from meetings and the Human's own words (appointments, classes, trips, bookings), promises between people, unanswered questions, things blocked on outside conditions and things the Human explicitly asked to track; ordinary tasks given to Agents, finished items and Agents' requests and suggestions are left out. The call always uses medium reasoning effort, and its input carries the program-computed dates and weekdays of the next 14 days plus the end dates of this week, next week and this month; a relative period always resolves to its last day. When the day's Sessions exceed 100,000 characters in total, they are read in parts (a very large single Session is split by lines), each part seeing the items left by the previous one, and only the net result is written in one transaction at the end. An item with due_at is removed once past; an item without a time is kept three months (deleting and recreating restarts the clock), and the list endpoint filters expired items too. If any check fails (source numbers, deletion ownership, 200-character content, ISO times or the 100-items-per-person cap), nothing is written; a failure of this step fails that person's Daily for the day, which retries as a calendar task, and the Daily output is rewritten unchanged. Each person runs independently: scheduling isolates people and retries only those who failed; a manual daily without a subject keeps going after one person fails and reports the errors at the end.
  • The day read maintains memory.md in the same task as the Daily, before it: the day's raw conversations go to the model, which returns only add, replace and delete edits (a day too large for one task falls back to separate calls that read it in parts above 100,000 characters, with the Daily); an edit drawn from a Session must quote the Human's words found in that Session. The checked edits become one new TeamHub user_memory_revisions version for the day, projected to the read-only me/memory.md. Days without Sessions skip the step. Each written Weekly updates it the same way, and TeamHub calls POST /internal/v1/profile-memory-syncs about a minute after each Profile publication, merging the newer Revision against memory.md's base_profile_revision_id (the first one builds memory.md); a sync that keeps failing leaves the last good version. Apart from those Profile syncs, memory.md has no daytime writes. Rollback: pnpm --filter @linger/node-services memory:user-memory:rollback <memory-subject-key> <version>.
  • Weekly on its set day (weekly): per AnotherMe, isolated and in parallel, each Subject with its own collection and prompt. It still waits for the same midnight's Daily task to succeed (when every AnotherMe fails, Daily still blocks Weekly). One person's Weekly failure does not interrupt others; when a person finishes, timeline/weekly/<weekStart>.md is published atomically as PostgreSQL Raw and Chunks, with no Wiki Revision or graph update. A week without useful information calls no model, writes no empty report and publishes no index; an existing report stays unchanged. Last week's content never stands in for this week's activity. A missing Daily for a day with conversations fails and retries; it never counts as no information. Intermediate products live under updates/weekly/<weekStart>/<memorySubjectKey>/.

Sync write boundary

Raw files, the Wiki projection and Graph files of one Human/Agent share a workspace file lock. Writes queue in process first, and separate CLI or service processes exclude each other through the same proper-lockfile protocol; the lock is renewed continuously and a lock left by a crash can be recovered after it expires. A write that loses lock ownership fails closed and never keeps writing alongside the new holder. Atomic JSON and directory replacement still bound each individual write.

The Wiki sync sends TeamHub no Session content: after it indexes the subject's Session Raw it calls POST /v0/personal-wikis/{actorId}/memory-refresh with the date, and for a meeting (a recording run, --capture-id) the recording's capture_id. TeamHub reads the Sessions from memory_index and keys the refresh intent by the Session set it saw, so a retry of the same Sessions replays the same refresh. A meeting is not admitted while another refresh runs (in_progress): the sync waits until the Wiki is no longer refreshing and sends it again, at most three times, then logs memory_meeting_wiki_refresh_skipped (ERROR) and leaves the meeting to the night's refresh. No run waits for a publication: Daily, memory.md and Tracking Items read the raw conversations.

The Daily and Weekly file scope is enforced by apps/node-services/src/memory/sync-policy.ts, and the prompt shows the same boundary:

  • Daily reads the day's raw conversations, Pocket Notes and yesterday's Daily when available, and the day read maintains memory.md at the same time. The report is written to timeline/daily/<date>.md and published as PostgreSQL Raw.
  • Weekly reads the Dailies of seven specific dates and last week's user dynamic, writes only timeline/weekly/<weekStart>.md and publishes PostgreSQL Raw.
  • Neither stage may delete or generate the old User, Project, Client or Workflow files. Each batch takes a snapshot before writing; a failure midway restores the changed paths, so no half batch remains.

These steps call Memory module functions directly, never a shell or pnpm child process.

Memory update generation runs through src/runtime-model-call.ts on the Computer of that Human's Another Me, with Codex's or Claude's default model. The day read uses a step-limited Agent loop that validates the profile edits and the Daily separately. Tracking and Dreaming use structured model output. Session facts take at most 16 windows per call and return each window's facts by window_sha256. The personal knowledge query itself calls no second answering model; it only does deterministic search and Evidence packaging, and the final answer comes from the Agent Runtime that made the tool call. LLM Wiki generates candidate pages step by step through the same Runtime; see src/llm-wiki/AGENTS.md.

Manual commands

pnpm --filter @linger/node-services memory:daily -- --date 2026-09-01
pnpm --filter @linger/node-services memory:weekly -- --week-start 2026-08-31
pnpm --filter @linger/node-services memory:wiki-sync -- --date 2026-09-01

The :collect, :prompt and :apply actions of memory:daily and memory:weekly remain for manual inspection.

wiki-sync indexes the Session Raw, requests the refresh and finishes once TeamHub returns its refresh_id; the result records accepted / in_progress and no longer waits for the new Revision. TeamHub queues, generates and safely publishes in the background; the persistent outbox in the publication transaction then commands Memory to materialize the latest Wiki and update the graph. already_latest can still materialize the current safe Revision directly, and no_sources triggers no generation.

PostgreSQL tests

tests/memory/*.postgres.test.ts, chunk-index-sql.test.ts and session-source-sql.test.ts are skipped by default and run only when these variables are set, and only against a dedicated localhost database:

  • MEMORY_SOURCE_TEST_DATABASE_URL: the database name must be memory_source_test. The tests clear its Tenants and Accounts.
  • MEMORY_CHUNK_TEST_DATABASE_URL: the database name must be memory_chunk_test.
  • MEMORY_WIKI_TEST_DATABASE_URL: the database name must match memory_wiki_test_<hex>.

Initialize each database with the TeamHub migrations first (in apps/teamhub, run TEAMHUB_DATABASE_URL=<database URL> go run ./cmd/migrate). Tests that share a database clear each other's tables, so run them serially:

cd apps/node-services
pnpm exec vitest run --no-file-parallelism tests/memory/*.postgres.test.ts tests/memory/chunk-index-sql.test.ts tests/memory/session-source-sql.test.ts

Failure isolation between refresh and indexing

Wiki publication and Graph revisions do not depend on Chunk indexing succeeding. Raw is indexed on Memory's own schedule, never from a publication, so the hourly index fills any gap. syncAll keeps handling the other Agents after one fails and still marks the whole run failed at the end. When the generation stage finds the PostgreSQL original text missing or unavailable, it falls back only to local Raw whose current manifest and hash check pass.

Room Raw attaches only to Session pages whose URI and business date both match, so different dates never mix under one provenance. When the cross-process lock is lost, the current operation and its later file publications fail in isolation while the service process and the later queue survive. An older hourly retry of the same date can be superseded by a newer scan, but the closing dependencies of Daily/Weekly must still complete.

Failure visibility and manual recovery

The two kinds of automatic retry each have a final or persistent failure state, shown by one read-only endpoint (skips of source triggers are recorded only in logs):

  • Calendar tasks (Session/Daily/Weekly) retry only the failed subjects, after 15 s, 1 min, 3 min, 10 min and 1 h; the sixth failure moves the task to the calendar's failed list. Dependencies are isolated per subject: subjects still failing in the prerequisite are held in Daily/Weekly while the others run; when the prerequisite is exhausted, only those subjects' downstream tasks stop with Dependency exhausted and an ERROR memory_calendar_dependency_exhausted log. A failure that cannot be attributed to subjects still blocks the whole downstream task.
  • A failed Raw Chunk source is recorded as state='failed' (including documents never stored that have only a normalization failure record); the hourly reconciliation and start-up keep retrying it automatically, with no manual action.

Wiki indexing has no Memory queue: TeamHub's publication job retries until Memory indexes the published Wiki, and the Memory scheduler reconciles Daily and Weekly reports every minute.

GET /internal/v1/health/failures returns scheduler (exhausted, pending_retries and their totals), raw_chunk_sources (failed sources and counts per owner), and session_facts (latest failed fact sync per Agent). Each list holds at most 200 items, error summaries are cut to 300 characters, and no text is included. An unconfigured store returns {"configured": false}, a failed query returns {"error": "unavailable"}, and the other parts still return 200.

Session fact diagnostics are process-local (scope: "process", observed_since) and reset on restart; zero means no observed failed syncs, not that all persisted sources have facts. Each failed Agent reports failed_windows, last_failure_at, and last_error_kind from its latest finished sync. A sync that aborts before a complete count reports failed_windows: null. Only a successful sync for the same Agent clears its failure; partial extraction keeps it visible. Existing sync triggers retry the unfinished windows. The memory.session_fact_extraction_failed log includes error_kind, agent_id, tenant_id, owner_id, and the batch's windows count. Error kinds use structured model-call errors (or unknown), never text matching; these added fields contain no source text or model output.

POST /internal/v1/health/failures:retry accepts only scheduler.task_keys (all tasks when omitted); any other field returns 400. Calendar tasks and the dependents they stopped go back to pending with zero attempts and the request wakes the scheduler, which runs them within about a second; the request itself does not run them. The response holds only the requested parts and their counts.

# The service token derives from LINGER_SECRET_SEED with the node-services-internal label.
MEMORY_TOKEN="$(node -e 'const c=require("node:crypto");process.stdout.write(Buffer.from(c.hkdfSync("sha256",process.env.LINGER_SECRET_SEED,"simplexai/service-credential/v1","node-services-internal",32)).toString("base64url"))')"
curl -s -H "Authorization: Bearer $MEMORY_TOKEN" \
  http://127.0.0.1:4319/internal/v1/health/failures
curl -s -X POST -H "Authorization: Bearer $MEMORY_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"scheduler":{"task_keys":["daily:2026-09-29"]}}' \
  http://127.0.0.1:4319/internal/v1/health/failures:retry

Production history rebuild

Use this to regenerate one Human's Daily and Weekly with the current pipeline from original data such as Room messages, recordings and My Pages documents, and to bring their Personal Wiki up to date with those Sessions. The rebuild never regenerates a published Wiki: its refreshes are ordinary incremental updates that send only the sources the Wiki has not published, so pages already written stay as they are. A Human without a published Wiki gets one initialized by the first step. Start the rebuild inside the Node Services Pod:

node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js start --owner <Human Actor ID>
# Pilot: replay only the Sessions of the last few days
node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js start --owner <Human Actor ID> --from 2026-09-23
node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js status --owner <Human Actor ID>

--owner without a matching Another Me returns 404 (each Human has at most one); with a known Subject use --subject <memory_subject_key> instead. The CLI derives the service token from LINGER_SECRET_SEED in the Pod environment, reads the port from NODE_SERVICES_HTTP_ADDRESS (4319 by default), calls POST / GET /internal/v1/rebuild on the local 127.0.0.1 and prints JSON. The production calendar must be running, or it returns 409.

On start the service asks TeamHub for the dates on which the Subject actually has Sessions: the conversation message dates of the Rooms the Human or its Another Me is in, and the dates of finished recording transcripts, all by local business date, matching Session collection. The range runs from the earliest Session date to today in the local time zone; --from only cuts earlier dates, and the end date is always today. The chain below is then queued, each step depending on the previous one:

  1. init:rebuild:<subject>: requests one Wiki refresh over the current sources (My Pages, Connected App files and the Sessions the index already holds), waits for the publication to catch up with them, and repeats until TeamHub returns already_latest (no_sources when there is nothing).
  2. index-initial:rebuild:<subject>: index once, running the Another Me's full source sync and indexing the published Wiki (wiki sources) directly; the step finishes when both calls return.
  3. Replay only the dates with Sessions, in time order: wiki:rebuild:<subject>:<date> (indexes the logical date's Session Raw and requests a Wiki refresh; it finishes once TeamHub accepts the request and does not wait for publication; when a refresh is already running, TeamHub folds these Sessions into that refresh's follow-up) → that day's daily:rebuild:…, which reads the date's Session Raw rather than the Wiki (skipped for today, which the midnight calendar finishes). After the last Session date of a week (week_start is Monday), if that week has ended, weekly:rebuild:<subject>:<Monday> runs; the current unfinished week is skipped. In this phase only the Wiki sync indexes the replayed Session Raw, because the Wiki reads Sessions from the index.
  4. index-final:rebuild:<subject>: run the source sync once more and index the wiki pages and the daily and weekly reports (period_summary sources).
  • The first rebuild of a Subject moves its local derived state (<subject>/memory and that Subject's dated products under updates/) in the Subject's lane to <subject>/memory.pre-rebuild-<time>, and deletes that owner's wiki and period_summary sources with their chunks from memory_index.sources (WikiIndex.clearOwner); other owners are untouched. The original-source rows are rebuilt from the original data and are kept. Each Subject keeps only its latest archive; older ones are removed by the daily file cleanup.
  • Index hold: while the Subject has pending rebuild steps, the publication job's Wiki and source indexing, the report indexing after Daily/Weekly, the scheduler's minute-long report reconciliation and the startup and hourly source syncs all skip that Another Me (log memory_rebuild_index_hold_skipped); other users are not affected. The hold is derived from the pending rebuild steps in the saved calendar, so it survives restarts; it ends when step 4 completes or the chain is exhausted (log memory_rebuild_index_hold_released, outcome completed or failed).
  • Rebuild steps occupy the Subject's lane: they never run alongside that Subject's calendar or Session trigger tasks, and other Subjects' tasks run in parallel as usual. Each scheduler check still queues due calendar tasks before rebuild steps.
  • Concurrency: one rebuild step runs at a time, and several Humans' rebuilds run one by one in queue order. This affects only rebuild steps, not the concurrency, order or priority of calendar tasks. A rebuild's Wiki receipt is written to updates/wiki-sync/<date>/<subject>/result.json, while the calendar still writes updates/wiki-sync/<date>/result.json, so concurrent rebuilds of the same date never overwrite each other.
  • Calling start again while rebuild steps exist returns the existing steps and adds no dates. To start over, add --restart: it forgets the Subject's rebuild steps, rediscovers the dates, archives and clears the owner's wiki and period_summary index; with steps still pending it returns 409. It leaves the published Wiki, the indexed Sessions and User Memory as they are.
  • status returns phase (init, index-initial, sessions, index-final, completed, failed), sessions.completed / sessions.total, current_task, current_date (the date in the Session phase), index_hold, from / to (the first and last Session dates), total, completed, pending, failed, running and a failure summary. completed means every step ran; the Wiki refreshes the date steps requested may still be generating while the Human's Personal Wiki reports refreshing. A failed step uses the calendar's 1, 2, 4 and 8 hour backoff; after the fifth failure that step and everything after it stop with Dependency exhausted and appear in GET /internal/v1/health/failures; each step run writes one memory_schedule_run log. After fixing the cause, pass the failed task keys (such as index-initial:rebuild:<subject>, or none to resume everything) to POST /internal/v1/health/failures:retry to continue from the break point, and the index hold takes effect again. A restart continues from the saved calendar.
  • Cost: each Session date requests one Wiki refresh and calls the Daily model once, and each finished week with Sessions calls Weekly once; a year of history means hundreds of model calls, taking hours to days. Replay the last 7 days with --from first, then run the whole chain with --restart.
  • During a rebuild the normal calendar still handles that Subject's current day, so today's Sessions may publish before history; they are replayed again in order later.

Wiki pages and reports in the one index

The published Wiki pages and the Daily/Weekly reports are sources of the same memory_index.sources and chunks as every other source; there are no wiki_* tables and no separate search.

  • A Wiki page is a wiki source (source_kind='wiki_page') of each of the Human's Another Mes, holding the current published Markdown with page_key, document_id, document_version_id and published_at in its metadata. Search chunks and reads only the page body without its inline citation labels ([R1], [R1, R2], (R2)), with the frontmatter's description and aliases under the title and the rest of the frontmatter (sources, relations, item IDs) left out; chunk offsets and lines refer to that text, while the stored version keeps the published Markdown for the graph and flashback. Navigation pages (index.md, readme.md, log.md, instructions.md) and empty pages are stored for the graph without chunks, so the search never returns them. There is no version history.
  • TeamHub's publication job calls Memory for each published Wiki (wiki-index.ts). Memory reads the published page list, downloads only the pages whose Document Version the index does not hold, checks each page's SHA and size, ingests each page through the chunk index, and deletes the wiki sources the publication no longer has. An API failure or an incomplete download leaves the current index; the job retries.
  • A report is a period_summary source (source_kind='period_report', metadata collection and period_start) built from timeline/daily and timeline/weekly; it is indexed when it is written and by the scheduler's minute-long reconciliation, which also handles edits and deletions. An unreadable directory keeps the indexed reports.
  • readIndexedWiki and indexedWikiRevision read the wiki sources for flashback, the knowledge graph and the meeting summary; the revision is a digest of the indexed page versions.
  • A failed index does not roll back the Wiki or report generation.
  • Session Raw is indexed as room and recording sources, and those are the Wiki's Session sources: TeamHub reads them from this index (wiki_session_sources), freezes each at its source_versions SHA-256, and the generator gets that text with its routing. There is no evidence Document. Session pages cite the Room messages their facts come from, and a recording its transcript Document Version.

All sources share one keyword ranking: tokenizeSearchText writes the weighted tsvector (heading A, body and title B, path C), GIN with ts_rank_cd recalls, the lexical rerank (search/lexical-rerank.ts) orders by query-term coverage, and the query, each alternates wording and the time window vote by RRF (k=60).

Configuration and deployment

The index tables are created by TeamHub migrations; the Memory role requires TEAMHUB_DATABASE_URL and does not start without it. No PostgreSQL extension is needed.

Index timing

Each successful Wiki publication (init included) makes TeamHub's publication job ask Memory to index it; Daily/Weekly are indexed after they are generated. Generation does not wait for indexing; one collection is indexed at a time per process, and a database advisory lock serializes processes.

Historical Wiki and Daily/Weekly rebuilds use the separate entry in "Production history rebuild".