Linger is An Open-Source Personal Agent That Runs Fully on Your PC. We aim to break Big Tech’s grip on AI by bringing powerful local AI to everyday computers through efficient model–harness co-design, so everyone can run their own.
Memory service
Memory is the memory role of apps/node-services; its code lives in src/memory/. It is an internal service of the unified Wiki, not a standalone knowledge product: it serves HTTP and runs the scheduled sync in process. It maintains the Agent's experiential
Memory and the derived subject graph. After each publication, Memory projects the Wiki pages as readable files.
The unified Wiki's wiki scheduled task collects and stages the full Session Raw and advances the source cursors;
one LLM Wiki update writes the Session pages and the entity pages that Session supports, without a separate
Session summary model first. Daily and Weekly use their own period report runner: Daily writes timeline/daily/,
Weekly writes timeline/weekly/. Each generated report's full Markdown is written to PostgreSQL as a Raw source,
and its Chunks come from that same version; reports submit no Wiki refresh and are not OpenWiki nodes. The prompt,
output schema and write validation use the same scope. The file system still projects timeline/daily/ and
timeline/weekly/; the current version in PostgreSQL is the query authority. Period reports are read through the
Memory browsing entry and do not enter OpenWiki's native graph.
Search
Memory keeps one index, memory_index.sources and chunks: Room Sessions, recordings, My pages, imported
mail, Pocket Notes, the Profile, the published Wiki pages (wiki) and the Daily and Weekly reports
(period_summary) are source kinds of it. Chunks and Session Facts use keyword search only (PostgreSQL simple
tsvector plus CJK bigrams, ranked with ts_rank_cd and then lexically reranked); no vector or rerank model is
called and there is no vector column. Wiki related pages have no "similar" edges, and flashback matches only names
and aliases.
File layout
<AGENT_HOME>/agent-manager/workspaces/context-host/<agent_management_id>/
memory/
README.md
me/{memory,profile}.md
timeline/{daily,weekly}/<date>.md
timeline/meetings/YYYY-MM-DD HH-MM-SS.md
knowledge/{people,clients,projects,topics}/<title>.md
memory-state/
evidence/<date>/sessions/ # raw material and collection checkpoints
raw/ # index snapshots
AnotherMe mounts the personal directory read-only. Meeting and knowledge pages are projected as readable files
after the Wiki publishes; links follow the projected paths, and formal Document IDs and Wiki page keys stay stable.
Each Scheduler run and each LLM Wiki generation writes only structured logs (memory_schedule_run,
wiki.generation_phase); no run list is kept in memory or on disk.
The source trigger's pending targets and its discovery cursor live in Postgres
(memory_index.session_trigger_targets and memory_index.scheduler_state).
Session checkpoints and pending Wiki downstream markers are persisted too; they are not a run history.
The Memory role applies file retention at start and then once a day: the dated run records of sessionly, daily,
weekly and wiki-sync under updates/ are deleted after 7 days without changes, except that a day's Sessionly
records stay while that day's Wiki sync is still pending; each Agent keeps only its latest memory.pre-rebuild-*.
Apply writes no audit log and there is no manual refresh; old logs, indexes and manual refresh receipts under a
subject's memory/updates/ are deleted as well. The same pass deletes the retired scheduler files (updates/wiki-calendar.json* and updates/task-state/session-triggers/), whose state now lives in Postgres. updates/task-state/wiki-pending/ and the lock directories are not pruned.
memory/me/profile.md projects only the current Revision of TeamHub's unified private Profile. The page Profile and
AnotherMe use the same user_profile.md; it is generated by Settings → Add source or AnotherMe's personal-profile
Skill, merging the old body with new sources when a body exists. Generation uses TeamHub's Profile model
configuration with no second Soul call; the weekly step does not rewrite the profile.
The Profile does not enter AnotherMe's prompt; memory/me/memory.md does: built from the Profile and kept current by the Daily and Weekly steps, it is the read-only projection of TeamHub user_memory_revisions.
Memory does no profile inference and does not rewrite identical content; when the authoritative profile is missing
it restores the draft projection and clears the old stable body. This file is not a Wiki source and does not enter
the Subject Graph. The Chunk sync separately reads the current Profile Revision through
memory_source.another_me_subjects and writes it into memory_index as a profile Raw
(teamhub://personal-profile/revisions/<id>) for knowledge search. Each AnotherMe keeps one Profile source, replaced
when a new Revision is generated, and the Profile takes no part in Session fact extraction.
Both Markdown knowledge trees are read as OKF v0.2. The root index.md declares okf_version: "0.2"; an ordinary
concept page must start with parseable YAML frontmatter and a non-empty type. The optional generated, verified,
sources, status and stale_after must satisfy OKF v0.2 when present. Top-level status uses only draft,
stable and deprecated; business states of Sessions, Projects and Clients use extension fields such as
session_status, project_status and relationship_status. When reading or graphing existing Session Memory whose
top-level status still holds a lifecycle value (such as summarized, in_progress or waiting_for_user), Memory
records the missing session_status in memory and treats the document status as stable; it neither rewrites the
source file nor relaxes new writes.
Before writing a batch to disk, Memory checks OKF and the required fields of each content type; the content validator owns these checks and loads no graph schema. If any file fails, nothing in the batch is written.
Daily and Weekly models return one JSON object of report fields (title,
description, source_refs, one array per section, Weekly work_intensity)
under a strict JSON schema; they never write Markdown, paths or headings. The
application renders the whole file: YAML frontmatter, the fixed English H2
headings (DAILY_REPORT_SECTIONS / WEEKLY_REPORT_SECTIONS in
@linger/knowledge-format/sections), one bullet per item, None observed.
for an empty section, the fixed no-baseline sentence when there is no
yesterday Daily or previous Weekly, and a final ## Sources section. Daily
writes only the day's report and Weekly only the week's user dynamic; neither
deletes files. A candidate the schema or the renderer rejects is corrected
within the memory kind's three attempts; an invalid candidate never reaches
Memory. A manual apply validates the saved
model JSON against the same schema and renders it without calling the model.
Models select evidence only through source_refs, the short refs of this
collection, and cite inside item text as [[source:<source_ref>]]. The
application renders source titles, YAML sources and Markdown links. Daily
cites the day's raw conversations, one source_ref per Room; Weekly cites the
week's Daily reports and marks the previous week only as a comparison
baseline. Raw internal paths or URIs in model text, unselected or unknown
refs, duplicates and malformed markers are rejected. Weekly runs only when a
current-week Daily has a list item in an activity section (Daily Work,
Resolved Items, Unresolved Items, Tomorrow Focus or Pattern), and its output
must select at least one Daily.
Daily reads the day's raw conversations, yesterday's Daily and the Pocket Note; it never waits for the
Wiki, since Room chats have no Session pages. Tracking Items read the same raw text and Pocket Note.
Client, Contact, Project and Topic pages are maintained only by the Wiki generator; Memory keeps no
entity files.
The shared OKF layer keeps valid YAML and fixes only a zero-indented root mapping field that the parser locates exactly. When the nesting cannot be confirmed, or there is another syntax error or a conflict, the candidate is still rejected.
TeamHub data access
Memory connects to TeamHub's database through TEAMHUB_DATABASE_URL, but reads only the read-only memory_source
views and functions that TeamHub migrations define, and writes only memory_index. The role has one connection pool
(src/memory/database.ts, at most 10 connections): indexing, collection, queries and command lines all use it, and
it closes when the role stops or the command line ends. tests/memory/database-boundary.test.ts ensures that
Memory's SQL references only these two schemas and that the pool is created only there.
another_me_subjects: the one definition of an AnotherMe subject, anactiveAnotherMe and the Human it represents.subject_rooms,conversation_messages,message_attachments: the Rooms a subject may remember (the AnotherMe or the Human is a listed member and the Room is not deleted), with their conversation messages and attachments (themessage_attachmentsview expands themessages.attachmentslist).subject_recordings: the finished text transcripts in those Rooms. Private questions and task requests go only to the recorder's own AnotherMe.finalized_meetingsandhuman_can_read_document(): a meeting is listed while both its recording and transcript Documents are active;hidden_meetingsholds the meetings the Space's Human removed (hid) from Meetings.profile_revision_dates: when each Profile Revision was written, used as the Profile Raw's date.pocket_notesandpersonal_documents(): Pocket Notes (without deleted dates), and the current personal documents in My Pages and in the mail-import Wiki sources; a native page gives the Markdown Export Artifact of its current Revision.
When TeamHub changes a table these views depend on, update the views in the same migration;
memory_source_views_test.go checks the authorization rules in the views.
Starting
NODE_SERVICES_ROLES=memory pnpm --filter @linger/node-services dev
The root pnpm dev already starts Memory in the node-services process.
Core configuration:
AGENT_HOME: the Agent state directory,~/.mirrorby default. Memory writes to the Agent Context Host root<AGENT_HOME>/agent-manager/workspaces/context-host, the same directory the Agent role mounts; both come from the same function insrc/paths.ts.LINGER_SECRET_SEED: the service credential seed; Memory refuses to start without it. The credential derived from it with thenode-services-internallabel authenticates internal clients calling Memory (shared with TeamHub's other calls to Node Services): today MCP Server queries, development LLM Wiki generation log reports, TeamHub's Wiki graph update commands and the rebuild CLI. The Agent Service's flashback calls Memory directly in the same process, not over HTTP. The credential derived with theteamhub-internallabel authenticates Memory's calls to TeamHub (shared with Node Services' other calls to TeamHub).TEAMHUB_BASE_URL: reads the latest published Wiki and requests a Wiki refresh after Sessions are indexed.- Business dates use the process's local time zone (
TZ): the collection window, Session dates, meeting directories, Pocket Note reads and calendar scheduling all use it, and there is no separate time zone variable. Compose containers default to UTC; setTZin.envwhen needed. - These features are always on, with no switch: the in-process scheduled sync (hourly Wiki, daily Daily and weekly
Weekly), the targeted refresh after a Room is quiet for 10 minutes, the separate Session refresh of formal
recording transcripts, Session fact extraction and fact recall, the nightly tracking-item pass, and the nightly
revision of a person's reminder guidelines from the day's proactive reminder feedback. Session facts need
TEAMHUB_DATABASE_URL. - Built-in constants (no longer environment variables): calendar tasks and their retries run at most 4 AnotherMes
at once (a deployment has one account, usually one AnotherMe), and each person's Room Sessions also run 4 at a
time;
weeklyalways runs on Monday; Memory model requests share a 10-minute business cancellation, response header and body read timeout, with no streaming; the Chunk reconciliation runs hourly, with at most 4 Sources indexed at once per Agent. TEAMHUB_DATABASE_URLis required: the Memory role keeps its scheduler state in that database and does not start without it. It also enables the Raw Chunk persistent index and search.- The full Raw Chunk reconciliation runs once at service start and then at the built-in interval.
- Session facts: after Raw Chunk sync finishes an Agent (without blocking indexing), Memory extracts Session facts
into
memory_index.session_facts, following Hindsight retain. Facts come from Room conversations, recording transcripts, Pocket Notes and imported mail; My pages, the Wiki, period reports and the Profile stay chunk-only. Mail is chosen by the- Class:line TeamHub writes into it:self_sent,direct_inboundandsystem_notificationare extracted, and so is every message of a thread with aself_sentmessage; marketing, subscriptions and drafts are not. Each window's Context line names its origin, so the one prompt knows a transcript may not label speakers, mail must tell the sender from the user, and a notification may hold no facts. A window's Event Date is its first Room message time, else its source date at local midnight; it is shown in the process time zone, so a message sent just after midnight belongs to the Human's own day rather than the UTC one. Consecutive Chunks form windows of at most 3,000 characters, extracted withapps/node-services/prompts/memory/facts/fact-extraction.md(the Hindsight concise prompt) at low reasoning effort. Up to 16 windows form one batch, and each batch runs as one ModelCall chain of kindmemory, keyedmemory.facts:<sha256 of the batch's sorted window hashes>so a restart asks the same key. Its calls run in order. Each model attempt queues separately, so chat can start before the next step, retry or correction. An answer still invalid after them is split into two halves, and windows whose facts drift into another language are extracted once more; both are steps of the same chain (first,split-<n>,<step>-drift). Each sync keeps at most two batches on the queue at once. Backfill write transactions are short and share Memory's connection pool with recall. Windows are recorded by content hash inmemory_index.session_fact_windows, so only new or changed windows are extracted and failed windows retry on the next sync; facts of unchanged windows are rebound to the Chunk with the same content in the new Raw version. - Fact recall: a Raw Chunk query opens a separate fact channel that recalls facts by keyword (the query's words
merged with
alternates); with an event time window, facts inside the window that match the keywords get a second vote, fused by RRF, and the response returns at most 30factsat the top level. Without keyword matches it returns none. Each fact carries its Raw'sevidenceIdandsectionIdfor a follow-up read. Facts take no part in the Chunk ranking and take at most a quarter of the response's output budget (below); with or without the fact channel, both lists are ranked the same. A summaries-only search omits facts; a failed fact recall only omitsfacts.
Memory's endpoints mount on the shared Node Services listener (127.0.0.1:4319 by default), all under the
/internal/v1/ prefix and protected by the workload token. The process answers GET /healthz, GET /startupz and
GET /readyz.
POST /internal/v1/published-wiki-graph-updates: called after TeamHub atomically publishes a new Wiki Revision; updates only the AnotherMe graph of that owner and skips an identical Revision.POST /internal/v1/pocket-note-tracking-refreshes: TeamHub calls it about five minutes after today's Pocket Note changes. It reads the note as it is then, bytenant_id + owner_actor_id + date, and updates that Human's Tracking Items from this page alone: open items from the page that it no longer mentions are deleted, the rest is the nightly comparison (add, update in place), and expired items are not cleaned up. Without an AnotherMe it answers204; a model or TeamHub failure answers503and TeamHub retries.- A meeting follow-up needs no endpoint: once the hourly Wiki or a recording's Session collection stages a formal
transcript under
evidence/<date>/sessions/recordings/, Tracking Items are updated in the background from that transcript alone (no cleanup, never holding the Wiki run); a failure is logged as ERROR and skipped, and that night's Daily catches up. POST /internal/v1/personal-knowledge-graph-visualization: protected by the workload token; reads the existing AnotherMe graph fortenant_id + owner_actor_idand returns embedded HTML without internal subject identity. The request is read-only and triggers no graph build; it returns404when there is no graph.POST /internal/v1/query: protected by the workload token; the one knowledge search over every source kind, returning two separately ranked lists (below) for the question, optionalalternates, optional absolute dates, optionalscope(originalsorsummaries, one list with the whole budget) and optionalpage.POST /internal/v1/personal-knowledge/evidence: protected by the workload token; reads at most 8 selected Chunks with an opaqueraw_Evidence ID returned by a query. The service revalidates the ID against the database's current Raw version and content hash and accepts no file paths.GET /internal/v1/health/failures,POST /internal/v1/health/failures:retry: protected by the workload token; a read-only failure summary and manual recovery, see "Failure visibility and manual recovery" below.POST /internal/v1/rebuild,GET /internal/v1/rebuild: protected by the workload token; rebuild one Another Me's Wiki, Daily and Weekly over its actual Session dates, see "Production history rebuild" below.
The MCP Server calls these endpoints after TeamHub authorizes the Context and the specific Tool. AnotherMe, General
and Builder Agents of one Human share that Human's knowledge in meta / alter_ego Contexts. TeamHub resolves the
hidden personal_knowledge_agent_id from the current registration's Human owner and Tenant; MCP passes it to Memory
as agent_id, while agent_actor_id keeps the real caller. The model cannot choose sources and cannot treat Room
members or recent speakers as knowledge owners; when unauthorized or a source is unavailable, Memory does not fall
back to another Agent's knowledge. Memory's Subject, Tenant, Human and Raw snapshot checks stay unchanged, as do the
graph and Raw generation flows. There is no cross-user Agent transfer flow, and the general MCP credential and
session revocation policy is unchanged.
The model has one search, teamhub knowledge search (query + alternates? + dateFrom/dateTo? + eventFrom/eventTo? + scope? + page?),
and one read, teamhub knowledge read (evidenceId + sectionRequests[{sectionId, offset?}]). Actor, Tenant,
Human owner, sources and file paths are set by trusted services, never by the model. For overviews the Agent
answers from the summaries list when it states what it needs; for details, exact words and counts it answers
from the items list of original sources. The search does not build the graph; the graph only serves the
Workspace visualization and flashback. Memory builds OpenWiki's native WikiGraph from the indexed pages of
the published Wiki when it is read: one node per page with a relative path ID, source/target edges only from
parseable Markdown links, plus the inferred mention edges in inferredEdges. Nothing is stored;
flashback keeps the graph of the indexed revision in memory. Without a published Wiki there is no graph, and
Memory files are never scanned as a fallback. No shared-source or root edges are built.
Old Sessions are not dropped from the graph after a seven- or fourteen-day window; date filtering is stated
explicitly by the query.
The visualization is rendered per request; no HTML is written to disk.
Raw takes the currently authorized version in PostgreSQL as the authority; the file system is only staging and
projection. Raw is checked by Tenant, Human, Agent and content hash; publishing Raw does not change the graph, and
original-text queries do not depend on the graph. memory-state/raw/ is the current original-text storage path, not
old graph data; cleaning old graphs must keep it and all Memory and Wiki bodies.
A Raw Chunk query deduplicates by immutable Raw source and returns at most 20 items. Three kinds of time can come
with a query, and none filters a source. The two windows are each one more recall channel (following Hindsight's
temporal window) that also requires the query keywords; as_of only reorders the fused list:
event_from/event_to, when the asked event happened, for an event that may have been told later: a fact votes when its event interval overlaps the window, or, with no interval (something said rather than an event), when its source date falls inside it; a mention in the window does not count for an event elsewhere, unlike Hindsight. A period report or a Session page is about its own day and votes by that date. A conversation, transcript, note or mail Chunk only knows when it was recorded, so it takes no event vote: its events reach the Agent through the facts list, as in Hindsight, where time is judged on facts only. In a benchmark of events told later, a vote through facts gave original Chunks no better rank than none.date_from/date_to, when the asked records were made (mail received in January, Monday's meeting): a Chunk or a fact votes when its source's local date falls inside the window.as_of, the day a changing value or state is asked for (today for the current one): Hindsight's recency boost without a cross-encoder. Each fused candidate's rank maps to a base score from 1.0 down to 0.1, multiplied by1 + 0.2 * (recency - 0.5); recency is 1.0 on that day and falls linearly to 0.1 sixty days before it (Hindsight uses a year; a changing value is restated within weeks), a source after that day takes the 0.1 floor, and an undated source is neutral (0.5). Close matches move toward the record nearest before the day; a weak match is never lifted over a strong one.
A window channel takes at most 60 by ts_rank_cd, then picks 10 round-robin over 8 time buckets so no single day
fills it. Each wording (the query and each alternate) runs its own time channels. A Wiki page other than a Session
page is dated by the Wiki's publication, not by what it is about, so no time channel takes it; a Session page
(sessions/<date>/<session>.md) is dated by its Session. TeamHub forwards the Agent's dateFrom/dateTo,
eventFrom/eventTo and asOf unchanged. None of them excludes a source: in LongMemEval the model writes the current
date into a time parameter almost every time, and a filter would exclude the earlier conversation that holds the
answer. For "what was done in some month" search by entity and action and verify the event time from the original
text, which lets later records recount past events. Relative event times resolve against the date of the source they
appear in, and an explicit historical cutoff still applies.
Raw and Wiki queries may also pass alternates: at most 6 synonymous wordings or translations, each at most 200
characters. Each wording runs its own keyword search in parallel with the query and votes as its own RRF list, so a
synonym match is not buried under documents that only repeat the query's original words; inside a time window
each wording votes again with the window's matches, ranked by distinct-term coverage like its keyword route. Raw's BM25 fallback scan and Session Fact keywords use the query's words merged
with all alternates. It adds no model call and no network round trip while the user waits.
One search returns two separately ranked lists, because keyword scores cannot compare a long Daily report
with a short chat message fairly: items holds original sources (room, recording, mail, my_pages,
pocket_note, profile, at most 20) and summaries holds the Wiki and Daily/Weekly reports (wiki,
period_summary, at most 10). Each list runs its own keyword search (60 Chunks), fusion and coverage rerank,
then keeps one entry per source. An entry has only citeAs (S<n> for items, W<n> for summaries),
evidenceId, kind, title, timeHint, sections (at most 4 matched and neighboring sections to read),
roomId for a Room conversation, and either content (text around its best match, keeping the time and
speaker heading of its message) or snippet (about 200 characters around the best match).
Memory keeps every search and read result within an output budget of JSON characters (UTF-16 units); the CLI passes results through unchanged. The Daemon limits one tool result to 15% of the model's context window, between 4,000 and 25,000 tokens, and to 12,000 tokens when the window is unknown. Codex counts four bytes per token and cuts the middle out of a longer result, so it keeps limit×4/3 characters of three-byte text such as CJK; Claude's Bash keeps the same number of characters and saves a longer result to a file. The budget is 90% of those characters for the model the owner's calls run on, leaving room for the CLI's labels; at no more than three bytes per character, a bounded result reaches the Agent whole on either Runtime. The window comes from the owner's Computer through the model call layer; an unknown window, or an offline Computer, gets the unknown-window limit.
The top entries of each list carry content: from a budget of 20,000 characters up, 5 items and 3 summaries
of about 1,500 characters; below it, 3 items of about 1,200 and 2 summaries of about 800. Session facts take
at most a quarter of the budget, the best-ranked first. Over budget, Memory drops in order: the lowest-ranked
entries without content (alternating lists), facts from the tail, content beyond the floors (3 items, 2
summaries), then shortens floor content down to 600 characters; floor entries are never dropped. Room left
over gives more entries content, items first. A scope search gives its one list the items' share.
- When entries, content or facts were left out, or more entries exist, the result carries the gap
results_trimmed;nextPagenames the page that continues the same search, laid out again from the first page so it starts exactly after the last shown entry. Status follows the gaps: entries with no gap aregrounded, entries with any gap (this one included) arepartial, and no entries isempty.groundedstill means only that the search matched, not that the evidence answers the question.
Raw query candidates, MCP read input and output, and Memory HTTP reads use one Evidence ID protocol: raw_ followed
by the first 16 characters of the source ID, an underscore and the first 16 characters of the content hash, naming
one original snapshot (^raw_[a-f0-9]{16}_[a-f0-9]{16}$); Wiki pages and reports use the same IDs. A read resolves
the prefix within the currently authorized owner scope and checks the current version; an ambiguous prefix or a
changed version returns not found. An entry's sections list only its matched chunks and their neighbors. The model must pass IDs back unchanged; it
cannot build, convert or replace them with file paths. The format is defined by the Memory OpenAPI, and the public
validation rule exported by the generated client is shared by MCP and Memory HTTP; a valid format does not grant
access, and reads still verify the trusted identity and the database's current snapshot. While relevant candidates
are unread, the Agent cannot conclude that information is missing; authorized cross-Room Summary text and Raw
original text are both usable, and the Room file projection boundary does not narrow what the personal knowledge
tools are authorized to read.
The Raw directory comes from deterministic splitting of the source text by
apps/node-services/src/memory/search/chunks.ts, and the second step reads only the Chunks the Agent explicitly
selects. The requested sections share the read budget left after the title, source and directory, in request order;
a section's unused share carries over to the next. A section that does not fit returns a non-empty first page with
truncated and nextOffset (a UTF-16 offset that never splits a surrogate pair), and reading on from nextOffset
continues exactly. Unknown Chunks go into missingSections. A read confirms again that the Evidence ID still names the currently authorized
source version in the database, so old IDs, retired sources and arbitrary caller paths cannot bypass the boundary.
With the persistent index enabled, Memory reconciles the current Raw source snapshots every hour by default; start-up
itself triggers no scan. The Chunk flow only reads sources and writes to the database; it never creates,
overwrites or restores original files or the memory-state/raw/ projection. The only deletion is the projection of a
deleted Room (see below). The immutable normalized Raw text and manifest live in PostgreSQL
memory_index.source_versions, one current version per source. Memory prepares the full chunks first, then in one
transaction saves the Raw version, replaces all of the source's old chunks, switches sources.active_sha256 and
deletes the old version; the Session Wiki update reads and checks the committed PostgreSQL version. Any failure in the
transaction rolls it back and the old version and keyword index stay usable.
For one Agent, the full sync, dated Raw sync and publication callback sync share a serial queue and a cross-process
lock; triggers with the same key that arrive during a run merge into one follow-up, which rereads the current
authoritative source, so an old trigger never writes back an old version. One Agent sync indexes at most 4 different
Sources in parallel (a built-in constant); different Agents run their own queues in parallel. An identical content
hash and full index profile skip the source, and queries always read the active version of the last complete
committed transaction.
memory-state/raw/ is maintained by the upstream collection flow for manual inspection and compatible tools; a
missing local file is never a reason to delete original text from the database, and a file whose content hash
differs cannot overwrite the current PostgreSQL version.
The original Version bytes of ordinary Documents, Connected App Documents and recording transcripts live only in
object storage; TeamHub's Wiki dispatch reads the object each time and checks its length and SHA-256, with no second
copy in PostgreSQL.
memory_index.source_versions holds the normalized canonical Raw shared by Chunks and the Session Wiki, with
PostgreSQL as the primary store; memory-state/raw/ is only a deterministic mapping and takes no part in choosing the
current version.
The reconciliation covers Sessionly Room messages (text, author, time and the text attachments read), the transcripts
of finished recordings, the current Document Version in My Pages, Wiki sources a Human uploaded or imported directly,
and Connected App content already materialized as the same Document; one Connector Document is not indexed twice
because it is also in My Pages. On the next reconciliation My Pages deletes sources no longer in TeamHub's source
list, cascading their versions, chunks and Session facts; a new version replaces only that source's chunks. The Daily/Weekly reports are derived
summaries, indexed as their own period_summary sources and never as Raw again. When a current safe Wiki page
references an exact historical Version of a Document, that Version enters Raw/Chunk ahead of the Document's current
head, so a Wiki reference never points at the uploaded original while the Chunk indexes a later export. My Pages
and imported mail enter only when the Wiki whitelist would admit them (TeamHub's source_rejection_reason is empty):
images, scanned PDFs, spreadsheets, data, log and code files, archives, and audio or video stay out of the index,
because the Wiki never records them.
Memory does not normalize Documents: it reads My pages and imported mail as the Markdown TeamHub stored in
document_texts (one River job per version), through memory_source.personal_documents and, for
Documents whose indexed hash differs, personal_document_markdown. A My pages Document is dated by its Version; an
imported mail message by the send time in its - Timestamp: header and named by its # subject line, because TeamHub
stores every message at the time of the import (the Version time and file name are used only when the header has no
readable time or subject). The normalizer TeamHub's job calls is
apps/node-services/src/document-normalizer, served by LLM Wiki in a restricted worker thread of this process. It
converts PDF, DOC/DOCX, PPTX, XLSX, ODT/ODS/ODP, EPUB, EML/MHTML, Office templates and SVG into canonical Markdown.
There is no OCR: a PDF gives only its text layer, so a scanned PDF has no text, and raster images are unsupported.
Conversion needs no network, Docling, LibreOffice, extra API key or separate service. After installing the
repository's locked dependencies, pnpm dev uses this path; CI and the production image carry the same parsers.
The index excludes generic ZIP/gzip, audio/video and legacy XLS/PPT through document_wiki_source_heads, without mistaking ZIP-based Office/EPUB for archives.
There are no backends for macro-enabled Office (PPTM/XLSM/XLTM), Box Notes, Outlook MSG, Apple Pages, AFP/EBCDIC or
DocLang. AsciiDoc and LaTeX uploaded as UTF-8 text/* can be read as text but have no dedicated parser; .adoc with
application/octet-stream and .tex with application/x-tex are refused. Complex PDF layouts need their own
acceptance.
A version the normalizer cannot read has no ready text in TeamHub and is simply not indexed; Memory keeps no failure record of its own, and the next version is indexed once its text is ready.
Markdown is split into Chunks by heading level and paragraph boundaries; consecutive timestamped turns merge into a
conversation Chunk with context, so one sentence never takes a whole result. Ordinary blocks do not overlap; only
very long blocks use a small overlap. Each record keeps the Raw relative path, heading path, one-based line numbers and
content hash. CJK bigrams and English or numeric whole words are generated in the application and written into a
field-weighted search_tsv; the GIN path ranks with ts_rank_cd. This is PostgreSQL FTS, not strict Okapi BM25.
At query time each channel (the query's lexical match, each alternates lexical match, the time window) takes its
top 50, merged by RRF with k=60; adjacent results from the same source are deduplicated, preferring three per
source before backfilling. The Chunk ranking may keep several matches from one source, but Personal Knowledge output
deduplicates by immutable Raw source and returns one evidence ID per source: the top Chunk is the preview, and the
full Chunk directory is offered for the second step.
When neither requested list has a valid chunk match, Memory runs a BM25 fallback over the text of the current PostgreSQL Raw versions within
the same Tenant, Human owner, Agent and date range (per list, at most 30 recent sources and 3 MB of text). It
reads no local files, never scans all Raw, and never revives a retired My Pages source from a local Document
projection. Fallback candidates can be verified by the same boundary's second-step Evidence read; a database query
error returns partial with raw_chunk_search_unavailable rather than posing as an empty result.
The index lives in the separate memory_index.sources / memory_index.chunks tables of TeamHub's PostgreSQL. The
source table also records JSON metadata such as source_kind (Room messages, recording transcript, Document, Pocket
Note pocket_note), origin_kind (Room, Recording, My Pages, Wiki source, Pocket Note pocket_note), version and
URI. Both candidate SQL queries always filter by the Tenant, Human owner and Personal Knowledge Agent the server
resolved. Each source keeps one active_sha256; a CAS on the hash read earlier, in the same transaction, deletes all
of the source's old Chunks and writes all new ones. A slow task therefore cannot overwrite another version committed
while it prepared, and other sources' chunks are unaffected. Old and new chunks are never spliced incrementally.
The skip condition checks the Raw, the index profile, the chunk count and the content version they belong to, so a
changed active hash is never mistaken for complete old chunks.
PostgreSQL sources are never cleaned up from a local directory listing. Sources are deleted only with their owner's
lifecycle: Documents follow TeamHub's authoritative list, with the scan start time protecting concurrent new
publications; Session and recording sources record room_id, and after a Room is deleted
(memory_source.deleted_rooms) the reconciliation deletes its local Raw projection and index while keeping the
Session summaries; after an AnotherMe is revoked (memory_source.revoked_another_me_agents) the full sync deletes its
sources (failure records go with the source rows).
Only a Raw snapshot saved successfully during sync can become searchable original text; without a snapshot the read
returns raw_source_not_found, and a summary page never stands in for the original. Raw manifests and snapshots carry
the content hash, source identity, anchored file and source version checks; when these fail, old Raw Evidence IDs stop
working. Raw evidence identity comes from the source and the content hash, is not written into native graph nodes,
and search does not go through the graph, so deleting old graphs does not change these evidence IDs.
My Pages and Wiki source documents read their Chunk content directly from TeamHub, build the manifest in memory and write it to PostgreSQL without a local snapshot; the manifest anchors the exact file (an uploaded file's Document Version, a native page's Export Artifact) and its content hash. These sources become Raw Chunk candidates once normalized and stored; candidate SQL always filters by the server-resolved Tenant, Human owner and Personal Knowledge Agent and cannot bypass authorization.
Session original snapshots are laid out by the Session's local business date, not by write time or the messages'
UTC date: memory-state/raw/<YYYY>/<MM>/<DD>/<roomId>/ holds the current manifest.json and Markdown named by content
hash. The raw collection input is still written before the summary to
updates/sessionly/<date>/<subjectKey>/<tenantId>/<roomId>/collection.json; the Raw search snapshot is saved after
the Session summary is written and holds message times, authors, text and the text of attachments read successfully,
without the Provider transcript. Each message heading is ## <local time with offset> · <author>, such as
## 2026-11-11T11:45:38+08:00 · Ada, in the process time zone, so Agents and fact extraction read the time the Web
client shows. Snapshots of earlier days keep the UTC headings they were written with; both forms carry an offset. A standalone recording replaces the <roomId> subpath with recordings/<captureId>;
its Raw is the full original transcript without uploader speaker labels. The formal audio is still managed by
TeamHub's object storage.
Raw Evidence identity comes from the source ID and content hash, independent of graph nodes or local directories. The database holds the currently authorized sources and version text, and file system directories are only staging and projection; choosing the current version, identity checks and text reads all use the database. Without a database source, local manifests are never enumerated to restore candidates, and a database error cannot pose as a successful file fallback. When no Chunk matches, BM25 may run over the Raw text in the database, never over the disk as a second authority. A returned Evidence ID is rechecked against the database's current version and hash when read.
Writing a Raw projection still needs atomic replacement and the workspace lock, and old files of the same source are
reclaimed only after the new projection is published; this produces or changes no graph. The memory-state/raw/
directory name refers only to the original-text projection, not to the deleted old graph format. Graph cache cleanup
must keep original text and version data; a lost projection does not affect sources in the database, and reads do not
rebuild projections.
The graph comes only from the indexed published Wiki; without one there is no graph. The visualization supports zoom, drag, search, native page-type filters and node details, and writes no HTML.
The first Raw Chunk step returns only candidates for the Agent to choose from; it produces no user answer and does not mean the text was read:
{
"schemaVersion": 2,
"status": "partial",
"items": [
{
"citeAs": "S1",
"evidenceId": "raw_3faaaaaaaaaaaaaa_7cbbbbbbbbbbbbbb",
"kind": "room",
"title": "2026-09-09 Hardware partnership",
"timeHint": "2026-09-09",
"content": "## 2026-09-09T02:10:00.000Z · Alice\n\nWe collect the hardware requirements; they confirm the delivery date.",
"sections": [{ "sectionId": "chunk-1a2b3c4d-0", "title": "Messages" }],
"roomId": "room_hw"
}
],
"summaries": [
{
"citeAs": "W1",
"evidenceId": "raw_8dcccccccccccccc_2eaaaaaaaaaaaaaa",
"kind": "wiki",
"title": "Hardware partnership",
"timeHint": "2026-09-10",
"snippet": "Open work: delivery date pending the partner's confirmation.",
"sections": [{ "sectionId": "chunk-5e6f7a8b-2", "title": "Open work" }]
}
],
"gaps": [{ "code": "results_trimmed" }],
"indexedAt": "2026-09-10T00:00:00.000Z",
"page": 1,
"nextPage": 2
}
The Agent then selects the Chunks it needs with teamhub knowledge read; a successful response:
{
"schemaVersion": 1,
"evidenceId": "raw_3faaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_7cbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"title": "September 9 hardware partnership call",
"source": {
"kind": "room",
"ref": "raw_3faaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa_7cbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"label": "September 9 hardware partnership call"
},
"timeHint": "2026-09-09",
"directory": [
{ "id": "chunk-1a2b3c4d-0", "title": "Action Items", "level": 2 }
],
"sections": [
{
"id": "chunk-1a2b3c4d-0",
"title": "Action Items",
"level": 2,
"content": "- Us: collect the hardware requirements.\n- Them: confirm the delivery date.",
"offset": 0,
"truncated": false
}
],
"missingSections": [],
"complete": true
}
source.ref is an opaque source reference, not a display field, and cannot be used as a file path. Only
sections[].content is text that has been read; the Agent answers the original question from it together with the
current developer instructions, Profile and Personality, and never sends candidate objects or source references to
the user as they are.
Scheduled tasks
Index timing
Both kinds of index are isolated by Tenant / Human owner, and Raw is also bound to the user's active AnotherMe:
- Raw indexing does not wait for a Wiki: every active Another Me in
memory_source.another_me_subjectsis a subject, including a Human who has only My Pages, Pocket Notes or a Profile. A Wiki publication never triggers Raw indexing. Imported mail is not placed in My Pages but is indexed as themailorigin. - Raw independently checks the Human's latest My Pages bodies, mail import sources and authorized Session text every hour; Session updates and ordinary Wiki refreshes do not trigger Raw again.
- A Wiki publication indexes only the Wiki pages.
- TeamHub's durable publication job calls Memory and retries until indexing succeeds; Memory keeps no queue or receipt of its own, and a repeated call indexes only pages whose Document Version it does not hold. An indexing failure never rolls back the published Wiki.
- Subjects come from the database, not from local directories. My pages are read from TeamHub's
document_texts, so no snapshot is prepared and no Wiki refresh is needed first; an old Wiki reference never overrides a file's latest version.
The one index (memory_index.sources and chunks, for every source kind including wiki and period_summary) is created by TeamHub migrations; deploy TeamHub before Memory. Runtime switches stay explicit configuration, and the root .env is never changed automatically.
The calendar saves its planning cursor, unfinished tasks and success receipts in one
memory_index.scheduler_state row in TeamHub's database, so the Memory role does not start without
TEAMHUB_DATABASE_URL. The first start checks from midnight of the current business day; after
downtime the saved cursor catches up the missed closed dates in Session → Daily → Weekly order.
Saved successes never rerun and failed tasks survive restarts. Missed ordinary hours are not
rescanned one by one; only closed dates and the current hour are collected. A calendar saved in
another time zone is ignored and planning starts over in the current one.
Memory has one in-process scheduler and one timer; there is no cron or extra background service. Each pass plans calendar work, admits rebuild steps and runs the periodic work that is due: the source trigger scan and the report reconciliation every minute, the Raw Chunk sync at start and then hourly, and file retention at start and then daily. After a pass the timer sleeps until the next local hour, the next due calendar or rebuild retry, or the next periodic work, and never longer than one minute. A task that settles, a queued rebuild and a manual retry wake it at once; passes stay at least one second apart. The source check reads only database metadata; it never reads message text or transcripts or calls a model on each pass, and it does not block the hourly tasks from being queued. Tasks run in one lane per person (memory subject): one person's Session, Wiki, Daily and rebuild run strictly in order, different people run in parallel, and one person being stuck (for example a meeting request waiting for a running Wiki refresh) does not block others. The calendar tasks for everyone (hourly Wiki, Daily, Weekly) still run in order, and each person's part enters that person's lane. At most four source triggers run at once. The two entries below run with the in-process Scheduler; an explicit replay plan does not enable them.
- Quiet Room trigger: checks the last activity by the
conversationmessages Memory already uses, notrooms.updated_at, and ignoresagent_runtimeandtask_terminal_notice. After 10 minutes without a new message it collects only the target subject, Room and business date's Session; when the item is dequeued it checks the quiet state again, and a new message across midnight restarts the timer. Queueing may run it later than 10 minutes. It shares the(created_at, message_id)checkpoint with the hourly task; after a success with no new messages it does not summarize again, and a new message restarts the timer without restarting the Room process. - Recording-ready trigger: each Capture maps to its own file
evidence/<date>/sessions/recordings/<captureId>.md; several recordings in one Room are never merged and do not wait for the Room to be quiet. OnlyfinalizedCaptures with a formal transcript Document Version are handled. Late finished sources are found by the Capture update time and the version creation time, deduplicated bydocumentVersionId + sha256, and a regenerated transcript updates the same Session. The file date is fixed to the local date when the first formal transcript Document was created; the actual recording time is kept in metadata. The Room keeps only permissions and file delivery; recording participants are not inferred, and its chat and run logs are not read. The text is read through TeamHub's authorized content API, never from local Artifact storage, and in-progress HLS, ASR staging and audio bytes are never read. A subject with standalone recording handling no longer puts the Capture transcript attachment into the Room Session as well. When the recorder is the subject's own Human, the same query also readsvoice_capture_questionsand renders them by recording time (the answer cutoff minusrequested_transcript_start_offset_ms) as two sections, "Questions asked during the meeting" and "Requests the recorder gave Another Me during the meeting" (Ask requests recognized as tasks go into the second, marked with whether they were handed to Another Me). They follow the transcript into the Session Evidence, Raw and Chunks; empty sections are omitted, and other members' subjects never read this private content. Recording notes are retired and are no longer a meeting summary source. The deduplication version is thendocumentVersionId:sha256:<first 16 characters of the private sections' SHA-256>, so a new question or a change in a task request's sending state restages the Session; new question creation times also count for discovery. The meeting page only requires the checkpoint to start with the current transcript'sdocumentVersionId:sha256.
One UTF-8 transcript is capped at 4,000,000 bytes and keeps the full original text and timecodes. The version checkpoint advances only after the Session Evidence and Raw are saved and the downstream pending marker is on disk. Exceeding the cap, a permission failure, a byte count or hash mismatch, or invalid UTF-8 never advances the checkpoint. One Raw is capped at 5,000,000 bytes; formal collection no longer calls the old summary model segment by segment.
The first enablement discovers only sources after the enablement time and never backfills history. Later runs use a
persisted discovery watermark with a 48-hour overlap window; after downtime longer than the window they look back from
the last watermark, and pending items already found are never lost when they leave the window across days.
A pending target and its backoff are saved in Postgres before it is queued; failures back off
from one minute to at most one hour, and a success deletes the target. After five failed runs the
same version is skipped with a memory_session_trigger_skipped ERROR log, so it no longer holds the
subject's lane; a newer message or recording version triggers it again, and a skipped target is
dropped once it leaves the discovery window. Because the scan runs once
a minute, a Room is summarized between 10 and 11 minutes after its last message (plus queueing),
and a ready recording within a minute.
A restart restores pending items; after a Session is written, the subject's Wiki downstream recovery marker is saved
before the checkpoint is committed. When a targeted source finishes later than the midnight Daily of its date, the
same subject's Daily → Wiki → Graph for that date runs again, and the recovery marker is cleared only after the whole
chain succeeds, without updating other subjects. Per-person lanes hold only within one process and are no
distributed multi-process lock; each data root still needs exactly one Scheduler process.
The calendar tasks run each AnotherMe isolated and in parallel, at most 4 at once. A slot is taken only when the subject's lane is free, so a subject still busy with its own earlier work does not hold one:
- Hourly
wiki: lists every authorized AnotherMe (a deployment usually has one) and runs each in isolation. The chain is Session Raw staging → indexing that Raw. Only the 00:00 run that closes the previous day, and a meeting's recording, then ask TeamHub for a Wiki refresh; the other hourly runs and a quiet Room only index (--index-only). At midnight each subject checks all changed sources, even if its conversations were already indexed or only My Pages changed. The nightly refresh is not awaited, so the Daily does not wait for the Wiki. A failure of one person leaves only that person's Wiki downstream pending and logsmemory_subject_refresh_failed; it does not mark the wholememory_schedule_runfailed or interrupt others. Only when every AnotherMe fails is the runfailed, retried hourly. Each(memory subject, Room)saves the last successfully handled Message inevidence/<date>/sessions/.checkpoints/<roomId>.json. The midnight run handles the previous day, catching up the messages after the last read and the pending Wiki updates; from 01:00 runs handle the new day. Daytime runs without new Room messages skip; whether the night generates is decided by TeamHub from changes across all sources. An explicitly named Room that does not match, a database failure and an attachment read failure still report errors. Raw is restaged only when new messages appear after that position; the input always holds the Session's full Room message history, and OpenWiki keeps old facts from the existing safe Wiki pages. Staged files live atevidence/<date>/sessions/<roomId>.md; the checkpoint advances only after the Raw is saved and the downstream pending marker is on disk. Imported messages are converted to business time and reordered by instant and Message ID, using the same comparison as the checkpoint, so several messages in the same second are not generated twice because of their physical import order. The checkpoint file format is unchanged and history need not be cleared. The Wiki generator (src/llm-wiki/agent-wiki-backend.ts) writes Client, Contact, Project and Topic pages from the Raw; only meetings get Session pages. The Scheduler and the source triggers share one Raw collection. The Wiki downstream pending marker lives atupdates/task-state/wiki-pending/<date>/<subjectKey>.json. Graph building checks historical Memory/Wiki concept pages against OKF: a page missingtypeor a path's required fields, or with broken YAML, is skipped and recorded inmemory_graph_skip/memory_schedule_run.graphSkips. Historical Client/Contact/Project pages with onlyinvalid_statusorinvalid_stale_afterstill enter the graph. Source files are never deleted. The hourly task can stillsucceed, butmemory_graph_skip,memory_subject_refresh_failedand amemory_schedule_runwith skips or failures all log"level":"ERROR"so log collection raises an alert; other AnotherMes in the same calendar refresh are not marked failed because of it. New writes still fail closed as a whole batch, and unqualified model output never reaches disk. - Daily at 00:00 (
daily): per AnotherMe, in parallel, after the closing Raw staging and without waiting for the Wiki. The day read (day-read.ts, one memory Agent task) reads the day's raw conversations once and writes the memory.md edits, thentimeline/daily/<date>.md, which is indexed beside the Wiki pages as aperiod_summarysource, never as Raw. Daily keeps no long-lived entities and builds no graph. After the day read, the same person'sprompts/tracking/tracking-items.mdupdates that Human's Tracking Items (memory_index.tracking_items). The input is the full text of every Session that day (Pocket Notes included) and the existing items; the model returns only creations and deletions, and items are never edited in place. It keeps only dated arrangements from meetings and the Human's own words (appointments, classes, trips, bookings), promises between people, unanswered questions, things blocked on outside conditions and things the Human explicitly asked to track; ordinary tasks given to Agents, finished items and Agents' requests and suggestions are left out. The call always uses medium reasoning effort, and its input carries the program-computed dates and weekdays of the next 14 days plus the end dates of this week, next week and this month; a relative period always resolves to its last day. When the day's Sessions exceed 100,000 characters in total, they are read in parts (a very large single Session is split by lines), each part seeing the items left by the previous one, and only the net result is written in one transaction at the end. An item withdue_atis removed once past; an item without a time is kept three months (deleting and recreating restarts the clock), and the list endpoint filters expired items too. If any check fails (source numbers, deletion ownership, 200-character content, ISO times or the 100-items-per-person cap), nothing is written; a failure of this step fails that person's Daily for the day, which retries as a calendar task, and the Daily output is rewritten unchanged. Each person runs independently: scheduling isolates people and retries only those who failed; a manualdailywithout a subject keeps going after one person fails and reports the errors at the end. - The day read maintains memory.md in the same task as the Daily, before it: the day's raw conversations go to the model, which returns only add, replace and delete edits (a day too large for one task falls back to separate calls that read it in parts above 100,000 characters, with the Daily); an edit drawn from a Session must quote the Human's words found in that Session. The checked edits become one new TeamHub
user_memory_revisionsversion for the day, projected to the read-onlyme/memory.md. Days without Sessions skip the step. Each written Weekly updates it the same way, and TeamHub callsPOST /internal/v1/profile-memory-syncsabout a minute after each Profile publication, merging the newer Revision against memory.md'sbase_profile_revision_id(the first one builds memory.md); a sync that keeps failing leaves the last good version. Apart from those Profile syncs, memory.md has no daytime writes. Rollback:pnpm --filter @linger/node-services memory:user-memory:rollback <memory-subject-key> <version>. - Weekly on its set day (
weekly): per AnotherMe, isolated and in parallel, each Subject with its own collection and prompt. It still waits for the same midnight's Daily task to succeed (when every AnotherMe fails, Daily still blocks Weekly). One person's Weekly failure does not interrupt others; when a person finishes,timeline/weekly/<weekStart>.mdis published atomically as PostgreSQL Raw and Chunks, with no Wiki Revision or graph update. A week without useful information calls no model, writes no empty report and publishes no index; an existing report stays unchanged. Last week's content never stands in for this week's activity. A missing Daily for a day with conversations fails and retries; it never counts as no information. Intermediate products live underupdates/weekly/<weekStart>/<memorySubjectKey>/.
Sync write boundary
Raw files, the Wiki projection and Graph files of one Human/Agent share a workspace file lock. Writes queue in
process first, and separate CLI or service processes exclude each other through the same proper-lockfile protocol;
the lock is renewed continuously and a lock left by a crash can be recovered after it expires. A write that loses lock
ownership fails closed and never keeps writing alongside the new holder. Atomic JSON and directory replacement still
bound each individual write.
The Wiki sync sends TeamHub no Session content: after it indexes the subject's Session Raw it calls
POST /v0/personal-wikis/{actorId}/memory-refresh with the date, and for a meeting (a recording run,
--capture-id) the recording's capture_id. TeamHub reads the Sessions from memory_index and keys the refresh
intent by the Session set it saw, so a retry of the same Sessions replays the same refresh. A meeting is not
admitted while another refresh runs (in_progress): the sync waits until the Wiki is no longer refreshing and
sends it again, at most three times, then logs memory_meeting_wiki_refresh_skipped (ERROR) and leaves the
meeting to the night's refresh. No run waits for a publication: Daily, memory.md and Tracking Items read the raw
conversations.
The Daily and Weekly file scope is enforced by
apps/node-services/src/memory/sync-policy.ts, and the prompt shows the same
boundary:
- Daily reads the day's raw conversations, Pocket Notes and yesterday's Daily when available, and the day read
maintains memory.md at the same time. The report is written to
timeline/daily/<date>.mdand published as PostgreSQL Raw. - Weekly reads the Dailies of seven specific dates and last week's user dynamic, writes only
timeline/weekly/<weekStart>.mdand publishes PostgreSQL Raw. - Neither stage may delete or generate the old User, Project, Client or Workflow files. Each batch takes a snapshot before writing; a failure midway restores the changed paths, so no half batch remains.
These steps call Memory module functions directly, never a shell or pnpm child process.
Memory update generation runs through src/runtime-model-call.ts on the Computer of that Human's Another Me, with
Codex's or Claude's default model. The day read uses a step-limited Agent loop that validates the profile edits and
the Daily separately. Tracking and Dreaming use structured model output. Session facts take at most 16 windows per
call and return each window's facts by window_sha256. The personal knowledge query itself calls no second answering
model; it only does deterministic search and Evidence packaging, and the final answer comes from the Agent Runtime
that made the tool call. LLM Wiki generates candidate pages step by step through the same Runtime; see
src/llm-wiki/AGENTS.md.
Manual commands
pnpm --filter @linger/node-services memory:daily -- --date 2026-09-01
pnpm --filter @linger/node-services memory:weekly -- --week-start 2026-08-31
pnpm --filter @linger/node-services memory:wiki-sync -- --date 2026-09-01
The :collect, :prompt and :apply actions of memory:daily and memory:weekly remain for manual inspection.
wiki-sync indexes the Session Raw, requests the refresh and finishes once TeamHub returns its refresh_id; the result records
accepted / in_progress and no longer waits for the new Revision. TeamHub queues, generates and safely publishes in
the background; the persistent outbox in the publication transaction then commands Memory to materialize the latest
Wiki and update the graph. already_latest can still materialize the current safe Revision directly, and no_sources
triggers no generation.
PostgreSQL tests
tests/memory/*.postgres.test.ts, chunk-index-sql.test.ts and session-source-sql.test.ts are skipped by default
and run only when these variables are set, and only against a dedicated localhost database:
MEMORY_SOURCE_TEST_DATABASE_URL: the database name must bememory_source_test. The tests clear its Tenants and Accounts.MEMORY_CHUNK_TEST_DATABASE_URL: the database name must bememory_chunk_test.MEMORY_WIKI_TEST_DATABASE_URL: the database name must matchmemory_wiki_test_<hex>.
Initialize each database with the TeamHub migrations first (in apps/teamhub, run
TEAMHUB_DATABASE_URL=<database URL> go run ./cmd/migrate). Tests that share a database clear each other's tables,
so run them serially:
cd apps/node-services
pnpm exec vitest run --no-file-parallelism tests/memory/*.postgres.test.ts tests/memory/chunk-index-sql.test.ts tests/memory/session-source-sql.test.ts
Failure isolation between refresh and indexing
Wiki publication and Graph revisions do not depend on Chunk indexing succeeding. Raw is indexed on Memory's own schedule, never from a publication, so the hourly index fills any gap. syncAll keeps handling the other Agents after one fails and still marks the whole run failed at the end. When the generation stage finds the PostgreSQL original text missing or unavailable, it falls back only to local Raw whose current manifest and hash check pass.
Room Raw attaches only to Session pages whose URI and business date both match, so different dates never mix under one provenance. When the cross-process lock is lost, the current operation and its later file publications fail in isolation while the service process and the later queue survive. An older hourly retry of the same date can be superseded by a newer scan, but the closing dependencies of Daily/Weekly must still complete.
Failure visibility and manual recovery
The two kinds of automatic retry each have a final or persistent failure state, shown by one read-only endpoint (skips of source triggers are recorded only in logs):
- Calendar tasks (Session/Daily/Weekly) retry only the failed subjects, after 15 s, 1 min, 3 min, 10 min and 1 h; the sixth failure moves the task to the calendar's
failedlist. Dependencies are isolated per subject: subjects still failing in the prerequisite are held in Daily/Weekly while the others run; when the prerequisite is exhausted, only those subjects' downstream tasks stop withDependency exhaustedand an ERRORmemory_calendar_dependency_exhaustedlog. A failure that cannot be attributed to subjects still blocks the whole downstream task. - A failed Raw Chunk source is recorded as
state='failed'(including documents never stored that have only a normalization failure record); the hourly reconciliation and start-up keep retrying it automatically, with no manual action.
Wiki indexing has no Memory queue: TeamHub's publication job retries until Memory indexes the published Wiki, and the Memory scheduler reconciles Daily and Weekly reports every minute.
GET /internal/v1/health/failures returns scheduler (exhausted, pending_retries and their totals),
raw_chunk_sources (failed sources and counts per owner), and session_facts (latest failed fact sync per Agent).
Each list holds at most 200 items, error summaries are cut
to 300 characters, and no text is included. An unconfigured store returns {"configured": false}, a failed query
returns {"error": "unavailable"}, and the other parts still return 200.
Session fact diagnostics are process-local (scope: "process", observed_since) and reset on restart;
zero means no observed failed syncs, not that all persisted sources have facts. Each failed Agent reports
failed_windows, last_failure_at, and last_error_kind from its latest finished sync. A sync that aborts
before a complete count reports failed_windows: null. Only a successful sync for the same Agent clears
its failure; partial extraction keeps it visible. Existing sync triggers retry the unfinished windows.
The memory.session_fact_extraction_failed log includes error_kind, agent_id, tenant_id, owner_id,
and the batch's windows count. Error kinds use structured model-call errors (or unknown), never text
matching; these added fields contain no source text or model output.
POST /internal/v1/health/failures:retry accepts only scheduler.task_keys (all tasks when omitted);
any other field returns 400. Calendar tasks and the dependents they stopped go back to pending with
zero attempts and the request wakes the scheduler, which runs them within about a second; the
request itself does not run them. The response holds only the requested parts and their counts.
# The service token derives from LINGER_SECRET_SEED with the node-services-internal label.
MEMORY_TOKEN="$(node -e 'const c=require("node:crypto");process.stdout.write(Buffer.from(c.hkdfSync("sha256",process.env.LINGER_SECRET_SEED,"simplexai/service-credential/v1","node-services-internal",32)).toString("base64url"))')"
curl -s -H "Authorization: Bearer $MEMORY_TOKEN" \
http://127.0.0.1:4319/internal/v1/health/failures
curl -s -X POST -H "Authorization: Bearer $MEMORY_TOKEN" \
-H 'content-type: application/json' \
-d '{"scheduler":{"task_keys":["daily:2026-09-29"]}}' \
http://127.0.0.1:4319/internal/v1/health/failures:retry
Production history rebuild
Use this to regenerate one Human's Daily and Weekly with the current pipeline from original data such as Room messages, recordings and My Pages documents, and to bring their Personal Wiki up to date with those Sessions. The rebuild never regenerates a published Wiki: its refreshes are ordinary incremental updates that send only the sources the Wiki has not published, so pages already written stay as they are. A Human without a published Wiki gets one initialized by the first step. Start the rebuild inside the Node Services Pod:
node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js start --owner <Human Actor ID>
# Pilot: replay only the Sessions of the last few days
node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js start --owner <Human Actor ID> --from 2026-09-23
node --conditions=linger-production apps/node-services/dist/memory/rebuild-cli.js status --owner <Human Actor ID>
--owner without a matching Another Me returns 404 (each Human has at most one); with a known Subject use
--subject <memory_subject_key> instead. The CLI derives the service token from LINGER_SECRET_SEED in the Pod
environment, reads the port from NODE_SERVICES_HTTP_ADDRESS (4319 by default), calls POST / GET /internal/v1/rebuild on the local 127.0.0.1 and prints JSON. The production calendar must be running, or it returns 409.
On start the service asks TeamHub for the dates on which the Subject actually has Sessions: the conversation message
dates of the Rooms the Human or its Another Me is in, and the dates of finished recording transcripts, all by local
business date, matching Session collection. The range runs from the earliest Session date to today in the local time
zone; --from only cuts earlier dates, and the end date is always today. The chain below is then queued, each step
depending on the previous one:
init:rebuild:<subject>: requests one Wiki refresh over the current sources (My Pages, Connected App files and the Sessions the index already holds), waits for the publication to catch up with them, and repeats until TeamHub returnsalready_latest(no_sourceswhen there is nothing).index-initial:rebuild:<subject>: index once, running the Another Me's full source sync and indexing the published Wiki (wikisources) directly; the step finishes when both calls return.- Replay only the dates with Sessions, in time order:
wiki:rebuild:<subject>:<date>(indexes the logical date's Session Raw and requests a Wiki refresh; it finishes once TeamHub accepts the request and does not wait for publication; when a refresh is already running, TeamHub folds these Sessions into that refresh's follow-up) → that day'sdaily:rebuild:…, which reads the date's Session Raw rather than the Wiki (skipped for today, which the midnight calendar finishes). After the last Session date of a week (week_startis Monday), if that week has ended,weekly:rebuild:<subject>:<Monday>runs; the current unfinished week is skipped. In this phase only the Wiki sync indexes the replayed Session Raw, because the Wiki reads Sessions from the index. index-final:rebuild:<subject>: run the source sync once more and index thewikipages and thedailyandweeklyreports (period_summarysources).
- The first rebuild of a Subject moves its local derived state (
<subject>/memoryand that Subject's dated products underupdates/) in the Subject's lane to<subject>/memory.pre-rebuild-<time>, and deletes that owner'swikiandperiod_summarysources with their chunks frommemory_index.sources(WikiIndex.clearOwner); other owners are untouched. The original-source rows are rebuilt from the original data and are kept. Each Subject keeps only its latest archive; older ones are removed by the daily file cleanup. - Index hold: while the Subject has pending rebuild steps, the publication job's Wiki and source indexing,
the report indexing after Daily/Weekly, the scheduler's minute-long report reconciliation and the startup
and hourly source syncs all skip that Another Me (log
memory_rebuild_index_hold_skipped); other users are not affected. The hold is derived from the pending rebuild steps in the saved calendar, so it survives restarts; it ends when step 4 completes or the chain is exhausted (logmemory_rebuild_index_hold_released,outcomecompletedorfailed). - Rebuild steps occupy the Subject's lane: they never run alongside that Subject's calendar or Session trigger tasks, and other Subjects' tasks run in parallel as usual. Each scheduler check still queues due calendar tasks before rebuild steps.
- Concurrency: one rebuild step runs at a time, and several Humans' rebuilds run one by one in queue order. This
affects only rebuild steps, not the concurrency, order or priority of calendar tasks. A rebuild's Wiki receipt is
written to
updates/wiki-sync/<date>/<subject>/result.json, while the calendar still writesupdates/wiki-sync/<date>/result.json, so concurrent rebuilds of the same date never overwrite each other. - Calling
startagain while rebuild steps exist returns the existing steps and adds no dates. To start over, add--restart: it forgets the Subject's rebuild steps, rediscovers the dates, archives and clears the owner'swikiandperiod_summaryindex; with steps still pending it returns 409. It leaves the published Wiki, the indexed Sessions and User Memory as they are. statusreturnsphase(init,index-initial,sessions,index-final,completed,failed),sessions.completed / sessions.total,current_task,current_date(the date in the Session phase),index_hold,from/to(the first and last Session dates),total,completed,pending,failed,runningand a failure summary.completedmeans every step ran; the Wiki refreshes the date steps requested may still be generating while the Human's Personal Wiki reportsrefreshing. A failed step uses the calendar's 1, 2, 4 and 8 hour backoff; after the fifth failure that step and everything after it stop withDependency exhaustedand appear inGET /internal/v1/health/failures; each step run writes onememory_schedule_runlog. After fixing the cause, pass the failed task keys (such asindex-initial:rebuild:<subject>, or none to resume everything) toPOST /internal/v1/health/failures:retryto continue from the break point, and the index hold takes effect again. A restart continues from the saved calendar.- Cost: each Session date requests one Wiki refresh and calls the Daily model once, and each finished week with
Sessions calls Weekly once; a year of history means hundreds of model calls, taking hours to days. Replay the last
7 days with
--fromfirst, then run the whole chain with--restart. - During a rebuild the normal calendar still handles that Subject's current day, so today's Sessions may publish before history; they are replayed again in order later.
Wiki pages and reports in the one index
The published Wiki pages and the Daily/Weekly reports are sources of the same memory_index.sources and
chunks as every other source; there are no wiki_* tables and no separate search.
- A Wiki page is a
wikisource (source_kind='wiki_page') of each of the Human's Another Mes, holding the current published Markdown withpage_key,document_id,document_version_idandpublished_atin its metadata. Search chunks and reads only the page body without its inline citation labels ([R1],[R1, R2],(R2)), with the frontmatter's description and aliases under the title and the rest of the frontmatter (sources, relations, item IDs) left out; chunk offsets and lines refer to that text, while the stored version keeps the published Markdown for the graph and flashback. Navigation pages (index.md,readme.md,log.md,instructions.md) and empty pages are stored for the graph without chunks, so the search never returns them. There is no version history. - TeamHub's publication job calls Memory for each published Wiki (
wiki-index.ts). Memory reads the published page list, downloads only the pages whose Document Version the index does not hold, checks each page's SHA and size, ingests each page through the chunk index, and deletes thewikisources the publication no longer has. An API failure or an incomplete download leaves the current index; the job retries. - A report is a
period_summarysource (source_kind='period_report', metadatacollectionandperiod_start) built fromtimeline/dailyandtimeline/weekly; it is indexed when it is written and by the scheduler's minute-long reconciliation, which also handles edits and deletions. An unreadable directory keeps the indexed reports. readIndexedWikiandindexedWikiRevisionread thewikisources for flashback, the knowledge graph and the meeting summary; the revision is a digest of the indexed page versions.- A failed index does not roll back the Wiki or report generation.
- Session Raw is indexed as
roomandrecordingsources, and those are the Wiki's Session sources: TeamHub reads them from this index (wiki_session_sources), freezes each at itssource_versionsSHA-256, and the generator gets that text with its routing. There is no evidence Document. Session pages cite the Room messages their facts come from, and a recording its transcript Document Version.
All sources share one keyword ranking: tokenizeSearchText writes the weighted tsvector (heading A, body
and title B, path C), GIN with ts_rank_cd recalls, the lexical rerank (search/lexical-rerank.ts) orders by
query-term coverage, and the query, each alternates wording and the time window vote by RRF (k=60).
Configuration and deployment
The index tables are created by TeamHub migrations; the Memory role requires TEAMHUB_DATABASE_URL
and does not start without it. No PostgreSQL extension is needed.
Index timing
Each successful Wiki publication (init included) makes TeamHub's publication job ask Memory to index it; Daily/Weekly are indexed after they are generated. Generation does not wait for indexing; one collection is indexed at a time per process, and a database advisory lock serializes processes.
Historical Wiki and Daily/Weekly rebuilds use the separate entry in "Production history rebuild".