Garry's Opinionated OpenClaw/Hermes Agent Brain
Knowledge layer map: graph, entities, facts, takes, timeline
Line numbers refer to gbrain 36bd05806a.
Scope: the knowledge layer of gbrain v0.60.733.0 (commit 36bd05806a, read-only reference at garrytan/gbrain). Eval references are in garrytan/gbrain-evals (commit 9ecc5b20).
Legend: unmarked claims were read in code and cite file:line. "[inferred]" marks conclusions drawn from comments, naming, or standard Postgres behavior that I did not run.
1. Summary
- The knowledge layer is one graph (
links, with provenance columns), two entity-side tables (page_aliases, mention index), three fact-like stores (facts,takes,timeline_entries), and several DB-only ledgers (withdrawals, purge tombstones, belief events). - Markdown is canonical for file-backed knowledge. The
takesandfactstables are derived indexes over fenced Markdown tables. Withdrawals, purge tombstones, belief events, and DB-only facts are not rebuildable from Markdown. (docs/architecture/system-of-record.md:44-100) - Postgres features in use: recursive CTEs (BFS with array visited sets),
pg_trgm,jsonbcontainment,jsonb_to_recordsetbatch writes,NULLS NOT DISTINCT(PG15+), HNSW onhalfvec/vector,tsvectorGIN, advisory and row locks, and SQL triggers. - Search-time graph signals are multiplicative boosts from link counts (backlinks, adjacency, cross-source, session demotion), not embeddings.
- Identity resolution is exact matches, then a same-name trigram check, then slugify.
whoisis exact-only. - Facts are fence-first, with a DB-only fallback. Forget is a DB withdrawal plus a Markdown strike. Purge is a separate owner-only tombstone operation.
2. Authority: what can be rebuilt from Markdown
| Store | Table(s) | Authority | Rebuild source / note |
|---|---|---|---|
| Page body, frontmatter | pages, content_chunks |
FS canonical; DB-only pages exist | sync + extract (system-of-record.md:50-75) |
| Markdown-derived links | links (origin_page_id set) |
Derived | Re-extract from body and frontmatter |
| Manual DB-only edges | links (origin NULL) |
DB-only | Not in Markdown (link-temporal-schema.ts:6-8) |
| Wanted links | wanted_links |
Derived; page text is truth | wanted-links.ts:1-14 |
| Temporal evidence | link_transitions |
Derived from origin page | Replaced with origin's derived links (link-temporal-schema.ts:6-8) |
| Temporal state cache | link_relationships |
Derived cache | Rebuilt from transitions; graph_generation_seq invalidates query caches |
| Edge contradiction proposals | link_edge_proposals |
Operational state [inferred] | Apply/undo state, not Markdown |
| Takes | takes (+ synthesis_evidence) |
FS canonical via ## Takes fence |
takes-write.ts:1-12; takes-fence.ts:1-30 |
| Facts | facts |
FS canonical when file-backed; DB-only otherwise | system-of-record.md:138-143 |
| Forget withdrawals | fact_withdrawals |
DB-only, authoritative | withdrawal-schema.ts:84-93; system-of-record.md:378-400 |
| Purge tombstones | fact_purges, take_purges, page_purges |
DB-only, authoritative | purge-schema.ts:36-64 |
| Belief events | belief_events, belief_page_lifetimes, belief_commit_marks, belief_retier_manifest, belief_redactions |
DB-only by design | belief/schema.ts:17-93; system-of-record.md:98 |
| Timeline rows | timeline_entries |
Derived from Markdown | system-of-record.md:57-60 |
| Tags | tags |
Derived from frontmatter; tag_source='added' rows are DB-claimed |
page-state/tags.ts:12-14 |
| Page versions | page_versions |
Audit history; "rebuildable in principle but not in practice" | system-of-record.md:72-74 |
| Entity aliases | page_aliases |
Derived from frontmatter/declared/subject | v110-page-aliases.ts:32-44; v206 |
| Calibration profiles | calibration_profiles |
Published snapshots [inferred: history not rebuildable] | v068-calibration-profiles-v0-36.ts:34-60 |
Note: docs/architecture/system-of-record.md:98 lists the belief ledger as DB-only. The design doc (docs/designs/BELIEF_LEDGER.md) still says "Status: design, not built," but src/core/belief/ contains about 4,100 lines of implementation. The code is the more recent source. Treat the belief ledger as shipped but confirm with the owners.
3. Links and the graph
3.1 Schema
links (src/schema.sql:549-584):
- Columns: endpoints
from_page_idandto_page_id(FK cascade),link_type,context,link_source,link_kind,origin_page_id(FK set null),origin_field,resolution_type,assertion_tense,created_at. - Unique key:
UNIQUE NULLS NOT DISTINCT (from_page_id, to_page_id, link_type, link_source, origin_page_id). The comment ties this to PG15+. - Indexes:
from_page_id,to_page_id,link_source,origin_page_id, and a partial index onto_page_idwherelink_type='supersedes'.
wanted_links (schema.sql:595-611): authored references whose target does not exist yet. Columns include producer (body | frontmatter), ref_kind (slug | name), target_source_id, target_ref, checked_at. Unique on (origin_page_id, producer, ref_kind, target_source_id, target_ref). A live target updated after checked_at makes the origin stale for extraction (wanted-links.ts:1-14).
code_edges_chunk and code_edges_symbol (schema.sql:472-515): call-graph edges between content_chunks. Unique on (from_chunk_id, to_chunk_id, edge_type) and (from_chunk_id, to_symbol_qualified, edge_type). Writes use ON CONFLICT DO NOTHING (engine-sql/code-edges.ts:46-55). The PGLite path splits binds below 30,000 per statement because of a parameter-bridge limit (code-edges.ts header).
3.2 Writes
addLink(engine-sql/links.ts:102-157) resolves both endpoints and upserts in one statement. Endpoint lookups takeFOR KEY SHARE(links.ts:118-124) so a concurrent hard delete linearizes. A missing endpoint raisesPageMissingError.- Batch writes (
addLinksBatch, links.ts:216-256) pass one JSONB document and usejsonb_to_recordsetwithON CONFLICT ... DO NOTHING. replaceDerivedLinks(derived-links.ts:170) replaces all links owned by one origin page, plus itswanted_linksrows, in one transaction. It checks origin revision and endpoint revisions and raisesDerivedLinkEndpointChangedErroron mismatch.link_source = 'mentions'rows are body-text mentions. They feed orphan and traversal logic but are excluded from backlink counts and ranking (schema.sql:553-556; graph-rank.ts:116).
3.3 Reads
getLinks/getBacklinks(links.ts:366, 437) are indexed point reads with privacy, source, and temporal filters.- Link types that do not count for ranking come from a code constant:
NON_RANKING_LINK_TYPES = ['amends'](link-ranking-policy.ts:21). The SQL predicate islink_type <> ALL(ARRAY['amends'])(link-ranking-policy.ts:33-35).
3.4 traverse_graph: the BFS walk
traverseGraph (links.ts:659) and traversePathsDetailed (links.ts:711) share bfsWalkCtes (links.ts:609-648). This is a WITH RECURSIVE query over an array state, not a path enumeration:
- Row 0 is the seed set,
array_agg(id ORDER BY slug, id). - Each step expands the frontier with
LATERAL, dedupes neighbors withNOT EXISTS (... unnest(visited)), orders them by(slug, id), and caps each layer withLIMIT. visitedgrows asbfs.visited || n.next. The walk stops whendepth >= maxLayer.- A window function
row_number() OVER (ORDER BY depth, ord)gives each page apos.
Constants (engine-constants.ts):
TRAVERSE_WALK_ROW_CAP = 20000(line 48): total pages visited.pos > capis dropped.TRAVERSE_PATH_ROW_CAP = 5000(line 37): returned edges. Excess setstruncated: true.- Default
depth5 (links.ts:659).frontierCapbounds new pages per layer.
Privacy and scope apply at the seed, at each step, and on both endpoints of every edge. This keeps graph reads inside the caller's source scope (comment at links.ts:672-674).
3.5 rank_entities
rankEntitiesStatement (graph-rank.ts:115-185) returns entities ordered by inbound, outbound, or both degree. Its structure:
readable(MATERIALIZED): pages that the caller may read (live, unquarantined, unarchived source, optional private exclusion).retired(MATERIALIZED): pages withcanonical: false,superseded_by,merged_into, or a status wordalias|redirect|redirected(entities/resolve.ts:210-215).chain(WITH RECURSIVE): follows redirect targets up toMAX_RETIREMENT_HOPS = 5(resolve.ts:217).fold(MATERIALIZED,DISTINCT ON): the first live, readable target of each retired chain.edges(MATERIALIZED,SELECT DISTINCT): endpoints folded to targets, self-links dropped, non-ranking types dropped.ends/d: counts of in, out, and distinct neighbors per node, filtered by entity type.
Ordering is key DESC, slug COLLATE "C", source_id COLLATE "C". The doc comment (graph-rank.ts:20-30) gives a benchmark: about 3 s on Postgres 16 with 200k pages and 985k links, local and remote.
3.6 Search boosts (not SQL-side)
Two boost paths, both computed after fusion in src/core/search/:
- Backlink boost (
applyBacklinkBoost, hybrid.ts:257-300): counts distinct linking pages, excluding self-links and mention links. Factor1 + 0.05 * ln(1 + count). Examples from the comment: 1 link gives about 1.035, 10 about 1.12, 100 about 1.23. Results below the floor threshold are skipped. Counts come fromreadBacklinkCounts(read-enrichment.ts:243-263), keyed bypage_id. - Adjacency and cross-source boost (graph-signals.ts:54-68): among the top
DEFAULT_TOP_K = 20results, a page linked from at leastADJACENCY_MIN_HITS = 2other top-K pages getsADJACENCY_BOOST = 1.05. At leastCROSS_SOURCE_MIN_HITS = 2distinct other sources getsCROSS_SOURCE_BOOST = 1.10. Session-prefix duplicates are demoted toSESSION_DEMOTE = 0.95. Fail-open: an error returns input unchanged. The SQL isreadAdjacencyBoosts(read-enrichment.ts:265-287), with candidate ids inlined as integers to avoid a bad generic plan. - Hub dampening (search/hub-dampening.ts:11-14):
hubWeight(n, H) = 1 / (1 + ((max(n,1)-1)/H)^2). Call sites were not traced.
BrainBench Cat 27 (gbrain-evals/eval/runner/cat27-graph-signals.ts:1-12) A/B-tests these signals on PGLite.
3.7 Derived-link provenance
Derived links carry origin_page_id and origin_field, so one page's re-extraction replaces only its own rows. Attendance and frontmatter-link rules are in docs/architecture/key-files/graph-and-facts.md.
4. Entities and identity resolution
4.1 Resolver ladder
resolveEntitySlug (src/core/entities/resolve.ts:60-168) returns a slug or null. Its order, as read:
- Exact slug, when the input looks like a slug. A retired exact match follows its redirect (
followRetirement). - Exact own name: a live page whose slug basename equals the input wins over another page's alias (resolve.ts comment at 1.25).
- Organization directory variant (
orgDirVariants) andalias_exacthits frompage_aliases(resolve.ts:530-534). - Same-name trigram candidate (only for multi-token inputs with
sameName):lower(title) % $2 OR slug ILIKE '%'||$3||'%',ORDER BY GREATEST(similarity(title), similarity(slug)) DESC, slug,LIMIT 5(resolve.ts:538-552). Accepted only ifisFactEntityPageandsameEntityNameboth hold. - Bare-name prefix expansion (
findPrefixCandidates, resolve.ts:687). A single prefix candidate resolves. Several return{miss: 'unverified'}. - Fallback to a deterministic
slugifyby the caller. It does not prefix a directory (resolve.ts:53 comment).
sameEntityName (resolve.ts:195-199): both sides normalize to a sorted, de-duplicated token set, with accents folded, punctuation dropped, NAME_NOISE_TOKENS removed (resolve.ts:166). "Example, Alice" matches "Alice Example". "Alicia Example" does not. The token list is code state, so the Rust port must copy it exactly.
4.2 Trigram use
pg_trgm is required by schema (CREATE EXTENSION IF NOT EXISTS pg_trgm, schema.sql:4) and indexed:
idx_pages_trgmonpages(title gin_trgm_ops)(schema.sql:282).idx_pages_slug_trgmonpages(slug gin_trgm_ops)(schema.sql:286).
Uses:
resolve.ts:544, 592, 883-891:%plussimilarity().similar-pages.ts:58-59: title%plussimilarity >= TITLE_SIMILARITY_FLOOR = 0.55(similar-pages.ts:23).engine-sql/pages.ts:775-779: fuzzy page lookup.engine-sql/takes.ts:343-371:query <% t.claimwithword_similarity. Ordered by score, then weight.resolve.ts:868-918,tryFuzzyMatch. Two modes:- Same-name mode (default):
LIMIT 5, exactly oneisFactEntityPage+sameEntityNamecandidate wins. Two namesakes are ambiguous and return null. - Phantom-canonical mode: a winner needs score >= 0.7 and a margin of at least 0.1 over the runner-up (resolve.ts:868-918).
- Same-name mode (default):
- If pg_trgm is missing (
42883or the operator error), the fuzzy arm is skipped and the resolver falls through to slugify (resolve.ts:920-925).
[inferred] The % operator uses the pg_trgm default threshold 0.3, and <% uses word_similarity_threshold 0.6. No override was found in the repo, but I did not check the config table or extension defaults.
4.3 whois: exact-match rule
whois (src/core/ops/whois.ts:1-22) answers only on an exact match. It accepts an email address or a name, and matches the frontmatter email/emails value, the page title, or a declared alias. Folding: case, accents (NFKD minus marks, stroke letters folded), compatibility width forms, and surrounding whitespace or quotes. An email query compares whole addresses case-insensitively, with no accent folding. There is no nickname, typo, prefix, or semantic match, and no search fallback.
Candidate SQL (engine-sql/exact-identity.ts:41-100) is a UNION of index-served equality probes, one per arm:
lower(p.title) = ANY(keys):idx_pages_source_title_lower.regexp_replace(p.slug, '^.*/', '') = ANY(basenames):idx_pages_slug_basename.page_aliases.alias_norm = ANY(keys):(source_id, alias_norm)index.p.frontmatter @> {email: x}and three other shapes: GINidx_pages_frontmatter. Each address gets its own arm, because anORplans worse.
Limits: query at most 320 characters (WHOIS_MAX_QUERY_CHARS), at most 20 types, at most 100 candidates (WHOIS_CANDIDATE_CAP, whois.ts:24-27). Output includes ambiguous: matches.length > 1.
4.4 Name variants and near-name index
name-variants.ts:23-28 classifies seven variant kinds; typo and surname are weak and need a single matching page overall. near-name.ts builds an in-process index (names 3-64 chars, at most 200,000 entities), not SQL.
4.5 Retirement
A retired page hands its identity to a target: superseded_by or merged_into, or canonical: false, or a status word. Chains stop at MAX_RETIREMENT_HOPS = 5 and at cycles (resolve.ts:210-234). rank_entities folds retired edges onto the target. The redirect field is read from frontmatter with regex (graph-rank.ts:80-83).
5. Facts
5.1 Schema
facts (created by migration v045, src/core/schema-migrations/v045-facts-hot-memory-v0-31.ts:113-160; later columns from v047, v051, v067, v089, v122, v166, v215, v273):
- Identity:
id BIGSERIAL,source_id,entity_slug,fact,kind(event|preference|commitment|belief|fact|idea),visibility(private|world),notability,attributed_to. - Validity and dedup:
valid_from,valid_until,expired_at,superseded_by,consolidated_at,consolidated_into. - Provenance:
source,source_session,confidence REAL(0..1),row_numandsource_markdown_slug(fence round-trip),source_window JSONB. - Typed claim:
claim_metric,claim_value,claim_unit,claim_period,event_type,dimension,value,dim_status. - Embedding:
halfvec(N)orvector(N)(N default 1536, v045:32), withembedding_model,embedded_at, andembedded_text_hash = md5(fact)gating comparability.
Indexes: partial expired_at IS NULL indexes on entity, session, and time; an unconsolidated index; a partial HNSW index on live embeddings, skipped if the dimension exceeds the pgvector limit for the type (v045:99-106); a keyword index idx_facts_fts (fts-index.ts:1-25); and a partial unique index on (source_id, source_markdown_slug, row_num).
halfvec requires pgvector >= 0.7. Migration v045 falls back to vector below that (v045:60-75). PGLite always uses halfvec (v045:89-91).
5.2 remember: write path
Verb (src/core/verbs.ts:67-120): requires provenance for a single fact (max 500 chars). entity is the subject. ttl is optional. Batch items are capped at 20.
Implementation (src/core/facts/write-single.ts:1-20):
- Resolve the entity (section 4).
- Dedup via
decideSingleFact: exact fingerprint first (always runs), then embedding cosine at the model's calibrated threshold. - If the top candidate scores at or above threshold with the same
kindbut different text, the new fact supersedes it. Same text is a duplicate and returns the existing id. - Fence-first write to Markdown. Legacy DB-only fallback for thin clients, unparented sources, or
sync.write_throughoff (fence-write.ts:314-365).
Thresholds (src/core/facts/supersession-threshold.ts:14-30): cosine supersession applies only to calibrated provider:model@dims. The one entry is voyage:voyage-4@1024 = 0.95. Uncalibrated models insert without superseding, and conflict review handles the pair. Operators can set facts.supersession_thresholds, or off.
The fingerprint (withdrawal-schema.ts:27-40): gbrain_fact_fingerprint = sha256 hex of a normalized claim (lowercased, punctuation translated to spaces, whitespace collapsed, trimmed). gbrain_fact_fingerprint_v1 trims, lowercases with the host's lower(), collapses whitespace and hashes; it does no punctuation folding. The fingerprint is stored in fact_withdrawals and fact_purges, so it must match byte-for-byte.
Concurrency (engine-sql/facts.ts:88-170): when entity_slug is set, the insert takes pg_advisory_xact_lock(hashtextextended(source_id || ':' || entity_slug, 0)). The supersede path also updates the old row inside the same transaction. The comment in write-single.ts:14-17 says dedupe is sequential, so two identical submissions racing can both insert.
5.3 recall
recall (src/core/ops/facts.ts:298-330) reads facts by entity, since, or session_id. query also searches pages. Remote callers get world-visibility facts only, except a caller bound to the brain owner, which also reads private facts (worldOnlyFacts); source grants and member access still apply. Dedupe candidates for a write use findCandidateDuplicates (engine-sql/facts.ts:580-636): k clamped to [1, 20] (default 5), filtered by source_id, entity_slug, expired_at IS NULL, valid_until in the future, and embedding <=> query when comparable. The keyword arm (findKeywordCandidates, engine-sql/facts.ts:638-666) uses plainto_tsquery, to_tsvector @@ q, and ts_rank_cd.
5.4 forget: withdrawal
forget (src/core/verbs.ts:484-516) takes a fact id and an optional reason. It is idempotent. The result is ok, path (fence | legacy_db | not_found | already_expired), and a reason.
The mechanism (forget.ts:1-15, withdrawal.ts:1-30, withdrawal-schema.ts:84-100):
- Insert a row into
fact_withdrawals (source_id, visibility, subject, fact_hash, withdrawn_at), primary key on the four key columns.subject='*'means every entity. - Trigger
facts_preserve_withdrawal(BEFORE INSERT OR UPDATE OF fact, source_id, visibility, entity_slug, expired_at) re-applies withdrawal to matching new rows. - Expire the row (
expired_at,valid_until). Strike the Markdown row and append the reason. Withdrawal is DB-first and does not require a filesystem owner (system-of-record.md:378-392). - A forgotten claim without an entity is withdrawn for every entity. Remembering the same text again is refused with
fact_withdrawn(withdrawal.ts:20).
"Forget expires, never deletes" is the invariant (purge.ts:1-7).
5.5 purge_fact: separate and destructive
purge_fact (src/core/facts/purge.ts:1-30) runs only from the trusted local CLI. It is not a mode of forget. One transaction:
- Write
fact_purges(text-free tombstone), andtake_purgesfor verbatim takes. - Hide and rederive derived rows (
derivation_inputs,needs_rederive). - Delete the fact row and drop fence rows from stored page bodies and
page_versionssnapshots. - Redact carried request intents. Drop P8 review rows.
- Post-commit: rewrite the Markdown file and commit
gbrain: purge fact <hash8>.
Refusal triggers (purge-schema.ts:124-187): facts_refuse_purged (BEFORE INSERT OR UPDATE OF fact, source_id, visibility, entity_slug), takes_refuse_purged, and pages_refuse_purged block reinsertion of purged content.
5.6 Consolidation
consolidateFact (engine-sql/facts.ts:668-670) sets consolidated_at and consolidated_into (a take id). The caller is the publication path in src/core/persistence/prepared-maintenance.ts:780-800: a consolidation request creates a take with kind='fact', holder='self', then marks each source fact as consolidated into that take, inside the same transaction. Consolidated facts are excluded by idx_facts_unconsolidated.
5.7 Trajectory
find_trajectory (ops/insights.ts:301) calls findTrajectory (engine-sql/facts.ts:672-740):
- Filters:
entity_slug,source_idorsource_ids,expired_at IS NULL, world visibility for remote callers,projectionEligibleSql(quarantine and rederive gates), optionalclaim_metric,kind(metricorevent),since/untilonvalid_from. ORDER BY valid_from DESC, id DESC LIMIT n, then reversed to chronological.clampSearchLimit(limit, 100, 500).- Embeddings returned only when
embedding_modelmatches the config value andembedded_text_hash = md5(fact).
The math lives in TypeScript (src/core/trajectory.ts:1-25): a regression fires when a comparable pair (same metric, unit, period, strictly later) drops at least 10% (GBRAIN_TRAJECTORY_REGRESSION_THRESHOLD). Drift is 1 - mean(cosine(emb[i], emb[i-1])), clamped to [0,1], null with fewer than 3 embedded points.
6. Takes, beliefs, calibration
6.1 Takes
takes (migration v037, src/core/schema-migrations/v037-takes-and-synthesis-evidence.ts:25-49; later changes v043, v048, v080, v094, v142, v251):
id BIGSERIAL,page_id(FK cascade),row_num,UNIQUE (page_id, row_num).claim,kind(fact|take|bet|hunch),holder TEXT,weight REALin [0,1] (grid-rounded, v048),since_date,until_date,source,superseded_by,active.- Resolution:
resolved_at,resolved_outcome,resolved_value,resolved_unit,resolved_source,resolved_by,resolved_quality(correct|incorrect|partial|unresolvable, v043). embedding VECTOR(N), with N matched to config (v142), HNSW partial onactive AND embedding IS NOT NULL.
Authority: markdown ## Takes fence is canonical. The DB row is a mirror written through addTakesBatch (takes-write.ts:1-20). Writers that create DB-only takes are forbidden because the next sync clobbers them. Resolution columns are rendered into the fence when any row on the page has them (takes-fence.ts:85-93).
6.2 Scorecard and calibration
getScorecard(engine-sql/takes.ts:599-648): Brier over resolvedcorrect/incorrectbets asAVG((weight - 1{correct})^2), plus countstotal_bets,resolved,correct,incorrect,partial,unresolvable_count. Scoped by holder, domain prefix, since/until, sources, and privacy.getCalibrationCurve(engine-sql/takes.ts:650-695): bins byFLOOR(weight::numeric / bucket),LEAST(..., max_idx). The comment (lines 678-681) explains thatNUMERICis required because float bucketing differs between Postgres and PGLite at boundaries (e.g. 0.7/0.1 = 6.999...). Rust must use exact decimal or integer arithmetic here.- Domain assignment:
take_domain_assignments(v094), with aggregators in src/core/calibration/domain-aggregators.ts:1-20. The aggregator set is closed in code; domain names are open in pack manifests. - Profiles:
calibration_profiles(v068:34-60), withdomain_scorecards JSONB,active_bias_tags TEXT[]with GIN, apublishedflag, voice-gate fields, andmodel_id. - Contradiction runs and an expiring judge cache (engine-sql/takes.ts:141-292).
6.3 Synthesis evidence
synthesis_evidence (v037): a synthesis page cites (take_page_id, take_row_num) with a citation_index. Primary key (synthesis_page_id, take_page_id, take_row_num). Derived from markdown by extract takes (system-of-record.md:73).
6.4 Beliefs: the ledger
"Beliefs" in gbrain names the belief ledger, not the takes. It records what the brain had recorded, and when (BELIEF_LEDGER.md section 1).
belief_events (src/core/belief/schema.ts:17-52):
event_key BYTEA UNIQUE,source_id,source_incarnation UUID,page_lifetime UUID,slug,subject_kindinpage|fact|take|link|link_transition|coverage,subject_key,event_type,origininlive|unmanaged|baseline|backfill|reconcile,trust_tier,visibility,publication_id UUID,ord,recorded_at DEFAULT clock_timestamp(),valid_from,valid_until,value JSONB,value_hash,prior_hash,state_hash,cause JSONB,claim_fp,purged BOOLEAN,redacted_fields TEXT[].- Partial index on
claim_fp; indexes on subject, page lifetime, and publication.
Supporting tables: belief_page_lifetimes (partial unique on open lifetimes, schema.ts:54-64), belief_commit_marks, belief_retier_manifest, belief_redactions.
Writes happen in withBeliefBatch (belief/ledger.ts:1-25): every committing transaction records one event per changed logical key, before the terminal receipt. Source rows are read under the held page keys.
Purge and redaction are mirrored into the ledger (belief/purge-mirror.ts, belief/purge.ts).
Withdrawal, purge, and redaction are compared in section 5.4-5.5. Forget keeps the expired row and the withdrawal. Purge keeps only text-free tombstones.
7. Timeline
timeline_entries (schema.sql:645-676):
page_id,date DATE,source TEXT DEFAULT '',summary,detail,event_page_id(FK cascade, for Life Chronicle event projections),created_at.- Unique index
idx_timeline_dedup ON (page_id, date, md5(summary), source). The md5 key exists because a raw long summary overflowed the btree row limit (#3737, schema.sql:657-660). - Partial unique
idx_timeline_event_dedup ON (event_page_id, date) WHERE event_page_id IS NOT NULL.
Writes: addTimelineEntry and batch insert via jsonb_to_recordset with ON CONFLICT (page_id, date, md5(summary), source) DO NOTHING (engine-sql/timeline.ts:38-84).
add_timeline_entryis markdown-first (ops/timeline.ts:45-75). The Markdown Timeline section and inline[Source: ..., YYYY-MM-DD]citations are canonical (system-of-record.md:57-60).- Event projection
upsertEventProjection(engine-sql/timeline.ts:255-268) writes rows withevent_page_id. Reads hide rows whose event page is soft-deleted. last_seen(engine-sql/timeline.ts:225-260): latest dated entry for an entity, considering both direct page matches andevent.whoentries in event frontmatter (jsonb_array_elements_text). Usesdate <= as-of.
8. Tags
tags (schema.sql:616-628): page_id (FK cascade), tag, tag_source (frontmatter | added | NULL legacy), UNIQUE(page_id, tag). Indexes on tag and page_id.
- Frontmatter tags are import-owned: removing one from frontmatter deletes its
frontmatterrow (import-file.ts:828-835). - An explicit add claims the row as
added, so import keeps it (page-state/tags.ts:12-14). Whether the add rewrites frontmatter was not traced. getTags(engine-sql/tags.ts:12-33) is aSELECT DISTINCTover the page's rows.
9. Page versions
page_versions (schema.sql:679-690): page_id, compiled_truth, frontmatter JSONB, snapshot_at. Migrations add timeline, knowledge_revision, title, type, tags, is_deleted, source_path, change_reason, and segmented body references.
- Written on each page write by
createPageVersion(page-state/versions.ts:30-40). - Large pages (over
PAGE_INLINE_MAX_BYTES = 1 MiB, page-size-policy.ts:35) keep only the newestLARGE_PAGE_VERSIONS_KEPT = 5(page-body/limits.ts:125). Older rows are deleted withOFFSET, andmarkPrunedHistoryrecords the gap (versions.ts:35-40).
Rust engine implications
What must be preserved exactly:
- Fact fingerprint bytes:
gbrain_fact_fingerprintnormalization and sha256 hex. Withdrawal and purge tombstones key on it. A different normalizer silently un-withdraws facts on reimport. - Link identity: the five-part unique key with NULL-as-equal semantics, plus
link_source,origin_page_id, andorigin_field. Derived-link replacement is per origin. - Entity resolution order and token rule (
sameEntityName, its noise-token list, the retirement chain of 5 hops,followRetirement). Ambiguous namesakes stay unresolved. whoisexact arms and caps (320 chars, 20 types, 100 candidates) and its ambiguity flag.- Traversal contract: BFS by layer,
(slug, id)ordering,TRAVERSE_WALK_ROW_CAP = 20000,TRAVERSE_PATH_ROW_CAP = 5000,truncatedflags, depth default 5. - Boost constants and formulas (backlink log factor, adjacency 1.05 with min 2 hits, cross-source 1.10, session demote 0.95, top-K 20), and the floor gate.
- Calibration bucketing with exact decimal arithmetic.
- Supersession thresholds, including "uncalibrated model never cosine-supersedes."
- Timeline dedupe key
(page_id, date, summary-hash, source). - Fence formats for facts and takes, including strikethrough encodings (
superseded by #N,forgotten: <reason>).
What can be simplified:
- Recursive CTEs become in-memory BFS over per-source adjacency maps (forward and reverse). Keep the caps and ordering, not the SQL.
pg_advisory_xact_lockonsource:entitybecomes a per-key mutex in the single writer.FOR KEY SHAREandFOR UPDATEbecome write-path serialization, since an embedded store with one writer does not need row locks.- Triggers (
facts_preserve_withdrawal,facts_refuse_purged,takes_refuse_purged,pages_refuse_purged) become invariant checks inside the one write function. They must run in the same transaction as the write. - RLS (enabled on purge and belief tables when the role bypasses RLS) becomes application-level authorization.
- Keep the
near-nameindex in process, and keeppage_versionsretention as is.
Postgres feature dependencies:
| Feature | Used by | Rust replacement |
|---|---|---|
| WITH RECURSIVE, MATERIALIZED, DISTINCT ON | traverse, rank_entities, retirement chains | BFS and fold in code |
pg_trgm %, similarity, word_similarity |
resolver fuzzy arm, similar-pages, takes search | Trigram sets computed in Rust; must reproduce pg_trgm scores |
GIN jsonb @> |
whois email arms, frontmatter lookup | Hash index on normalized email and frontmatter keys |
jsonb_to_recordset |
batch link, takes, timeline inserts | Native batch insert |
NULLS NOT DISTINCT |
links unique key | Sentinel for absent origin |
| partial unique indexes | timeline event dedupe, belief lifetimes, takes | Key-prefixed unique tables |
| HNSW halfvec / vector | facts and takes semantic search | HNSW crate or equivalent, f16 or f32 storage |
| tsvector GIN, ts_rank_cd | facts keyword arm, page FTS | Full-text index; ranking will differ |
| advisory xact locks | facts insert, mentions, writeback | Single-writer or per-key locks |
| FOR UPDATE / FOR KEY SHARE | links, withdrawals, purge | Serialized write path |
| SQL triggers | withdrawal and purge guards | Checks in the write function |
| clock_timestamp(), UUID, BYTEA | belief ledger | Native types |
Risks:
- Trigram scores must match pg_trgm closely, because resolver tie-breaks sort on
similarity DESC, slug ASC. Without a reference implementation, add a golden test set from the TypeScript resolver. ts_rank_cdordering will not match BM25. The facts keyword dedupe arm is only a candidate source, so the risk is a different candidate set near the cutoff.- Calibration bucket boundaries: float arithmetic will misplace boundary weights.
- Authoritative DB-only state (withdrawals, purges, belief events, DB-only facts) has no Markdown backup. The embedded store must be backed up and migrated as the system of record. system-of-record.md:398-440 says rebuild does not recover these.
- The 3-second whole-graph
rank_entitiesfigure is from Postgres 16. In-memory Rust needs its own benchmark, and memory use for 985k edges must be measured. - Derived-link revision checks (
DerivedLinkEndpointChangedError) depend on page revisions. Keep the revision field and check it on publish.
Open questions
- Is pg_trgm at default thresholds (0.3 for
%, 0.6 for<%) in production, or is there a config override? The Rust port needs the exact value. [inferred defaults] - How many facts are DB-only (no fence) in real brains? system-of-record.md says they are not recoverable from Markdown, and the port's backup plan depends on the count.
- Is the belief ledger enabled by default? BELIEF_LEDGER.md says "not built," but src/core/belief/ is about 4,100 lines. Owners must confirm before porting.
- Where is hub dampening (
hubWeight) wired? No call site was traced. - Are
timeline_entriesevent-projection rows (event_page_id) fully derivable from event pages' frontmatter? - Are
calibration_profilespublished snapshots authoritative history, or recomputable views? - Does a tag add with
tag_source='added'write back to frontmatter? page-state/tags.ts was only partly read. - Which code-constant vocabularies must be copied exactly? At least NAME_NOISE_TOKENS (resolve.ts:166) and the correction cue lists in search/saved-facts-match.ts.