Skip to content

Local-first 🧠 memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.

TypeScript

feat: index fastpath, PEM/JWT redaction, vault connection health (v0.10.14) (#29)

Sol Aitken9d9636bfa433
diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json
index bf1ecf1b..c01438b3 100644
--- a/.claude-plugin/plugin.json
+++ b/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.10.12",
+ "version": "0.10.14",
"description": "Plugin-first second brain package for AI agents and humans.",
"author": {
"name": "Open Second Brain contributors"
diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json
index 3085ac75..63e329fe 100644
--- a/.codex-plugin/plugin.json
+++ b/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.10.12",
+ "version": "0.10.14",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"skills": "./skills",
"hooks": "./hooks/hooks.json",
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 88723868..6f694616 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,43 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [0.10.14] - 2026-05-25
+
+Three independent quality-of-life improvements bundled into one
+release: an indexer fastpath that skips reading unchanged files, a
+broader secret redactor that catches PEM private-key blocks and
+JWT-shaped tokens, and a new vault-level connection-health metric
+that surfaces orphan ratio and backlink density in `brain_digest`.
+
+### Added
+
+- `Store.touchDocument(relPath, mtime, size)` - targeted stat update
+ for re-arming the indexer fastpath without overwriting metadata.
+- `DocumentSummary.size` - exposed alongside `mtime` and
+ `contentHash` from `listDocuments()` so the indexer can compare
+ size cheaply before falling back to a content hash.
+- PEM private-key block redaction in `event-log.ts` - state-machine
+ pass that replaces multi-line `-----BEGIN ... PRIVATE KEY-----`
+ blocks with `[REDACTED PRIVATE KEY]`. Covers RSA, EC, DSA,
+ OpenSSH, PGP, and encrypted variants.
+- JWT-shaped token redaction in `event-log.ts` - conservative match
+ on three base64url segments (each at least 4 characters) replaced
+ with `***REDACTED_JWT_<last-4>`. Does not false-positive on
+ semver-style version numbers like `1.2.3`.
+- `DigestJsonConnectionHealth` in `brain_digest` JSON and markdown
+ output - reports `total_nodes`, `linked_nodes`, `orphan_nodes`,
+ `mean_backlinks`, `median_backlinks`, `link_density`. Computed
+ once from the existing backlink index over all preferences and
+ retired entries.
+
+### Changed
+
+- `indexer.ts` short-circuits SHA256 hashing when `mtime` and
+ `size` both match the stored summary, falling back to hash
+ comparison only when stats differ. On a hash-match-but-stat-drift
+ case the indexer calls `touchDocument()` to re-arm the fastpath
+ for the next run.
+
## [0.10.13] - 2026-05-24
Partner integration with [codegraph](https://github.com/colbymchenry/codegraph):
diff --git a/openclaw.plugin.json b/openclaw.plugin.json
index bd0ce65a..9edd601c 100644
--- a/openclaw.plugin.json
+++ b/openclaw.plugin.json
@@ -2,7 +2,7 @@
"id": "open-second-brain",
"name": "Open Second Brain",
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults.",
- "version": "0.10.12",
+ "version": "0.10.14",
"activation": { "onStartup": true },
"skills": ["./skills"],
"contracts": {
diff --git a/package.json b/package.json
index 745d28a5..04f797f6 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.10.12",
+ "version": "0.10.14",
"type": "module",
"private": false,
"description": "Second brain for AI agents using Obsidian-compatible Markdown vaults. Works with Hermes, Claude Code, Codex, and OpenClaw.",
diff --git a/plugin.yaml b/plugin.yaml
index f0a82acd..881394cd 100644
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "0.10.12"
+version: "0.10.14"
description: "Hermes plugin entrypoint for OpenSecondBrain vault health checks and the optional MCP tool server."
author: "OpenSecondBrain contributors"
kind: standalone
diff --git a/plugins/codex/.codex-plugin/plugin.json b/plugins/codex/.codex-plugin/plugin.json
index e48f48f3..28d707b6 100644
--- a/plugins/codex/.codex-plugin/plugin.json
+++ b/plugins/codex/.codex-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "open-second-brain",
- "version": "0.10.12",
+ "version": "0.10.14",
"description": "Plugin-first second brain package for Codex, Hermes, Claude Code, OpenClaw, and other agent runtimes.",
"author": {
"name": "Open Second Brain contributors",
diff --git a/plugins/hermes/plugin.yaml b/plugins/hermes/plugin.yaml
index 1cd2b7f4..60d05577 100644
--- a/plugins/hermes/plugin.yaml
+++ b/plugins/hermes/plugin.yaml
@@ -1,5 +1,5 @@
name: open-second-brain
-version: "0.10.12"
+version: "0.10.14"
description: "Hermes adapter for OpenSecondBrain vault health checks and optional MCP tool server."
author: "OpenSecondBrain contributors"
kind: standalone
diff --git a/pyproject.toml b/pyproject.toml
index 60af9b6b..07b8afb1 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -10,7 +10,7 @@ build-backend = "setuptools.build_meta"
# CLI entry points (those moved to `package.json` `bin`).
[project]
name = "open-second-brain"
-version = "0.10.12"
+version = "0.10.14"
description = "Hermes Python shim for Open Second Brain. Most of the project (CLI, MCP server, OpenClaw plugin) is TypeScript on Bun; see package.json."
readme = "README.md"
requires-python = ">=3.11"
diff --git a/src/core/brain/digest.ts b/src/core/brain/digest.ts
index 6d3f754a..d3a28f46 100644
--- a/src/core/brain/digest.ts
+++ b/src/core/brain/digest.ts
@@ -216,6 +216,21 @@ export interface DigestJsonMergeSuggestion {
readonly jaccard: number;
}
+/**
+ * Vault-level connection-density health metric (v0.10.14).
+ * Surfaces the ratio of linked notes to total notes, orphan count,
+ * and backlink distribution β€” answering whether the vault grows as a
+ * network of ideas or as an "organized graveyard".
+ */
+export interface DigestJsonConnectionHealth {
+ readonly total_nodes: number;
+ readonly linked_nodes: number;
+ readonly orphan_nodes: number;
+ readonly mean_backlinks: number;
+ readonly median_backlinks: number;
+ readonly link_density: number;
+}
+
export interface DigestJson {
readonly schema_version: 1;
readonly generated_at: string;
@@ -243,6 +258,11 @@ export interface DigestJson {
* same `_brain.yaml:active.most_applied_*` settings.
*/
readonly most_applied: DigestJsonMostApplied;
+ /**
+ * Vault-level connection-density health metric (v0.10.14).
+ * Independent of the window β€” reflects current vault state.
+ */
+ readonly connection_health: DigestJsonConnectionHealth;
}
/**
@@ -298,6 +318,7 @@ export function renderDigest(
merge_suggestions: data.merge_suggestions,
agent_summary: data.agent_summary,
most_applied: data.most_applied,
+ connection_health: data.connection_health,
};
return Object.freeze({
content: JSON.stringify(payload, null, 2) + "\n",
@@ -325,6 +346,7 @@ interface DigestData {
readonly merge_suggestions: ReadonlyArray<DigestJsonMergeSuggestion>;
readonly agent_summary: ReadonlyArray<AgentSummaryEntry>;
readonly most_applied: DigestJsonMostApplied;
+ readonly connection_health: DigestJsonConnectionHealth;
}
function collectDigestData(
@@ -481,6 +503,7 @@ function collectDigestData(
merge_suggestions,
agent_summary,
most_applied,
+ connection_health: computeConnectionHealth(vault, preferences),
};
}
@@ -537,6 +560,56 @@ function pickTopReferenced(
}));
}
+/**
+ * Compute vault-level connection-density health metrics from the
+ * backlink index over all preferences and retired entries.
+ */
+function computeConnectionHealth(
+ vault: string,
+ preferences: ReadonlyArray<PreferenceWithPath>,
+): DigestJsonConnectionHealth {
+ const index = buildBacklinkIndex(vault);
+ const allIds = preferences.map(({ pref }) => pref.id);
+ // Also include retired entries.
+ const dirs = brainDirs(vault);
+ if (existsSync(dirs.retired)) {
+ for (const entry of readdirSync(dirs.retired, { withFileTypes: true })) {
+ if (!entry.isFile() || !entry.name.endsWith(".md")) continue;
+ if (!entry.name.startsWith("ret-")) continue;
+ allIds.push(entry.name.replace(/\.md$/, ""));
+ }
+ }
+
+ const totalNodes = allIds.length;
+ const counts = allIds.map((id) => backlinkCount(index, id));
+ const linkedNodes = counts.filter((c) => c > 0).length;
+ const orphanNodes = totalNodes - linkedNodes;
+
+ let meanBacklinks = 0;
+ let medianBacklinks = 0;
+ if (totalNodes > 0) {
+ const sum = counts.reduce((a, b) => a + b, 0);
+ meanBacklinks = Math.round((sum / totalNodes) * 100) / 100;
+ const sorted = [...counts].sort((a, b) => a - b);
+ const mid = Math.floor(sorted.length / 2);
+ medianBacklinks = sorted.length % 2 === 0
+ ? (sorted[mid - 1]! + sorted[mid]!) / 2
+ : sorted[mid]!;
+ }
+ const linkDensity = totalNodes > 0
+ ? Math.round((linkedNodes / totalNodes) * 1000) / 1000
+ : 0;
+
+ return Object.freeze({
+ total_nodes: totalNodes,
+ linked_nodes: linkedNodes,
+ orphan_nodes: orphanNodes,
+ mean_backlinks: meanBacklinks,
+ median_backlinks: medianBacklinks,
+ link_density: linkDensity,
+ });
+}
+
function isEmpty(data: DigestData): boolean {
return (
data.new_unconfirmed.length === 0 &&
@@ -903,6 +976,23 @@ function renderMarkdown(
}
lines.push("");
}
+ // Connection health is always rendered when vault has nodes β€” it
+ // reflects vault-wide structural state, not windowed change.
+ if (data.connection_health.total_nodes > 0) {
+ const ch = data.connection_health;
+ const pct = Math.round(ch.link_density * 100);
+ lines.push(
+ "## Connection health",
+ "",
+ `- Total nodes: ${ch.total_nodes}`,
+ `- Linked (β‰₯1 inbound): ${ch.linked_nodes}`,
+ `- Orphans (0 inbound): ${ch.orphan_nodes}`,
+ `- Mean backlinks: ${ch.mean_backlinks}`,
+ `- Median backlinks: ${ch.median_backlinks}`,
+ `- Link density: ${pct}%`,
+ "",
+ );
+ }
if (data.merge_suggestions.length > 0) {
lines.push(`## Merge suggestions (${data.merge_suggestions.length})`, "");
for (const item of data.merge_suggestions) {
diff --git a/src/core/event-log.ts b/src/core/event-log.ts
index 8159e65e..7745b789 100644
--- a/src/core/event-log.ts
+++ b/src/core/event-log.ts
@@ -19,11 +19,64 @@ const EVENT_RE = /^- (\d{2}:\d{2}) β€” @/;
const DATE_RE = /^(\d{4})\.(\d{2})\.(\d{2})$/;
const TIME_RE = /^(\d{2}):(\d{2})$/;
+/** Match PEM block headers for private-key-like types. */
+const PEM_BEGIN_RE = /-----BEGIN\s+(?:RSA\s+)?(?:EC\s+)?(?:DSA\s+)?(?:OPENSSH\s+)?(?:ENCRYPTED\s+)?(?:PGP\s+)?PRIVATE KEY-----/;
+const PEM_END_RE = /-----END\s+(?:RSA\s+)?(?:EC\s+)?(?:DSA\s+)?(?:OPENSSH\s+)?(?:ENCRYPTED\s+)?(?:PGP\s+)?PRIVATE KEY-----/;
+
+/**
+ * Conservative JWT-shaped token heuristic: three base64url segments
+ * (alphanumeric, `-`, `_`) separated by dots, total length β‰₯ 32 chars.
+ * Excludes common false positives like version numbers (`1.2.3`) by
+ * requiring each segment to be at least 4 characters.
+ */
+const JWT_RE = /\b[a-zA-Z0-9_-]{4,}\.[a-zA-Z0-9_-]{4,}\.[a-zA-Z0-9_-]{4,}\b/g;
+
+/** Redact PEM private-key blocks with a `<REDACTED>` marker. */
+function redactPemBlocks(text: string): string {
+ const lines = text.split("\n");
+ const out: string[] = [];
+ let inside = false;
+ for (const line of lines) {
+ if (!inside) {
+ if (PEM_BEGIN_RE.test(line)) {
+ inside = true;
+ out.push("[REDACTED PRIVATE KEY]");
+ } else {
+ out.push(line);
+ }
+ } else {
+ if (PEM_END_RE.test(line)) {
+ inside = false;
+ // The END line itself is consumed by the redaction marker.
+ }
+ // All interior lines are silently dropped.
+ }
+ }
+ return out.join("\n");
+}
+
+/** Mask standalone JWT-shaped tokens to last 4 chars. */
+function redactJwtTokens(text: string): string {
+ return text.replace(JWT_RE, (match) => {
+ const tail = match.slice(-4);
+ return `***REDACTED_JWT_${tail}`;
+ });
+}
+
/** Replace secret-like value assignments with `[REDACTED]`. */
-export function redactText(text: string): string {
+function redactSecretAssignments(text: string): string {
return text.replace(SECRET_ASSIGNMENT_RE, (_match, field, sep) => `${field}${sep}[REDACTED]`);
}
+/**
+ * Multi-pass redaction: PEM blocks β†’ JWT tokens β†’ secret assignments.
+ * PEM runs line-by-line first so multi-line blocks are consumed before
+ * the regex-based passes see the interior base64 data.
+ */
+export function redactText(text: string): string {
+ return redactSecretAssignments(redactJwtTokens(redactPemBlocks(text)));
+}
+
/**
* Current date as `YYYY.MM.DD` in `tz` (or host local).
*
diff --git a/src/core/search/indexer.ts b/src/core/search/indexer.ts
index 1c568807..c8a94e19 100644
--- a/src/core/search/indexer.ts
+++ b/src/core/search/indexer.ts
@@ -141,13 +141,32 @@ async function indexInto(
// present file from the index just because a single read failed.
seen.add(file.relPath);
try {
- const content = readUtf8(file.absPath);
- const contentHash = sha256(content);
const mtimeSec = Math.floor(file.stat.mtimeMs / 1000);
+ const size = file.stat.size;
const prev = existing.get(file.relPath);
- if (!opts?.force && prev && prev.contentHash === contentHash && prev.mtime === mtimeSec) {
+ // Fastpath: if mtime and size both match, skip the full
+ // content read + SHA256 hash. This is the same trade-off
+ // as `make` β€” mtime can theoretically be fooled but the
+ // probability is negligible for vault editing workflows.
+ if (!opts?.force && prev && prev.mtime === mtimeSec && prev.size === size) {
+ stats.unchanged++;
+ opts?.onFile?.({ path: file.relPath, kind: "unchanged" });
+ continue;
+ }
+
+ const content = readUtf8(file.absPath);
+ const contentHash = sha256(content);
+
+ // Fallback: fastpath missed (mtime or size changed). If
+ // the content hash still matches, the file is logically
+ // unchanged β€” update stored mtime/size to re-arm the
+ // fastpath on the next run, but skip chunk/link work.
+ if (!opts?.force && prev && prev.contentHash === contentHash) {
stats.unchanged++;
+ // Re-arm the fastpath: store the current stat so next
+ // run can skip the read entirely.
+ store.touchDocument(file.relPath, mtimeSec, size);
opts?.onFile?.({ path: file.relPath, kind: "unchanged" });
continue;
}
diff --git a/src/core/search/store.ts b/src/core/search/store.ts
index f7847df6..8ec6bee5 100644
--- a/src/core/search/store.ts
+++ b/src/core/search/store.ts
@@ -51,6 +51,7 @@ export interface DocumentSummary {
readonly id: number;
readonly contentHash: string;
readonly mtime: number;
+ readonly size: number;
}
export interface ChunkInput {
@@ -372,13 +373,13 @@ export class Store {
listDocuments(): Map<string, DocumentSummary> {
const rows = this.db
- .query<{ id: number; path: string; content_hash: string; mtime: number }, []>(
- "SELECT id, path, content_hash, mtime FROM documents",
+ .query<{ id: number; path: string; content_hash: string; mtime: number; size: number }, []>(
+ "SELECT id, path, content_hash, mtime, size FROM documents",
)
.all();
const map = new Map<string, DocumentSummary>();
for (const r of rows) {
- map.set(r.path, { id: r.id, contentHash: r.content_hash, mtime: r.mtime });
+ map.set(r.path, { id: r.id, contentHash: r.content_hash, mtime: r.mtime, size: r.size });
}
return map;
}
@@ -425,6 +426,20 @@ export class Store {
this.db.run("DELETE FROM documents WHERE id = ?", [id]);
}
+ /**
+ * Touch a document's mtime and size without changing its title,
+ * hash, or triggering chunk replacement. Used by the indexer's
+ * mtime-fastpath fallback to re-arm the stat cache after a
+ * same-content touch.
+ */
+ touchDocument(path: string, mtime: number, size: number): void {
+ const now = nowIso();
+ this.db.run(
+ "UPDATE documents SET mtime = ?, size = ?, updated_at = ?, indexed_at = ? WHERE path = ?",
+ [mtime, size, now, now, path],
+ );
+ }
+
// ── chunks ─────────────────────────────────────────────────────────────────
getChunksByDocument(documentId: number): ChunkRow[] {
diff --git a/tests/core/event-log.test.ts b/tests/core/event-log.test.ts
index 579f27e7..399f8630 100644
--- a/tests/core/event-log.test.ts
+++ b/tests/core/event-log.test.ts
@@ -27,6 +27,48 @@ describe("redactText", () => {
"api_key=[REDACTED] token: [REDACTED] password = [REDACTED]",
);
});
+
+ test("redacts PEM private-key blocks", () => {
+ const input = `Here is a key:
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpAIBAAKCAQEA0Z3VS5JJcds3xfn/ygWyF8PbnGy0AHB7MhgHcTz6sE2I2yPB
+aFDrBz9vFqU4yL3mW5mG3qFqR1vF0F3F1F1F1F1F1F1F1F1F1F1F1F
+-----END RSA PRIVATE KEY-----
+And some more text.`;
+ const result = redactText(input);
+ expect(result).toContain("[REDACTED PRIVATE KEY]");
+ expect(result).not.toContain("MIIEpA");
+ expect(result).toContain("And some more text.");
+ });
+
+ test("redacts EC private-key blocks", () => {
+ const input = "-----BEGIN EC PRIVATE KEY-----\nsecretstuff\n-----END EC PRIVATE KEY-----";
+ expect(redactText(input)).toBe("[REDACTED PRIVATE KEY]");
+ });
+
+ test("masks JWT-shaped tokens", () => {
+ const jwt = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";
+ const result = redactText(`token is ${jwt}`);
+ expect(result).toContain("***REDACTED_JWT_sw5c");
+ expect(result).not.toContain("eyJhbG");
+ });
+
+ test("does not mask version numbers as JWTs", () => {
+ expect(redactText("version 1.2.3 is fine")).toBe("version 1.2.3 is fine");
+ });
+
+ test("combined: PEM + JWT + secret assignment", () => {
+ const input = `api_key=secret123
+-----BEGIN PRIVATE KEY-----
+base64content
+-----END PRIVATE KEY-----
+bearer eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1c2VyIn0.abc123def456ghi789`;
+ const result = redactText(input);
+ expect(result).toContain("api_key=[REDACTED]");
+ expect(result).toContain("[REDACTED PRIVATE KEY]");
+ expect(result).not.toContain("base64content");
+ expect(result).toContain("***REDACTED_JWT_");
+ });
});
describe("validateEventTime", () => {
diff --git a/tests/core/search/indexer.test.ts b/tests/core/search/indexer.test.ts
index 58c5e981..d293e5e1 100644
--- a/tests/core/search/indexer.test.ts
+++ b/tests/core/search/indexer.test.ts
@@ -163,3 +163,25 @@ test("indexVault tolerates empty vault", async () => {
expect(stats.chunksTotal).toBe(0);
});
+test("mtime+size fastpath skips read; hash fallback detects same-content touch", async () => {
+ writeMd(vault, "a.md", "# Hello World\n\nSome content here.");
+ const cfg = makeConfig({ vault, dbPath });
+
+ const first = await indexVault(cfg);
+ expect(first.added).toBe(1);
+ expect(first.unchanged).toBe(0);
+
+ // Touch the file: change mtime without changing content.
+ // The fastpath should NOT fire (mtime differs) but the hash
+ // fallback should detect unchanged content.
+ const p = join(vault, "a.md");
+ const future = new Date(Date.now() + 100_000);
+ utimesSync(p, future, future);
+
+ const second = await indexVault(cfg);
+ // mtime changed β†’ fastpath miss β†’ read+hash runs β†’ same hash β†’ unchanged
+ expect(second.unchanged).toBe(1);
+ expect(second.updated).toBe(0);
+ expect(second.added).toBe(0);
+});
+