An always-on second brain you talk to. Voice notes in Telegram → typed, linked knowledge in your Obsidian vault. Runs 24/7 on the Claude subscription you already have.
Python ai-agentsai-automationclaudeclaude-codeebbinghausknowledge-graphllmmemory-systemobsidianpersonal-assistantpkmproductivitypythonsecond-braintelegram-botvoice-assistant
refactor(pipeline): daily run via session, drop claude -p from process.sh
smixscee53b303125
diff --git a/scripts/process.sh b/scripts/process.shindex 5bf8fb5..b2a22f6 100755--- a/scripts/process.sh+++ b/scripts/process.sh@@ -75,105 +75,17 @@ fiecho "ORIENT: daily=$DAILY_SIZE bytes, handoff=OK, graph=OK"# ── END ORIENT PHASE ──-# MCP timeout for stdio server (default 5 sec is too short)-export MCP_TIMEOUT=30000-export MAX_MCP_OUTPUT_TOKENS=50000--# ── 3-PHASE PIPELINE ──-# Phase 1: CAPTURE (classify entries → JSON)-# Phase 2: EXECUTE (create tasks, save thoughts → JSON)-# Phase 3: REFLECT (generate HTML report, update MEMORY)-# Each phase = fresh Claude context for better quality.--SESSION_DIR="$VAULT_DIR/.session"-mkdir -p "$SESSION_DIR"-CAPTURE_FILE="$SESSION_DIR/capture.json"-EXECUTE_FILE="$SESSION_DIR/execute.json"--# Find yearly goals file (auto-detect year)-YEARLY_GOALS=$(ls "$VAULT_DIR/goals/1-yearly-"*.md 2>/dev/null | tail -1)-YEARLY_GOALS_NAME=$(basename "$YEARLY_GOALS" 2>/dev/null || echo "1-yearly.md")--# MCP config (needed for Phase 2)-MCP_PROMPT="CRITICAL: MCP loads in 10-30 seconds. You are NOT in subprocess.-Algorithm: 1) Call. 2) Error? Wait 10 sec. 3) Call again. 4) Wait 20 sec. 5) Call — GUARANTEED to work.-DO NOT say MCP unavailable. It is available. Just wait and call."--cd "$VAULT_DIR"--# ── Phase 1: CAPTURE ──-echo "=== Phase 1: CAPTURE ==="-CAPTURE=$(claude --print --dangerously-skip-permissions \- -p "Today is $TODAY. Read .claude/skills/dbrain-processor/phases/capture.md and execute Phase 1.-Read daily/$TODAY.md, goals/3-weekly.md, goals/2-monthly.md, goals/$YEARLY_GOALS_NAME.-Classify each entry. Return ONLY JSON." \- 2>&1) || true--# Extract JSON from output (Claude may add text around it)-echo "$CAPTURE" | grep -o '{.*}' | python3 -c "-import sys, json-for line in sys.stdin:- try:- data = json.loads(line.strip())- json.dump(data, sys.stdout, ensure_ascii=False, indent=2)- sys.exit(0)- except: pass-# Fallback: save raw output-sys.stdout.write('{\"error\": \"failed to parse capture output\", \"raw\": \"see capture.log\"}')-" > "$CAPTURE_FILE" 2>/dev/null || echo '{"error": "capture failed"}' > "$CAPTURE_FILE"--echo "Capture saved: $(wc -c < "$CAPTURE_FILE") bytes"--# Check if capture produced valid entries-if grep -q '"error"' "$CAPTURE_FILE"; then- echo "WARN: Capture phase had issues, falling back to monolith mode"- # Fallback to monolith processing- REPORT=$(claude --print --dangerously-skip-permissions \- --mcp-config "$PROJECT_DIR/mcp-config.json" \- -p "Today is $TODAY. Execute daily processing according to dbrain-processor skill.-$MCP_PROMPT" \- 2>&1) || true-else- # ── Phase 2: EXECUTE ──- echo "=== Phase 2: EXECUTE ==="- EXECUTE=$(claude --print --dangerously-skip-permissions \- --mcp-config "$PROJECT_DIR/mcp-config.json" \- -p "Today is $TODAY. Read .claude/skills/dbrain-processor/phases/execute.md and execute Phase 2.-Read .session/capture.json for input data.-Read business/_index.md and projects/_index.md for context.-Create tasks in Todoist, save thoughts, update CRM. Return ONLY JSON.-$MCP_PROMPT" \- 2>&1) || true-- echo "$EXECUTE" | grep -o '{.*}' | python3 -c "-import sys, json-for line in sys.stdin:- try:- data = json.loads(line.strip())- json.dump(data, sys.stdout, ensure_ascii=False, indent=2)- sys.exit(0)- except: pass-sys.stdout.write('{\"error\": \"failed to parse execute output\"}')-" > "$EXECUTE_FILE" 2>/dev/null || echo '{"error": "execute failed"}' > "$EXECUTE_FILE"-- echo "Execute saved: $(wc -c < "$EXECUTE_FILE") bytes"-- # ── Phase 3: REFLECT ──- echo "=== Phase 3: REFLECT ==="- REPORT=$(claude --print --dangerously-skip-permissions \- -p "Today is $TODAY. Read .claude/skills/dbrain-processor/phases/reflect.md and execute Phase 3.-Read .session/capture.json and .session/execute.json for input data.-Read MEMORY.md, .session/handoff.md, .graph/health-history.json.-Generate HTML report, update MEMORY, record observations.-Return ONLY RAW HTML (for Telegram)." \- 2>&1) || true-fi--cd "$PROJECT_DIR"--echo "=== Claude output ==="+# ── PROCESS via the persistent interactive session (NO claude -p) ──+# The 3-phase claude -p pipeline is gone: after 2026-06-15 that bills against+# the Agent SDK credit. We drive the long-lived interactive session instead,+# which stays on the subscription. The Python entrypoint runs the daily+# processing through the shared session and prints the HTML report.+echo "=== Daily processing (interactive session) ==="+REPORT=$(cd "$PROJECT_DIR" && uv run python -m d_brain.pipeline daily 2>&1) || true++echo "=== pipeline output ==="echo "$REPORT"-echo "===================="+echo "======================="# Remove HTML comments (break Telegram HTML parser)REPORT_CLEAN=$(echo "$REPORT" | sed '/<!--/,/-->/d')diff --git a/src/d_brain/pipeline.py b/src/d_brain/pipeline.pynew file mode 100644index 0000000..45caafd--- /dev/null+++ b/src/d_brain/pipeline.py@@ -0,0 +1,47 @@+"""Headless pipeline entrypoint for the daily/weekly cron timers.++Replaces the old `claude --print -p` phases in process.sh: each command runs+through the shared persistent interactive session (subscription billing). The+surrounding shell shim still handles graph rebuild, memory decay, git and+Telegram delivery.++ uv run python -m d_brain.pipeline daily # prints the HTML report+ uv run python -m d_brain.pipeline weekly++Exit code 0 on success (report), 1 otherwise.+"""++import logging+import sys+from typing import Any++logger = logging.getLogger(__name__)+++def run(cmd: str, proc: Any) -> tuple[str, bool]:+ """Run a pipeline command against a processor. Returns (text, ok)."""+ if cmd == "daily":+ res = proc.process_daily()+ elif cmd == "weekly":+ res = proc.generate_weekly()+ else:+ return f"unknown command: {cmd}", False+ if res.get("report"):+ return res["report"], True+ return res.get("error", "pipeline error"), False+++def main() -> None: # pragma: no cover+ logging.basicConfig(level=logging.INFO)+ from d_brain.config import get_settings+ from d_brain.services.runtime import get_processor++ cmd = sys.argv[1] if len(sys.argv) > 1 else "daily"+ proc = get_processor(get_settings())+ text, ok = run(cmd, proc)+ print(text)+ sys.exit(0 if ok else 1)+++if __name__ == "__main__":+ main()diff --git a/tests/test_pipeline.py b/tests/test_pipeline.pynew file mode 100644index 0000000..47e4bd6--- /dev/null+++ b/tests/test_pipeline.py@@ -0,0 +1,41 @@+"""Tests for the daily/weekly pipeline entrypoint (no claude -p)."""++from d_brain.pipeline import run+++class FakeProc:+ def __init__(self, daily=None, weekly=None):+ self._daily = daily or {}+ self._weekly = weekly or {}++ def process_daily(self):+ return self._daily++ def generate_weekly(self):+ return self._weekly+++def test_run_daily_returns_report_and_ok():+ proc = FakeProc(daily={"report": "<b>done</b>", "processed_entries": 1})+ text, ok = run("daily", proc)+ assert ok+ assert text == "<b>done</b>"+++def test_run_daily_error_is_not_ok():+ proc = FakeProc(daily={"error": "no daily file", "processed_entries": 0})+ text, ok = run("daily", proc)+ assert not ok+ assert "no daily file" in text+++def test_run_weekly():+ proc = FakeProc(weekly={"report": "<b>weekly</b>", "processed_entries": 1})+ text, ok = run("weekly", proc)+ assert ok and text == "<b>weekly</b>"+++def test_run_unknown_command():+ text, ok = run("frobnicate", FakeProc())+ assert not ok+ assert "unknown" in text.lower()