Skip to content

Open source Grok Bot alternative, Bring your own agent and subscription: message Claude Code, Codex, Gemini CLI, Cursor, Copilot and 40+ AI agents as bots on your own computer, from iPhone, Mac, Windows, Linux or terminal. Agent-agnostic and free. Also a Muse and Dots alternative. Built in Rust.

feat(host): batch memory keeping and let bots search their chat history

leepokaia3501dc37caf
diff --git a/docs/features/context-and-memory.md b/docs/features/context-and-memory.md
index 8ae6ef6a..16be7eaa 100644
--- a/docs/features/context-and-memory.md
+++ b/docs/features/context-and-memory.md
@@ -35,15 +35,22 @@ its own context and compacts it itself.
## Memory (`host/src/chat/memory.rs`)
- Facts are plain markdown, one `- (YYYY-MM-DD) fact` per line:
- - `profile.md` holds who the user is. All of it goes into the prompt, up to 100 facts.
+ - `profile.md` holds who the user is. All of it goes into the prompt, up to 100 facts. When it grows past
+ 100, the keeper consolidates it to at most 60 (merging duplicates, dropping superseded facts); every fact it
+ drops moves to the log, so nothing is lost.
- `log/YYYY-MM.md` holds dated history, including `[note]` and `[episode]` lines. The newest 30 go into the
prompt, within 4,000 characters.
-- **Keeper:** after a turn ends normally, if the user's message is memorable (not "thanks"/"ok"), a one-shot
- agent of the bot's own harness extracts facts using Grok Bot's extraction prompt
- (`profile:` / `log:` / `note:` / `remove:`).
+- **Keeper:** after a turn ends normally, if the user's message is memorable (not "thanks"/"ok"), the exchange
+ is queued (`memory.unprocessed.<bot>`, so a restart keeps it). Once the bot has been quiet for 5 minutes, or 8
+ exchanges are queued, a one-shot agent of the bot's own harness extracts facts from all of them at once using
+ Grok Bot's extraction prompt (`profile:` / `log:` / `note:` / `remove:`). Running per exchange would buy
+ nothing: the frozen prompt only picks up new facts at the next compaction or session.
- On Claude it runs with a replaced system prompt, no tools, no settings, `persistSession: false` and the
`haiku` model.
- Other harnesses get the instructions inline, in `~/.codync/memory-keeper`.
+- **History search:** the built-in `memory` MCP server (`codync-host mcp memory`) gives the bot
+ `search_history`, a substring search (every word must appear) over its own chat: the user's messages and its
+ final replies, main chat and threads. Whatever the keeper didn't write down can still be found.
- **Episodes:** every 6 remembered exchanges (pending turns in `memory.episode.<bot>`), the keeper writes one
`[episode]` journal sentence.
- **Automatic names** (`chat/naming.rs`): a bot created without a name is called "New Bot" with `autoName` set;
@@ -73,7 +80,8 @@ its own context and compacts it itself.
afterwards.
- A session a harness has deleted (for example, one cleaned up after a month) can't be resumed. The bot
starts fresh, but its memory remains.
-- User-level memory shared across bots, project memory and the daily memory "dreaming" pass from Grok Bot
- are not implemented.
+- User-level memory shared across bots and project memory are not implemented. Consolidation runs only when the
+ profile outgrows the prompt, not as Grok Bot's daily "dreaming" pass.
+- `search_history` covers the bot's own chat, not group chats it took part in.
Reply threads have separate session/context lanes; see [groups and threads](groups-and-threads.md). Group/delegated requests do not become user facts in the memory keeper.
diff --git a/host/src/agent/bot.rs b/host/src/agent/bot.rs
index 0d4ef0ff..14f65f0d 100644
--- a/host/src/agent/bot.rs
+++ b/host/src/agent/bot.rs
@@ -826,6 +826,7 @@ impl Actor {
let builtin = [
Some("connectors"),
Some("team"),
+ Some("memory"),
Some("routines"),
self.cfg.computer.then_some("computer"),
composio.then_some("composio"),
diff --git a/host/src/api/devices.rs b/host/src/api/devices.rs
index f0fdb83c..78f4b837 100644
--- a/host/src/api/devices.rs
+++ b/host/src/api/devices.rs
@@ -34,6 +34,7 @@ const LOCAL_ONLY: &[&str] = &[
"pairing",
"computerCall",
"teamCall",
+ "memoryCall",
"routineCall",
"composioCall",
"connectorTarget",
diff --git a/host/src/api/mod.rs b/host/src/api/mod.rs
index c7ed69c2..5c2723c7 100644
--- a/host/src/api/mod.rs
+++ b/host/src/api/mod.rs
@@ -365,6 +365,9 @@ pub async fn dispatch(hub: &Arc<Hub>, caller: &Caller, method: &str, b: Value) -
)?,
"routineWebhook" => hub.routines.credentials(hub, str_arg(&b, "botId")?, str_arg(&b, "id")?)?,
"routineCall" => crate::routines::call(hub, str_arg(&b, "botId")?, str_arg(&b, "name")?, &b["arguments"])?,
+ "memoryCall" => {
+ crate::chat::memory::call(hub, str_arg(&b, "botId")?, str_arg(&b, "name")?, &b["arguments"]).await?
+ }
"teamCall" => {
crate::chat::team::call(hub, str_arg(&b, "botId")?, str_arg(&b, "name")?, &b["arguments"]).await?
}
diff --git a/host/src/chat/memory.rs b/host/src/chat/memory.rs
index c5128a30..93d4b197 100644
--- a/host/src/chat/memory.rs
+++ b/host/src/chat/memory.rs
@@ -3,10 +3,12 @@
//! every turn) and `log/YYYY-MM.md` (dated history), one `- (YYYY-MM-DD) fact`
//! per line, so the user and the agent can read, grep and edit them.
//!
-//! After each memorable exchange the bot's *keeper* runs a one-shot agent of the
-//! same harness that extracts facts (`profile:` / `log:` / `note:` / `remove:`);
-//! every [`EPISODE_INTERVAL`] exchanges it also writes a one-line `[episode]`
-//! journal entry. The bot sees memory through its frozen prompt (`context`).
+//! Memorable exchanges queue up for the bot's *keeper*, which runs a one-shot agent
+//! of the same harness over them once the bot goes quiet and extracts facts
+//! (`profile:` / `log:` / `note:` / `remove:`); every [`EPISODE_INTERVAL`]
+//! exchanges it also writes a one-line `[episode]` journal entry, and a profile
+//! that outgrows the prompt is consolidated. The bot sees memory through its
+//! frozen prompt (`context`) and searches its full chat with `search_history`.
use crate::LockExt;
use crate::agent::acp::{self, Acp, Incoming};
@@ -28,8 +30,10 @@ const PROFILE_HEADER: &str =
"# About the user\n\n<!-- Enduring facts, one per line as \"- (YYYY-MM-DD) <fact>\". -->\n\n";
const LOG_HEADER: &str = "# Memory log\n\n<!-- Dated facts, one per line as \"- (YYYY-MM-DD) <fact>\". -->\n\n";
-/// Profile facts shown in the prompt.
+/// Profile facts shown in the prompt; past this the keeper consolidates the profile.
const PROFILE_PROMPT_LIMIT: usize = 100;
+/// What a consolidation merges the profile down to, leaving room to grow.
+const PROFILE_TARGET: usize = 60;
/// Log facts shown in the prompt (newest first), within [`RECENT_CHAR_BUDGET`].
pub const RECENT_PROMPT_LIMIT: usize = 30;
const RECENT_CHAR_BUDGET: usize = 4_000;
@@ -42,6 +46,10 @@ pub const EPISODE_INTERVAL: usize = 6;
/// An exchange side is cut to this before it goes to the keeper.
const EXCHANGE_CHARS: usize = 8_000;
const KEEPER_TIMEOUT: Duration = Duration::from_secs(180);
+/// Quiet time after the last exchange before the keeper runs over the queued ones.
+const KEEPER_IDLE: Duration = Duration::from_secs(5 * 60);
+/// Queued exchanges that make the keeper run without waiting.
+const KEEPER_BATCH: usize = 8;
const EPISODE_PREFIX: &str = "[episode] ";
const NOTE_PREFIX: &str = "[note] ";
@@ -152,6 +160,28 @@ impl Memory {
facts
}
+ pub fn profile_facts(&self) -> Vec<Fact> {
+ parse_facts(&read(&self.profile_path()), Kind::Profile, &self.profile_path(), 0)
+ }
+
+ fn log_path(&self, at_ms: i64) -> PathBuf {
+ self.dir.join(LOG_DIR).join(format!("{}.md", &ymd(at_ms)[..7]))
+ }
+
+ /// Replaces the profile with `keep`, moving `demote` to the log first (a crash
+ /// in between leaves a duplicate, never a lost fact).
+ pub fn rewrite_profile(&self, keep: &[(String, i64)], demote: &[(String, i64)]) -> Result<()> {
+ let _g = WRITES.locked();
+ for (content, at) in demote {
+ append(&self.log_path(*at), LOG_HEADER, &normalize(content), *at)?;
+ }
+ let mut body = PROFILE_HEADER.to_owned();
+ for (content, at) in keep {
+ let _ = writeln!(body, "- ({}) {}", ymd(*at), normalize(content));
+ }
+ write_atomic(&self.profile_path(), &body)
+ }
+
/// Records a fact unless an equal one exists. Returns whether it was added.
pub fn add(&self, content: &str, kind: Kind, at_ms: i64) -> Result<bool> {
let content = normalize(content);
@@ -164,15 +194,9 @@ impl Memory {
}
let (path, header) = match kind {
Kind::Profile => (self.profile_path(), PROFILE_HEADER),
- Kind::Log => (self.dir.join(LOG_DIR).join(format!("{}.md", &ymd(at_ms)[..7])), LOG_HEADER),
+ Kind::Log => (self.log_path(at_ms), LOG_HEADER),
};
- let raw = read(&path);
- let mut body = if raw.is_empty() { header.to_owned() } else { raw };
- if !body.ends_with('\n') {
- body.push('\n');
- }
- let _ = writeln!(body, "- ({}) {content}", ymd(at_ms));
- write_atomic(&path, &body)?;
+ append(&path, header, &content, at_ms)?;
Ok(true)
}
@@ -203,6 +227,16 @@ impl Memory {
}
}
+fn append(path: &Path, header: &str, content: &str, at_ms: i64) -> Result<()> {
+ let raw = read(path);
+ let mut body = if raw.is_empty() { header.to_owned() } else { raw };
+ if !body.ends_with('\n') {
+ body.push('\n');
+ }
+ let _ = writeln!(body, "- ({}) {content}", ymd(at_ms));
+ write_atomic(path, &body)
+}
+
fn read(path: &Path) -> String {
std::fs::read_to_string(path).unwrap_or_default()
}
@@ -304,7 +338,7 @@ pub fn render(recall: &Recall, location: &Path) -> (String, bool) {
"Your memory lives in a folder at {}: {PROFILE_FILE} holds who the user is (kept in mind every turn) and {LOG_DIR}/ holds dated history.",
location.display()
),
- "Read or grep those files when you need older facts that are not listed here. Memory is updated automatically after each exchange: when the user asks you to remember or forget something, just confirm it in your reply and it will be recorded.".to_owned(),
+ "Read or grep those files when you need older facts that are not listed here, and use the search_history tool to find what was actually said in past chats. Memory is updated automatically in the background shortly after each conversation: when the user asks you to remember or forget something, just confirm it in your reply and it will be recorded.".to_owned(),
];
if !recall.profile.is_empty() {
lines.push("About the user:".into());
@@ -388,7 +422,7 @@ pub fn is_memorable(user: &str) -> bool {
pub fn extraction_system_prompt() -> String {
[
- "You maintain the long-term memory of a personal assistant. Read the latest exchange and decide what — if anything — is worth remembering for future, unrelated conversations.",
+ "You maintain the long-term memory of a personal assistant. Read the latest exchanges and decide what — if anything — is worth remembering for future, unrelated conversations.",
"",
"Tag each fact you keep with a category:",
"- \"profile\": enduring facts about who the user is and how to work with them — their name and how to address them, role, location, languages, lasting preferences and constraints, and important people or relationships. These are remembered indefinitely.",
@@ -397,7 +431,7 @@ pub fn extraction_system_prompt() -> String {
"",
"Do NOT record one-off request mechanics, what the assistant did this turn, general knowledge, or anything already present in the existing memory list.",
"",
- "If the new exchange updates or contradicts a fact in the existing memory list (e.g. the user moved, changed jobs, or renamed something), drop anything clearly superseded: output a line \"remove: <the exact existing fact text>\" and then add the corrected fact. Only remove facts that appear verbatim in the existing list — never invent removals.",
+ "If a new exchange updates or contradicts a fact in the existing memory list (e.g. the user moved, changed jobs, or renamed something), drop anything clearly superseded: output a line \"remove: <the exact existing fact text>\" and then add the corrected fact. Only remove facts that appear verbatim in the existing list — never invent removals.",
"",
"Write each fact as a self-contained statement, one per line: \"profile: <fact>\", \"log: <fact>\", or \"note: <fact>\" to add (e.g. \"profile: The user's name is Ian\", \"log: Planning a trip to Tokyo in October 2025\"), or \"remove: <existing fact>\" to drop a superseded one.",
"Output exactly NONE (and nothing else) when there is nothing to add or remove.",
@@ -405,14 +439,36 @@ pub fn extraction_system_prompt() -> String {
.join("\n")
}
-pub fn extraction_user_prompt(user: &str, agent: &str, existing: &[String]) -> String {
+pub fn extraction_user_prompt(turns: &[EpisodeTurn], existing: &[String]) -> String {
let existing = if existing.is_empty() {
"(empty)".to_owned()
} else {
existing.iter().map(|m| format!("- {m}")).collect::<Vec<_>>().join("\n")
};
let side = |s: &str| if s.trim().is_empty() { "(no message)".to_owned() } else { s.trim().to_owned() };
- format!("Existing memory:\n{existing}\n\nLatest exchange:\nUser: {}\nAssistant: {}", side(user), side(agent))
+ let body = turns
+ .iter()
+ .map(|t| format!("({})\nUser: {}\nAssistant: {}", ymd(t.ts), side(&t.user), side(&t.agent)))
+ .collect::<Vec<_>>()
+ .join("\n\n");
+ format!("Existing memory:\n{existing}\n\nLatest exchanges, oldest first:\n\n{body}")
+}
+
+pub fn consolidation_system_prompt() -> String {
+ [
+ format!("You maintain the long-term memory of a personal assistant. Its profile of the user (enduring facts about who they are and how to work with them) has grown past {PROFILE_PROMPT_LIMIT} facts and must be merged down to at most {PROFILE_TARGET}."),
+ "Merge duplicates and near-duplicates into one self-contained statement, drop facts clearly superseded by a newer one (each fact shows the date it was learned), and move anything that is history rather than who the user is (projects, events, one-off details) to the log.".to_owned(),
+ "Keep every distinct enduring fact and keep the user's wording where you can. Never invent facts.".to_owned(),
+ "Output one fact per line: \"profile: <fact>\" for the new profile, \"log: <fact>\" for what moves to the log. Output nothing else.".to_owned(),
+ ]
+ .join("\n")
+}
+
+pub fn consolidation_user_prompt(facts: &[Fact]) -> String {
+ let mut facts: Vec<&Fact> = facts.iter().collect();
+ facts.sort_by_key(|f| f.created_at);
+ let body = facts.iter().map(|f| format!("- ({}) {}", ymd(f.created_at), f.content)).collect::<Vec<_>>().join("\n");
+ format!("Profile, oldest first:\n{body}")
}
#[derive(Debug, Default, PartialEq, Eq)]
@@ -582,21 +638,58 @@ pub struct Exchange {
pub at: i64,
}
-/// Starts the bot's keeper: exchanges are remembered one at a time, in the
-/// background, and an unnamed bot is named from them (see `naming`). It stops when the bot actor drops the sender.
+fn unprocessed_key(bot_id: &str) -> String {
+ format!("memory.unprocessed.{bot_id}")
+}
+
+fn unprocessed(store: &Store, bot_id: &str) -> Vec<EpisodeTurn> {
+ store.kv_get(&unprocessed_key(bot_id)).and_then(|v| serde_json::from_str(&v).ok()).unwrap_or_default()
+}
+
+/// Starts the bot's keeper. Exchanges queue up (persisted, so a restart keeps them)
+/// and are remembered together once the bot has been quiet for [`KEEPER_IDLE`] or
+/// [`KEEPER_BATCH`] have piled up: the frozen prompt only picks facts up at the next
+/// compaction or session anyway, and one run over several exchanges sees their
+/// context. An unnamed bot is named right away (see `naming`). It stops when the
+/// bot actor drops the sender; what's queued is picked up by the next keeper.
pub fn spawn_keeper(hub: Arc<Hub>, bot_id: String) -> mpsc::UnboundedSender<Exchange> {
let (tx, mut rx) = mpsc::unbounded_channel::<Exchange>();
tokio::spawn(async move {
- while let Some(x) = rx.recv().await {
- let turn = EpisodeTurn {
- ts: x.at,
- user: acp::truncate(&x.user, EXCHANGE_CHARS),
- agent: acp::truncate(&x.agent, EXCHANGE_CHARS),
+ let mut queued = unprocessed(&hub.store, &bot_id);
+ let idle = tokio::time::sleep(KEEPER_IDLE);
+ tokio::pin!(idle);
+ loop {
+ let flush = tokio::select! {
+ x = rx.recv() => {
+ let Some(x) = x else { break };
+ let turn = EpisodeTurn {
+ ts: x.at,
+ user: acp::truncate(&x.user, EXCHANGE_CHARS),
+ agent: acp::truncate(&x.agent, EXCHANGE_CHARS),
+ };
+ if let Err(e) = crate::chat::naming::observe(&hub, &bot_id, turn.clone()).await {
+ tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "naming the bot failed");
+ }
+ // Re-read: another keeper of this bot (before a restart) may have flushed.
+ queued = unprocessed(&hub.store, &bot_id);
+ queued.push(turn);
+ if let Err(e) = hub.store.kv_set(&unprocessed_key(&bot_id), &serde_json::to_string(&queued).unwrap_or_default()) {
+ tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "couldn't queue an exchange for memory");
+ }
+ idle.as_mut().reset(tokio::time::Instant::now() + KEEPER_IDLE);
+ queued.len() >= KEEPER_BATCH
+ }
+ () = &mut idle, if !queued.is_empty() => true,
};
- if let Err(e) = crate::chat::naming::observe(&hub, &bot_id, turn).await {
- tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "naming the bot failed");
+ if !flush {
+ continue;
+ }
+ let batch = std::mem::take(&mut queued);
+ // Cleared before the run: a failing agent must not retry the same batch forever.
+ if let Err(e) = hub.store.kv_set(&unprocessed_key(&bot_id), "[]") {
+ tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "couldn't clear the memory queue");
}
- if let Err(e) = remember(&hub, &bot_id, x).await {
+ if let Err(e) = remember(&hub, &bot_id, batch).await {
tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "memory keeper failed");
}
}
@@ -604,29 +697,33 @@ pub fn spawn_keeper(hub: Arc<Hub>, bot_id: String) -> mpsc::UnboundedSender<Exch
tx
}
-async fn remember(hub: &Arc<Hub>, bot_id: &str, x: Exchange) -> Result<()> {
+async fn remember(hub: &Arc<Hub>, bot_id: &str, turns: Vec<EpisodeTurn>) -> Result<()> {
let Some(cfg) = hub.store.bot(bot_id)?.filter(|b| !b.deleted).map(|b| b.config) else { return Ok(()) };
- let user = acp::truncate(&x.user, EXCHANGE_CHARS);
- let agent = acp::truncate(&x.agent, EXCHANGE_CHARS);
+ let Some(at) = turns.last().map(|t| t.ts) else { return Ok(()) };
let id = bot_id.to_owned();
- let exchange_text = format!("{user}\n{agent}");
+ let exchange_text = turns.iter().map(|t| format!("{}\n{}", t.user, t.agent)).collect::<Vec<_>>().join("\n");
let existing = tokio::task::spawn_blocking(move || {
Memory::for_bot(&id).map(|mem| existing_for_extraction(&mem, &exchange_text))
})
.await??;
- let raw =
- one_shot(hub, &cfg, &extraction_system_prompt(), &extraction_user_prompt(&user, &agent, &existing)).await?;
+ let raw = one_shot(hub, &cfg, &extraction_system_prompt(), &extraction_user_prompt(&turns, &existing)).await?;
let extraction = parse_extraction(&raw, &existing);
let id = bot_id.to_owned();
- let (added, removed) = tokio::task::spawn_blocking(move || {
- Memory::for_bot(&id).and_then(|mem| apply(&mem, &extraction, &existing, crate::store::now_ms()))
+ let (added, removed, crowded) = tokio::task::spawn_blocking(move || {
+ Memory::for_bot(&id).and_then(|mem| {
+ let (added, removed) = apply(&mem, &extraction, &existing, at)?;
+ Ok((added, removed, mem.profile_facts().len() > PROFILE_PROMPT_LIMIT))
+ })
})
.await??;
- tracing::info!(bot = %bot_id, added, removed, "memory updated");
+ tracing::info!(bot = %bot_id, exchanges = turns.len(), added, removed, "memory updated");
+ if crowded && let Err(e) = consolidate(hub, &cfg).await {
+ tracing::warn!(bot = %bot_id, error = format!("{e:#}"), "consolidating the profile failed");
+ }
let mut pending = pending_episode(&hub.store, bot_id);
- pending.push(EpisodeTurn { ts: x.at, user, agent });
+ pending.extend(turns);
if pending.len() < EPISODE_INTERVAL {
return set_pending_episode(&hub.store, bot_id, &pending);
}
@@ -634,7 +731,7 @@ async fn remember(hub: &Arc<Hub>, bot_id: &str, x: Exchange) -> Result<()> {
let raw = one_shot(hub, &cfg, &episode_system_prompt(&cfg.name), &episode_user_prompt(&cfg.name, &pending)).await?;
let narrative = normalize(&raw);
if !narrative.is_empty() && !narrative.eq_ignore_ascii_case(NONE) {
- let at = pending.last().map_or(x.at, |t| t.ts);
+ let at = pending.last().map_or(at, |t| t.ts);
let id = bot_id.to_owned();
tokio::task::spawn_blocking(move || {
Memory::for_bot(&id).and_then(|mem| mem.add(&format!("{EPISODE_PREFIX}{narrative}"), Kind::Log, at))
@@ -644,6 +741,35 @@ async fn remember(hub: &Arc<Hub>, bot_id: &str, x: Exchange) -> Result<()> {
Ok(())
}
+/// The profile outgrew what the prompt shows: have the keeper merge it down to
+/// [`PROFILE_TARGET`] facts. Nothing is lost: facts it drops move to the log.
+async fn consolidate(hub: &Arc<Hub>, cfg: &BotConfig) -> Result<()> {
+ let id = cfg.id.clone();
+ let facts = tokio::task::spawn_blocking(move || Memory::for_bot(&id).map(|m| m.profile_facts())).await??;
+ let raw = one_shot(hub, cfg, &consolidation_system_prompt(), &consolidation_user_prompt(&facts)).await?;
+ let plan = parse_extraction(&raw, &[]);
+ let keep: Vec<&String> = plan.additions.iter().filter(|(_, k)| *k == Kind::Profile).map(|(c, _)| c).collect();
+ if keep.is_empty() || keep.len() > PROFILE_PROMPT_LIMIT {
+ bail!("the consolidated profile had {} facts", keep.len());
+ }
+ let now = crate::store::now_ms();
+ let dated = |c: &str| facts.iter().find(|f| dedupe_key(&f.content) == dedupe_key(c)).map_or(now, |f| f.created_at);
+ let kept: Vec<(String, i64)> = keep.iter().map(|c| ((*c).clone(), dated(c))).collect();
+ let kept_keys: HashSet<String> = kept.iter().map(|(c, _)| dedupe_key(c)).collect();
+ let mut demoted: Vec<(String, i64)> = facts
+ .iter()
+ .filter(|f| !kept_keys.contains(&dedupe_key(&f.content)))
+ .map(|f| (f.content.clone(), f.created_at))
+ .collect();
+ demoted.extend(plan.additions.iter().filter(|(_, k)| *k == Kind::Log).map(|(c, _)| (c.clone(), now)));
+ let id = cfg.id.clone();
+ let (before, after) = (facts.len(), kept.len());
+ tokio::task::spawn_blocking(move || Memory::for_bot(&id).and_then(|m| m.rewrite_profile(&kept, &demoted)))
+ .await??;
+ tracing::info!(bot = %cfg.id, before, after, "profile consolidated");
+ Ok(())
+}
+
/// Runs one prompt on a throwaway agent of the bot's harness and returns its reply.
/// Claude gets a real system prompt, no tools, no settings and no saved session;
/// other harnesses get the instructions inline.
@@ -731,6 +857,61 @@ async fn collect(acp: &Acp, inc: Option<Incoming>, out: &mut String) -> bool {
}
}
+// MARK: history search (the built-in `memory` MCP server)
+
+const SEARCH_DEFAULT: i64 = 10;
+const SEARCH_MAX: i64 = 30;
+const SEARCH_TERMS: usize = 8;
+const SEARCH_TEXT_CHARS: usize = 1_500;
+
+pub const INSTRUCTIONS: &str = "Use search_history to find what was actually said in your past chats with the user \
+(their messages and your final replies, including threads), beyond what your memory notes kept. \
+Search for distinctive words; every word must appear. Results are newest first.";
+
+pub fn tools() -> Value {
+ json!([{
+ "name": "search_history",
+ "description": "Search your full chat history with the user for messages containing every given word (case-insensitive). Returns dated messages, newest first.",
+ "inputSchema": {
+ "type": "object",
+ "properties": {
+ "query": {"type": "string", "description": "Words that must all appear, separated by spaces."},
+ "limit": {"type": "integer", "description": "Maximum results (default 10, at most 30)."},
+ },
+ "required": ["query"],
+ },
+ "annotations": {"readOnlyHint": true},
+ }])
+}
+
+pub async fn call(hub: &Arc<Hub>, bot_id: &str, name: &str, args: &Value) -> Result<Value> {
+ if name != "search_history" {
+ bail!("unknown memory tool: {name}");
+ }
+ let bot = hub.store.bot(bot_id)?.filter(|b| !b.deleted).ok_or_else(|| anyhow!("unknown bot"))?;
+ let terms: Vec<String> =
+ args["query"].as_str().unwrap_or_default().split_whitespace().take(SEARCH_TERMS).map(str::to_owned).collect();
+ if terms.is_empty() {
+ bail!("query must contain at least one word");
+ }
+ let limit = args["limit"].as_i64().unwrap_or(SEARCH_DEFAULT).clamp(1, SEARCH_MAX);
+ let hub = hub.clone();
+ let id = bot.config.id.clone();
+ let rows = tokio::task::spawn_blocking(move || hub.store.search_messages(&id, &terms, limit)).await??;
+ let results: Vec<Value> = rows
+ .iter()
+ .map(|e| {
+ json!({
+ "date": ymd(e.created_at),
+ "from": if e.kind == crate::store::EntryKind::User.as_str() { "user" } else { "you" },
+ "inThread": e.thread_id.is_some(),
+ "text": acp::truncate(e.data["text"].as_str().unwrap_or_default(), SEARCH_TEXT_CHARS),
+ })
+ })
+ .collect();
+ Ok(json!({"results": results}))
+}
+
/// `memory` API: the facts a bot has, for the Memory screen.
pub fn describe(bot_id: &str) -> Result<Value> {
let mem = Memory::for_bot(bot_id)?;
@@ -796,6 +977,40 @@ mod tests {
assert_eq!(parse_extraction(" none ", &[]), Extraction::default());
}
+ #[test]
+ fn consolidation_moves_dropped_facts_to_the_log() {
+ let mem = temp();
+ let t = parse_ymd("2026-08-01").expect("valid date");
+ for fact in ["Name is Kai", "Is called Kai", "Lives in Taipei"] {
+ assert!(mem.add(fact, Kind::Profile, t).expect("add"));
+ }
+ let now = parse_ymd("2026-09-30").expect("valid date");
+ mem.rewrite_profile(
+ &[("The user's name is Kai".into(), now), ("Lives in Taipei".into(), t)],
+ &[("Name is Kai".into(), t), ("Is called Kai".into(), t)],
+ )
+ .expect("rewrite");
+ let profile: Vec<String> = mem.profile_facts().into_iter().map(|f| f.content).collect();
+ assert_eq!(profile, ["The user's name is Kai", "Lives in Taipei"]);
+ let log = read(&mem.dir.join("log/2026-08.md"));
+ assert!(log.contains("- (2026-08-01) Name is Kai") && log.contains("- (2026-08-01) Is called Kai"));
+ let _ = std::fs::remove_dir_all(&mem.dir);
+ }
+
+ #[test]
+ fn batched_exchanges_are_dated_in_order() {
+ let t = parse_ymd("2026-09-30").expect("valid date");
+ let turns = [
+ EpisodeTurn { ts: t, user: "I use Swift 6".into(), agent: "Noted".into() },
+ EpisodeTurn { ts: t, user: String::new(), agent: "Done".into() },
+ ];
+ let prompt = extraction_user_prompt(&turns, &[]);
+ assert!(prompt.contains("Existing memory:\n(empty)"));
+ assert!(
+ prompt.contains("(2026-09-30)\nUser: I use Swift 6\nAssistant: Noted\n\n(2026-09-30)\nUser: (no message)")
+ );
+ }
+
#[test]
fn small_talk_is_not_memorable() {
assert!(!is_memorable("thanks!"));
diff --git a/host/src/main.rs b/host/src/main.rs
index 49b06e38..53c01bc8 100644
--- a/host/src/main.rs
+++ b/host/src/main.rs
@@ -204,6 +204,13 @@ enum McpServer {
#[arg(long, default_value_t = service::DEFAULT_PORT)]
port: u16,
},
+ /// Search the bot's own chat history.
+ Memory {
+ #[arg(long)]
+ bot: String,
+ #[arg(long, default_value_t = service::DEFAULT_PORT)]
+ port: u16,
+ },
/// Tools of the apps connected through Composio.
Composio {
#[arg(long)]
@@ -373,6 +380,7 @@ async fn main() -> Result<()> {
Sub::Mcp { server: McpServer::Computer { bot, port } } => mcp::serve(bot, port, mcp::Server::Computer).await,
Sub::Mcp { server: McpServer::Routines { bot, port } } => mcp::serve(bot, port, mcp::Server::Routines).await,
Sub::Mcp { server: McpServer::Team { bot, port } } => mcp::serve(bot, port, mcp::Server::Team).await,
+ Sub::Mcp { server: McpServer::Memory { bot, port } } => mcp::serve(bot, port, mcp::Server::Memory).await,
Sub::Mcp { server: McpServer::Composio { bot, port } } => mcp::serve(bot, port, mcp::Server::Composio).await,
Sub::Mcp { server: McpServer::Local { connector, port } } => mcp::serve_local(connector, port).await,
Sub::Mcp { server: McpServer::Remote { connector, port } } => mcp::serve_remote(connector, port).await,
diff --git a/host/src/mcp.rs b/host/src/mcp.rs
index e0327c24..87fa7761 100644
--- a/host/src/mcp.rs
+++ b/host/src/mcp.rs
@@ -15,6 +15,7 @@ pub enum Server {
Routines,
Computer,
Team,
+ Memory,
Composio,
}
@@ -125,6 +126,7 @@ pub async fn serve(bot: String, port: u16, server: Server) -> Result<()> {
Server::Computer => ("codync-computer", INSTRUCTIONS, tools()),
Server::Routines => ("codync-routines", crate::routines::INSTRUCTIONS, crate::routines::tools()),
Server::Team => ("codync-team", crate::chat::team::INSTRUCTIONS, crate::chat::team::tools()),
+ Server::Memory => ("codync-memory", crate::chat::memory::INSTRUCTIONS, crate::chat::memory::tools()),
Server::Composio => {
("codync-composio", crate::market::composio::INSTRUCTIONS, crate::market::composio::tools())
}
@@ -171,6 +173,7 @@ async fn call(port: u16, token: &str, bot: &str, params: &Value, server: Server)
Server::Routines => ("routineCall", Duration::from_secs(30)),
Server::Computer => ("computerCall", Duration::from_secs(60)),
Server::Team => ("teamCall", crate::chat::team::ASK_TIMEOUT + Duration::from_secs(30)),
+ Server::Memory => ("memoryCall", Duration::from_secs(30)),
Server::Composio => ("composioCall", Duration::from_secs(120)),
};
let res = crate::http()
@@ -187,7 +190,7 @@ async fn call(port: u16, token: &str, bot: &str, params: &Value, server: Server)
Ok(v) if ok => {
return match server {
Server::Computer => json!({"content": v["content"]}),
- Server::Team | Server::Routines | Server::Connectors => {
+ Server::Team | Server::Memory | Server::Routines | Server::Connectors => {
json!({"content": [{"type": "text", "text": v.to_string()}]})
}
Server::Composio => json!({"content": [{"type": "text", "text": v["result"].to_string()}]}),
diff --git a/host/src/store.rs b/host/src/store.rs
index f3e245f0..8d78f571 100644
--- a/host/src/store.rs
+++ b/host/src/store.rs
@@ -3,13 +3,14 @@
//!
//! Calls are synchronous. They're single-row or indexed queries measured in
//! microseconds, so async callers use them directly; the one bulk read (a fresh
-//! client's catch-up) is capped per bot.
+//! client's catch-up) is capped per bot, and the history search runs through `spawn_blocking`.
use crate::LockExt;
use anyhow::Result;
use rusqlite::{Connection, OptionalExtension, params};
use serde::{Deserialize, Serialize};
use serde_json::Value;
+use std::fmt::Write as _;
use std::path::Path;
use std::sync::Mutex;
@@ -689,6 +690,25 @@ impl Store {
Ok(rows)
}
+ /// Chat-visible messages (the user's, final replies) in any of `bot_id`'s lanes that
+ /// contain every term (case-insensitive for ASCII), newest first.
+ // ponytail: substring scan, no index (CJK needs no tokenizer this way); an FTS5 trigram table if it gets slow.
+ pub fn search_messages(&self, bot_id: &str, terms: &[String], limit: i64) -> Result<Vec<Entry>> {
+ let mut sql = format!(
+ "SELECT {ENTRY_COLS} FROM entries WHERE bot_id = ?1 AND
+ (kind = 'user' OR (kind = 'agent' AND json_extract(data, '$.final') = 1))"
+ );
+ for i in 0..terms.len() {
+ let _ = write!(sql, " AND instr(lower(json_extract(data, '$.text')), ?{}) > 0", i + 3);
+ }
+ sql.push_str(" ORDER BY seq DESC LIMIT ?2");
+ let mut args: Vec<rusqlite::types::Value> = vec![bot_id.to_owned().into(), limit.into()];
+ args.extend(terms.iter().map(|t| t.to_lowercase().into()));
+ let c = self.db.locked();
+ let mut st = c.prepare(&sql)?;
+ Ok(st.query_map(rusqlite::params_from_iter(args), row_entry)?.collect::<rusqlite::Result<_>>()?)
+ }
+
/// A thread's newest `limit` entries, oldest first.
pub fn thread(&self, bot_id: &str, root: &str, limit: i64) -> Result<Vec<Entry>> {
let c = self.db.locked();
@@ -933,6 +953,7 @@ impl Store {
#[cfg(test)]
mod tests {
use super::*;
+ use serde_json::json;
fn temp_store() -> Store {
let dir = std::env::temp_dir().join(format!("codync-test-{}", uuid::Uuid::new_v4()));
@@ -940,6 +961,28 @@ mod tests {
Store::open(&dir.join("t.db")).unwrap()
}
+ #[test]
+ fn history_search_matches_every_term_in_chat_messages() {
+ let s = temp_store();
+ let main = Lane::main("b1");
+ s.insert_entry(&main, EntryKind::User, 1, &json!({"text": "我們用 Swift 6 寫 Codync"})).unwrap();
+ s.insert_entry(&main, EntryKind::Agent, 1, &json!({"text": "swift narration", "final": false})).unwrap();
+ s.insert_entry(&main, EntryKind::Agent, 1, &json!({"text": "Swift 6 strict mode is on", "final": true}))
+ .unwrap();
+ s.insert_entry(&Lane::main("b2"), EntryKind::User, 1, &json!({"text": "Swift 6 elsewhere"})).unwrap();
+ let texts = |terms: &[&str]| -> Vec<String> {
+ let terms: Vec<String> = terms.iter().map(|t| (*t).to_owned()).collect();
+ s.search_messages("b1", &terms, 10)
+ .unwrap()
+ .into_iter()
+ .map(|e| e.data["text"].as_str().unwrap().to_owned())
+ .collect()
+ };
+ assert_eq!(texts(&["SWIFT", "6"]), ["Swift 6 strict mode is on", "我們用 Swift 6 寫 Codync"]);
+ assert_eq!(texts(&["codync", "寫"]), ["我們用 Swift 6 寫 Codync"]);
+ assert!(texts(&["narration"]).is_empty());
+ }
+
#[test]
fn rev_sync_and_unread() {
let s = temp_store();