Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.
TypeScript agentaiai-agentsall-in-onecrmcrm-systememaillinearmcpmessagingnotesnotionofficerustslackslack-alternativestartupstartup-toolstypescriptworkspace
feat(agent-inmem): compact at once for a long paste, and from 60% of the window (#7983)
Wolf Mermelsteinfe5e505d58ec
diff --git a/crates/agent_inmem/src/domain/agent/compaction.rs b/crates/agent_inmem/src/domain/agent/compaction.rsindex 05bc3685b5..5e41723005 100644--- a/crates/agent_inmem/src/domain/agent/compaction.rs+++ b/crates/agent_inmem/src/domain/agent/compaction.rs@@ -8,7 +8,8 @@//! log records it ([`ContextSummaryNotification`]) and it takes the place of//! the entries it covers; whatever was said meanwhile stays as it was. A turn//! waits for its summary only when the conversation would leave it too-//! little room otherwise ([`WAIT_AT`]).+//! little room otherwise ([`WAIT_AT`]), or when its own message is long+//! ([`LONG_MESSAGE_AT`]), so that a paste has room at once.//!//! A summary that fails leaves the history as it was, and the next is not//! tried until the conversation has grown by another [`RETRY_AFTER`] percent@@ -34,18 +35,24 @@ pub(crate) const SUMMARY_MODEL: &str = "google/gemini-3.8-flash";/// The share of the session model's context window, in percent, at which a/// summary starts.-const SUMMARIZE_AT: u64 = 40;+const SUMMARIZE_AT: u64 = 60;/// The share at which a turn waits for its summary: past it, the turn's own/// work could run out of room.const WAIT_AT: u64 = 75;+/// The share from which a message counts as long. A long message that+/// brings the conversation to [`SUMMARIZE_AT`] waits for its summary rather+/// than running on the whole history, so what was pasted has room at once.+const LONG_MESSAGE_AT: u64 = 10;/// The share the latest turns may take and still stay word for word, two/// turns at most.const RETAIN_UP_TO: u64 = 10;/// The share the conversation grows by, after a summary failed, before/// another is tried.const RETRY_AFTER: u64 = 10;-/// The share one message may take.-const MESSAGE_UP_TO: u64 = 25;+/// The share one message may take. A long one compacts the conversation+/// before it runs, so it needs room only beside a summary and the latest+/// turns.+const MESSAGE_UP_TO: u64 = 50;/// The window of a model the catalog has none for.const UNKNOWN_WINDOW: u64 = 128_000;/// Bytes of conversation per token. Low, since tool results are JSON full@@ -177,7 +184,7 @@ pub(super) struct Stopped;/// Before a turn on `prompt`: start a summary if the conversation has grown/// to need one, and wait for it if the conversation would otherwise leave-/// the turn too little room.+/// the turn too little room, or if `prompt` is long.////// A summary already running covers only what was said before it started,/// so once it lands the conversation can still be too long: then one more@@ -212,7 +219,8 @@ pub(super) async fn before_turn(/// Start a summary if the conversation with `prompt` has grown to need one,/// none is being written, and the last did not fail at nearly this size.-/// The summary to wait for when the turn would not fit without it.+/// The summary to wait for when the turn would not fit without it or its+/// message is long.fn start_summary(state: &Arc<AgentState>,connection: &ConnectionTo<Client>,@@ -251,7 +259,8 @@ fn start_summary(start = Some((covered, through, total, summary_cancel, settle));}}- (total >= share(window, WAIT_AT))+ let long = prompt_tokens(prompt) >= share(window, LONG_MESSAGE_AT);+ (long || total >= share(window, WAIT_AT)).then(|| {session.summarizingdiff --git a/crates/agent_inmem/src/domain/agent/test.rs b/crates/agent_inmem/src/domain/agent/test.rsindex 29883159ec..40da0f56fa 100644--- a/crates/agent_inmem/src/domain/agent/test.rs+++ b/crates/agent_inmem/src/domain/agent/test.rs@@ -863,8 +863,8 @@ fn big(bytes: usize) -> String {/// Run the session on `model` from here on. `other-model` is one the/// catalog has no window for, so it gets the 128k-token fallback: a summary-/// starts at 51,200 tokens, a turn waits from 96,000, and one message may-/// take 32,000.+/// starts at 76,800 tokens, a turn waits from 96,000 or on a message of+/// 12,800, and one message may take 64,000.fn use_model(state: &AgentState, model: &str) {state.store@@ -925,65 +925,109 @@ async fn a_long_conversation_is_summarized_in_the_background_while_its_message_rlet engine = Arc::new(CondensingEngine::new());let recorded = with_summaries(Arc::clone(&engine), async |connection, session, state| {use_model(&state, "other-model");- prompt(&connection, &session, &big(90_000)).await;- let first_prompt = match state+ // Six messages of 12,000 tokens each: under a long message, and+ // 72,000 tokens in all, short of a summary.+ for _ in 0..6 {+ prompt(&connection, &session, &big(36_000)).await;+ }+ let fifth_prompt = match state.store.get(&state.session_id).expect("the session").history- .first()+ .get(8){Some(HistoryEntry::User { request_id, .. }) => Some(request_id.clone()),_ => None,};- // 60,000 tokens with this one: the summary starts, and the message+ // 84,000 tokens with this one: the summary starts, and the message// does not wait for it.- prompt(&connection, &session, &big(90_000)).await;+ prompt(&connection, &session, &big(36_000)).await;summary_settled(&state).await;prompt(&connection, &session, "continue").await;let session = state.store.get(&state.session_id).expect("the session");- (session.summarizing.pending.is_none(), first_prompt)+ (session.summarizing.pending.is_none(), fifth_prompt)}).await;let turns = engine.turns.requests();+ let mut whole = Vec::new();+ for _ in 0..6 {+ whole.extend([big(36_000), "ok".to_owned()]);+ }+ whole.push(big(36_000));assert_eq!(- turns[1].messages,- [big(90_000), "ok".to_owned(), big(90_000)],+ turns[6].messages, whole,"the message that started the summary ran on the whole history");let summaries = engine.summaries.requests();assert_eq!(summaries.len(), 1);assert_eq!(summaries[0].model, compaction::SUMMARY_MODEL);assert_eq!(summaries[0].purpose, TurnPurpose::Summary);- assert!(summaries[0].messages[0].contains(&big(90_000)));+ assert!(summaries[0].messages[0].contains(&big(36_000)));assert!(- summaries[0].messages[0].len() < 100_000,- "the latest message is not in its own summary"+ summaries[0].messages[0].len() < 6 * 36_000,+ "the latest turn stays word for word, and the latest message is not in its own summary");assert_eq!(- turns[2].messages,+ turns[7].messages,[crate::domain::session::SUMMARY_PREAMBLE.to_owned(),"SUMMARY".to_owned(),- big(90_000),+ big(36_000),+ "ok".to_owned(),+ big(36_000),"ok".to_owned(),"continue".to_owned()],- "the next message reads the summary in place of the first turn"+ "the next message reads the summary in place of the first five turns");assert_eq!(recorded.summaries.len(), 1, "the log records the summary");assert_eq!(recorded.summaries[0].text, "SUMMARY");- assert_eq!(recorded.summaries[0].replaces, 2);- let (free, first_prompt) = recorded.out;+ assert_eq!(recorded.summaries[0].replaces, 10);+ let (free, fifth_prompt) = recorded.out;assert!(free, "the slot is free once the summary lands");- assert!(first_prompt.is_some());+ assert!(fifth_prompt.is_some());assert_eq!(- recorded.summaries[0].through, first_prompt,+ recorded.summaries[0].through, fifth_prompt,"the log names the summary's last turn by its prompt's request");}+/// A long paste compacts the conversation before it runs, rather than in+/// the background, so it has room at once.+#[tokio::test]+async fn a_long_message_waits_for_the_summary_it_starts() {+ let mut engine = CondensingEngine::new();+ // Long enough that a turn not waiting would run first.+ engine.delay = std::time::Duration::from_millis(200);+ let engine = Arc::new(engine);+ with_summaries(Arc::clone(&engine), async |connection, session, state| {+ use_model(&state, "other-model");+ // 60,000 tokens in messages short of a long one.+ for _ in 0..5 {+ prompt(&connection, &session, &big(36_000)).await;+ }+ // 20,000 tokens, past 12,800: long, and with it the conversation+ // reaches 80,000, past where a summary starts.+ prompt(&connection, &session, &big(60_000)).await;+ })+ .await;++ assert_eq!(engine.summaries.requests().len(), 1);+ assert_eq!(+ engine.turns.requests()[5].messages,+ [+ crate::domain::session::SUMMARY_PREAMBLE.to_owned(),+ "SUMMARY".to_owned(),+ big(36_000),+ "ok".to_owned(),+ big(60_000)+ ],+ "the long message ran on the summary of the first four turns"+ );+}+#[tokio::test]async fn a_turn_waits_for_its_summary_when_the_conversation_would_not_fit_its_model() {let mut engine = CondensingEngine::new();@@ -1063,9 +1107,11 @@ async fn a_failed_summary_is_tried_again_only_once_the_conversation_has_grown()let engine = Arc::new(FailedSummaryEngine::new());let recorded = with_summaries(Arc::clone(&engine), async |connection, session, state| {use_model(&state, "other-model");- prompt(&connection, &session, &big(90_000)).await;- // 60,000 tokens: a summary starts, and fails.- prompt(&connection, &session, &big(90_000)).await;+ for _ in 0..6 {+ prompt(&connection, &session, &big(36_000)).await;+ }+ // 84,000 tokens: a summary starts, and fails.+ prompt(&connection, &session, &big(36_000)).await;summary_settled(&state).await;prompt(&connection, &session, "continue").await;summary_settled(&state).await;@@ -1087,15 +1133,14 @@ async fn a_failed_summary_is_tried_again_only_once_the_conversation_has_grown()"one that grew it by a tenth of the window does");assert!(recorded.summaries.is_empty(), "nothing landed to record");+ let mut whole = Vec::new();+ for _ in 0..7 {+ whole.extend([big(36_000), "noted".to_owned()]);+ }+ whole.push("continue".to_owned());assert_eq!(- engine.turns.requests()[2].messages,- [- big(90_000),- "noted".to_owned(),- big(90_000),- "noted".to_owned(),- "continue".to_owned()- ],+ engine.turns.requests()[7].messages,+ whole,"the message ran on the whole history");}@@ -1138,8 +1183,9 @@ async fn a_summary_that_runs_long_lands_cut_at_its_limit() {let engine = Arc::new(engine);let recorded = with_summaries(Arc::clone(&engine), async |connection, session, state| {use_model(&state, "other-model");- prompt(&connection, &session, &big(90_000)).await;- prompt(&connection, &session, &big(90_000)).await;+ for _ in 0..7 {+ prompt(&connection, &session, &big(36_000)).await;+ }summary_settled(&state).await;}).await;@@ -1217,7 +1263,8 @@ async fn images_count_toward_the_context_and_are_each_read_for_the_summary() {let engine = Arc::new(CondensingEngine::new());with_summaries(Arc::clone(&engine), async |connection, session, state| {use_model(&state, "other-model");- for batch in 0..3 {+ // The fourth brings the conversation to 76,800 tokens.+ for batch in 0..4 {show_images(&connection, &session, batch * 12, 12).await;}summary_settled(&state).await;@@ -1225,14 +1272,14 @@ async fn images_count_toward_the_context_and_are_each_read_for_the_summary() {.await;let reads = engine.reads.requests();- assert_eq!(reads.len(), 24, "one read per image the summary covers");+ assert_eq!(reads.len(), 36, "one read per image the summary covers");assert!(reads.iter().all(|read| read.images.len() == 1&& read.model == compaction::SUMMARY_MODEL&& read.purpose == TurnPurpose::Summary));let summaries = engine.summaries.requests();assert_eq!(summaries.len(), 1);assert!(summaries[0].messages[0].contains("IMAGE photo-0.png: a red square"));- assert!(summaries[0].messages[0].contains("IMAGE photo-23.png: a red square"));+ assert!(summaries[0].messages[0].contains("IMAGE photo-35.png: a red square"));assert!(summaries[0].images.is_empty(),"the summary reads the descriptions, not the images"@@ -1247,37 +1294,38 @@ async fn images_a_failed_summary_read_are_not_read_again() {});with_summaries(Arc::clone(&engine), async |connection, session, state| {use_model(&state, "other-model");- // A summary of the first 24 images reads them all, then fails.- for batch in 0..3 {+ // A summary of the first 36 images reads them all, then fails.+ for batch in 0..4 {show_images(&connection, &session, batch * 12, 12).await;}summary_settled(&state).await;// Nine more grow the conversation past a tenth of the window, and- // the next summary covers 36.- show_images(&connection, &session, 36, 9).await;+ // the next summary covers 48.+ show_images(&connection, &session, 48, 9).await;summary_settled(&state).await;}).await;assert_eq!(engine.reads.requests().len(),- 36,- "the 24, then only the 12 the failed summary had not read"+ 48,+ "the 36, then only the 12 the failed summary had not read");let summaries = engine.summaries.requests();assert_eq!(summaries.len(), 1, "the second summary landed");assert!(summaries[0].messages[0].contains("IMAGE photo-0.png: a red square"));- assert!(summaries[0].messages[0].contains("IMAGE photo-35.png: a red square"));+ assert!(summaries[0].messages[0].contains("IMAGE photo-47.png: a red square"));}#[tokio::test]async fn a_conversation_longer_than_one_summary_call_reads_is_summarized_in_parts() {let engine = Arc::new(CondensingEngine::new());with_summaries(Arc::clone(&engine), async |connection, session, state| {- // Gemini's window: a summary starts at 419,430 tokens, past the- // 400,000 one summary call reads.+ // Gemini's window: a summary starts at 629,145 tokens, and covers+ // the three messages before the fourth, past the 400,000 one summary+ // call reads.use_model(&state, chat::domain::models::FREE_MODEL);- for _ in 0..3 {+ for _ in 0..4 {prompt(&connection, &session, &big(610_000)).await;}summary_settled(&state).await;@@ -1296,16 +1344,16 @@ async fn a_message_is_too_long_by_its_model_s_window() {Arc::clone(&engine),Arc::new(DisabledAiAdmissionService),async |connection, session, state| {- // 33,334 tokens: past a quarter of a 128k window.+ // 66,667 tokens: past half of a 128k window.use_model(&state, "other-model");let error = connection- .send_request(text_prompt(&session, &big(100_000)))+ .send_request(text_prompt(&session, &big(200_000))).block_task().await.expect_err("too long for this model");assert!(format!("{error:?}").contains("This message is too long"));use_model(&state, "anthropic/claude-sonnet-5-5");- prompt(&connection, &session, &big(100_000)).await;+ prompt(&connection, &session, &big(200_000)).await;},).await;diff --git a/crates/agent_inmem/src/outbound/manager/test/summary.rs b/crates/agent_inmem/src/outbound/manager/test/summary.rsindex 1d75c66948..fe78eb449f 100644--- a/crates/agent_inmem/src/outbound/manager/test/summary.rs+++ b/crates/agent_inmem/src/outbound/manager/test/summary.rs@@ -154,9 +154,15 @@ async fn a_restarted_manager_reads_a_summary_in_place_of_the_turns_it_covered().await.expect("the transport should attach");- // The second message starts a summary of the first turn and runs- // without waiting for it.- for (sent, text) in [(1, "a".repeat(90_000)), (2, "b".repeat(90_000))] {+ // The fourth message, short of a long one, brings the conversation to+ // 86,000 tokens: a summary of the first two turns starts, and the+ // message runs without waiting for it.+ for (sent, text) in [+ (1, "a".repeat(150_000)),+ (2, "b".repeat(36_000)),+ (3, "c".repeat(36_000)),+ (4, "d".repeat(36_000)),+ ] {sessions.send_action(id,@@ -208,7 +214,9 @@ async fn a_restarted_manager_reads_a_summary_in_place_of_the_turns_it_covered()[SUMMARY_PREAMBLE,"SUMMARY",- "b x90000",+ "c x36000",+ "ok",+ "d x36000","ok","after the restart"]diff --git a/docs/AGENT_GUIDE/ai-chat.md b/docs/AGENT_GUIDE/ai-chat.mdindex 0eb03801d0..261253859b 100644--- a/docs/AGENT_GUIDE/ai-chat.md+++ b/docs/AGENT_GUIDE/ai-chat.md@@ -983,14 +983,16 @@ the stack's private hostname. This also applies to images in earlier turns whenresuming a conversation after a service restart.A long conversation with the in-memory agent stays within its model's context by-itself. Once it fills 40% of the model's window, a summary of all but the latest+itself. Once it fills 60% of the model's window, a summary of all but the latestturns is written in the background, with each image described, and the agent readsthat summary in place of those turns from then on. The transcript keeps everymessage and shows a "Context compacted" line where the summary landed. The agentdoes not pause to summarize unless the conversation would otherwise not fit its-model, as after switching to a model with a smaller window. To recover an exact+model, as after switching to a model with a smaller window, or the message that+crosses 60% is itself long (a tenth of the window or more), as a large paste is:+that message waits for the summary so it has room at once. To recover an exactdetail from the summarized part, the agent runs a **ContentSearch** step scoped to-its own session. A message that alone takes more than a quarter of the model's+its own session. A message that alone takes more than half of the model'swindow is refused with "This message is too long. Split it into smaller messagesor attach it as a file." `/compact` is an ordinary message to every agent: thetranscript shows it as sent, followed by the agent's reply; a "Context compacted"