Skip to content

Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors

Rust

fix(profile): only an explicit cross-project scope counts as general

AkitaOnRails142b12d49c1b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2c9359fc..1f5af1b6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
+- Fixed the cross-project profile promoting a single project's "always" /
+ "never" / "by default" sentences as general preferences: only an explicit
+ cross-project scope ("in all my projects", "every project", "everywhere",
+ and the Portuguese equivalents) now waives the `min_projects` threshold,
+ in both the zero-LLM detector and the LLM classifier prompt. (#1148)
- Fixed the scheduled auto-improve tests' intermittent empty log captures
(the same latent flaw #1116 fixed for the hooks checkpoint test): a shared
`warn!` callsite's first-in-process execution on a bare thread caches
diff --git a/crates/ai-memory-consolidate/prompts/profile_classify_system.md b/crates/ai-memory-consolidate/prompts/profile_classify_system.md
index b2a56ccc..547a8573 100644
--- a/crates/ai-memory-consolidate/prompts/profile_classify_system.md
+++ b/crates/ai-memory-consolidate/prompts/profile_classify_system.md
@@ -12,9 +12,10 @@ each one, decide:
correction or habit of the user (what they want done, or never done, or
usually do). A question, a one-off task ("fix this test"), a description
of the code, a complaint, or a fact about the world is not one: keep false.
-- generality: "general" when the user said it holds beyond this project
- ("in all my projects", "always", "by default", "from now on"), "project"
- otherwise.
+- generality: "general" only when the user explicitly scoped the sentence
+ beyond this project ("in all my projects", "every project", "across
+ projects", "everywhere"). A bare "always", "never", "by default" or "from
+ now on" about one file, app or task stays "project".
- category: one of the categories listed in the request.
- statement: the preference as one short imperative line, at most 200
characters, as close to the user's own words as possible. Keep tool,
diff --git a/crates/ai-memory-consolidate/src/profile.rs b/crates/ai-memory-consolidate/src/profile.rs
index e4b12657..454fa52c 100644
--- a/crates/ai-memory-consolidate/src/profile.rs
+++ b/crates/ai-memory-consolidate/src/profile.rs
@@ -88,8 +88,9 @@ const PREFERENCE_MARKERS: &[&str] = &[
"by default",
"i usually",
"in all my projects",
- "in every project",
+ "every project",
"across all my projects",
+ "across projects",
"everywhere",
// Portuguese, in line with the recall router's pt-BR markers.
"sempre",
@@ -107,24 +108,23 @@ const PREFERENCE_MARKERS: &[&str] = &[
"por padrao",
"eu costumo",
"em todos os meus projetos",
+ "em todos os projetos",
"em todo projeto",
];
-/// Words that mark a statement as holding everywhere (design §5.2).
+/// Phrases that mark a statement as holding across projects (design §5.2).
+/// An explicit cross-project scope only: a bare "always" or "by default" is
+/// compatible with one file, app or task ("always run the tests for foo.rs")
+/// and must not waive the `min_projects` bar (#1148).
const GENERAL_MARKERS: &[&str] = &[
- "always",
- "by default",
- "from now on",
"in all my projects",
- "in every project",
+ "every project",
"across all my projects",
+ "across projects",
"everywhere",
- "sempre",
- "por padrão",
- "por padrao",
- "a partir de agora",
- "daqui pra frente",
+ // Portuguese, in line with the recall router's pt-BR markers.
"em todos os meus projetos",
+ "em todos os projetos",
"em todo projeto",
];
@@ -2248,9 +2248,45 @@ mod tests {
"Sempre rode os testes antes do commit."
]
);
- assert_eq!(found[0].generality, ProfileGenerality::General);
- assert_eq!(found[1].generality, ProfileGenerality::Project);
- assert_eq!(found[2].generality, ProfileGenerality::General);
+ // A bare "always" / "sempre" is still a candidate, but not a
+ // cross-project statement: it must earn `min_projects` (#1148).
+ assert!(
+ found
+ .iter()
+ .all(|f| f.generality == ProfileGenerality::Project)
+ );
+ }
+
+ #[test]
+ fn only_an_explicit_cross_project_scope_is_general() {
+ for project_only in [
+ "Always run the tests for foo.rs before pushing.",
+ "Never use the staging database for this campaign.",
+ "By default sign-up stays off in this app.",
+ "From now on use the new endpoint here.",
+ "Sempre rode os testes do módulo de pagamentos.",
+ "Por padrão use o banco local neste app.",
+ ] {
+ assert_eq!(
+ generality_of(project_only),
+ ProfileGenerality::Project,
+ "{project_only}"
+ );
+ }
+ for general in [
+ "In all my projects, write commit messages in English.",
+ "Use pnpm in every project.",
+ "Keep the same lint config across projects.",
+ "By default everywhere, prefer small commits.",
+ "Em todos os meus projetos, escreva commits em inglês.",
+ "A partir de agora em todo projeto use pnpm.",
+ ] {
+ assert_eq!(
+ generality_of(general),
+ ProfileGenerality::General,
+ "{general}"
+ );
+ }
}
#[test]
diff --git a/crates/ai-memory-consolidate/tests/suite/profile_harvest.rs b/crates/ai-memory-consolidate/tests/suite/profile_harvest.rs
index 8680af60..97311c26 100644
--- a/crates/ai-memory-consolidate/tests/suite/profile_harvest.rs
+++ b/crates/ai-memory-consolidate/tests/suite/profile_harvest.rs
@@ -307,6 +307,43 @@ async fn one_project_exceptions_stay_local_but_general_rules_travel() {
assert!(pages[0].1.contains("commit messages in English"));
}
+/// A bare "always" / "never" / "sempre" in one project is a project rule, not
+/// a cross-project statement: it needs `min_projects` projects before it
+/// reaches the profile (#1148). An explicit cross-project scope still travels
+/// from one project (control).
+#[tokio::test]
+async fn a_bare_always_in_one_project_is_not_promoted() {
+ let fx = fixture().await;
+ let alpha = project(&fx, "alpha").await;
+ prompt(
+ &fx,
+ alpha,
+ "Always run the payment tests before you push.",
+ 1,
+ )
+ .await;
+ prompt(&fx, alpha, "Never deploy the campaign page on Fridays.", 2).await;
+ prompt(&fx, alpha, "Sempre rode o lint do módulo de checkout.", 3).await;
+
+ let report = pass(&fx, &single_user()).await;
+ assert!(
+ profile_pages(&fx, "_global").is_empty(),
+ "a one-project 'always' must not become a profile entry: {report:?}"
+ );
+
+ prompt(
+ &fx,
+ alpha,
+ "Em todos os meus projetos, escreva as mensagens de commit em inglês.",
+ 4,
+ )
+ .await;
+ pass(&fx, &single_user()).await;
+ let pages = profile_pages(&fx, "_global");
+ assert_eq!(pages.len(), 1, "{pages:?}");
+ assert!(pages[0].1.contains("commit"), "{pages:?}");
+}
+
/// Tool output that reads like a preference is never harvested; the user's
/// own prompt with the same words is (control).
#[tokio::test]
@@ -363,11 +400,11 @@ async fn an_opted_out_project_is_never_harvested() {
prompt(
&fx,
client,
- "Always deploy with the AcmeCorp internal pipeline.",
+ "Always deploy with the AcmeCorp internal pipeline in every project.",
1,
)
.await;
- prompt(&fx, mine, "Always deploy with fly.io.", 2).await;
+ prompt(&fx, mine, "Always deploy with fly.io in every project.", 2).await;
pass(&fx, &single_user()).await;
let statements = candidate_statements(&fx);
@@ -413,7 +450,7 @@ async fn the_latest_ruling_supersedes_and_the_old_version_stays() {
async fn a_hand_edited_entry_is_never_clobbered() {
let fx = fixture().await;
let alpha = project(&fx, "alpha").await;
- prompt(&fx, alpha, "Always use pnpm.", 1).await;
+ prompt(&fx, alpha, "Always use pnpm in every project.", 1).await;
pass(&fx, &single_user()).await;
let (path, _, body) = profile_pages(&fx, "_global")[0].clone();
let global = ai_memory_store::lookup_global_scope(&fx.store.reader)
@@ -451,7 +488,7 @@ async fn a_hand_edited_entry_is_never_clobbered() {
.unwrap();
let beta = project(&fx, "beta").await;
- prompt(&fx, beta, "Always use pnpm!", 5).await;
+ prompt(&fx, beta, "Always use pnpm in every project!", 5).await;
let report = pass(&fx, &single_user()).await;
assert_eq!(
report.skipped_manual,
@@ -471,7 +508,7 @@ async fn a_hand_edited_entry_is_never_clobbered() {
async fn a_forgotten_entry_is_not_recreated_until_said_again() {
let fx = fixture().await;
let alpha = project(&fx, "alpha").await;
- prompt(&fx, alpha, "Always use pnpm.", 1).await;
+ prompt(&fx, alpha, "Always use pnpm in every project.", 1).await;
pass(&fx, &single_user()).await;
let path = profile_pages(&fx, "_global")[0].0.clone();
let global = ai_memory_store::lookup_global_scope(&fx.store.reader)
@@ -496,7 +533,7 @@ async fn a_forgotten_entry_is_not_recreated_until_said_again() {
// Said again, later than the entry was written.
let beta = project(&fx, "beta").await;
let later = (jiff::Timestamp::now().as_microsecond() - T0) / DAY_US + 1;
- prompt(&fx, beta, "Always use pnpm.", later).await;
+ prompt(&fx, beta, "Always use pnpm in every project.", later).await;
let report = pass(&fx, &single_user()).await;
assert_eq!(report.entries_written, 1, "{report:?}");
assert_eq!(profile_pages(&fx, "_global").len(), 1);
@@ -515,7 +552,7 @@ async fn a_private_profile_never_harvests_another_operator() {
shared,
Some("alice"),
ObservationKind::UserPrompt,
- "Always use pnpm.",
+ "Always use pnpm in every project.",
1,
)
.await;
@@ -524,7 +561,7 @@ async fn a_private_profile_never_harvests_another_operator() {
shared,
Some("bob"),
ObservationKind::UserPrompt,
- "Always use yarn berry.",
+ "Always use yarn berry in every project.",
2,
)
.await;
@@ -576,7 +613,7 @@ async fn a_team_profile_never_harvests_a_restricted_project() {
secret,
Some("alice"),
ObservationKind::UserPrompt,
- "Always use the ProjectZebra vault.",
+ "Always use the ProjectZebra vault in every project.",
1,
)
.await;
@@ -585,7 +622,7 @@ async fn a_team_profile_never_harvests_a_restricted_project() {
open,
Some("alice"),
ObservationKind::UserPrompt,
- "Always use pnpm.",
+ "Always use pnpm in every project.",
2,
)
.await;
@@ -595,7 +632,7 @@ async fn a_team_profile_never_harvests_a_restricted_project() {
open,
Some("bob"),
ObservationKind::UserPrompt,
- "Always use pnpm.",
+ "Always use pnpm in every project.",
3,
)
.await;
@@ -674,7 +711,7 @@ async fn a_personal_profile_admits_one_operators_habit() {
one,
None,
ObservationKind::UserPrompt,
- "Always run the ZebraWipe script before tests.",
+ "Always run the ZebraWipe script before tests in every project.",
1,
)
.await;
@@ -829,7 +866,13 @@ async fn injected_instructions_stay_data() {
async fn a_provider_failure_falls_back_to_the_zero_llm_path() {
let fx = fixture().await;
let alpha = project(&fx, "alpha").await;
- prompt(&fx, alpha, "Always use pnpm for installs.", 1).await;
+ prompt(
+ &fx,
+ alpha,
+ "Always use pnpm for installs in every project.",
+ 1,
+ )
+ .await;
let llm = FakeProfileLlm {
fail: true,
..FakeProfileLlm::default()
@@ -838,7 +881,11 @@ async fn a_provider_failure_falls_back_to_the_zero_llm_path() {
assert!(report.llm_fallbacks >= 1, "{report:?}");
let pages = profile_pages(&fx, "_global");
assert_eq!(pages.len(), 1, "{report:?}");
- assert!(pages[0].1.contains("Always use pnpm for installs."));
+ assert!(
+ pages[0]
+ .1
+ .contains("Always use pnpm for installs in every project.")
+ );
}
/// The core scenario on the LLM path: a habit said loosely in one project is
diff --git a/crates/ai-memory-mcp/tests/suite/admin_profile_routes.rs b/crates/ai-memory-mcp/tests/suite/admin_profile_routes.rs
index f0c0eb50..186b9d80 100644
--- a/crates/ai-memory-mcp/tests/suite/admin_profile_routes.rs
+++ b/crates/ai-memory-mcp/tests/suite/admin_profile_routes.rs
@@ -125,7 +125,7 @@ async fn call(
#[tokio::test]
async fn rebuild_converges_and_review_reports_entries_and_waiting_habits() {
let fx = fixture().await;
- prompt(&fx, "alpha", "Always use pnpm.").await;
+ prompt(&fx, "alpha", "Always use pnpm in every project.").await;
prompt(&fx, "alpha", "I prefer tabs over spaces.").await;
let (status, report) = call(
@@ -221,8 +221,13 @@ async fn review_and_rebuild_are_root_only_on_a_multi_user_server() {
#[tokio::test]
async fn apply_lists_the_lines_for_a_project_and_honours_consume() {
let fx = fixture().await;
- prompt(&fx, "alpha", "Always use pnpm.").await;
- prompt(&fx, "alpha", "Always keep tests beside the code.").await;
+ prompt(&fx, "alpha", "Always use pnpm in every project.").await;
+ prompt(
+ &fx,
+ "alpha",
+ "Always keep tests beside the code in every project.",
+ )
+ .await;
let (status, report) = call(
router(&fx, ProfileSettings::default()),
"POST",
diff --git a/docs/cross-project-profile.md b/docs/cross-project-profile.md
index 0fb1dcf4..f6d0bf85 100644
--- a/docs/cross-project-profile.md
+++ b/docs/cross-project-profile.md
@@ -179,8 +179,11 @@ text an agent merely *saw* cannot become your preference.
**2. Converge.** Statements are grouped by topic across projects. A group
becomes an entry when:
-- you stated it as general ("in all my projects", "always", "by default",
- "from now on"), or
+- you explicitly scoped it beyond the project ("in all my projects",
+ "every project", "across projects", "everywhere", or the Portuguese
+ equivalents). A bare "always", "never", "by default" or "from now on" still
+ makes a candidate, but it can describe one file, app or task, so it has to
+ earn the project threshold below, or
- the same choice shows up in at least `min_projects` projects (default 2).
A habit seen in only one project stays that project's business; `profile
diff --git a/docs/design-cross-project-profile.md b/docs/design-cross-project-profile.md
index 013b7dce..f4d5f60c 100644
--- a/docs/design-cross-project-profile.md
+++ b/docs/design-cross-project-profile.md
@@ -138,8 +138,10 @@ false` is never harvested.
A server job on the writer actor groups candidates across projects by topic.
A candidate becomes (or updates) a profile entry when:
-- the user said it was general ("in all my projects", "always", "by default"),
- or
+- the user explicitly scoped it beyond the project ("in all my projects",
+ "every project", "across projects", "everywhere", or the Portuguese
+ equivalents); a bare "always", "never", "by default" or "from now on" is
+ compatible with one file, app or task and does not count (#1148), or
- the same choice appears in at least `min_projects` distinct projects
(default 2).