Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
test(evals): pin the published seed's sha256; note harness vs server differences
AkitaOnRails8f417180a82d
diff --git a/evals/README.md b/evals/README.mdindex 8b47de9d..1e2970ac 100644--- a/evals/README.md+++ b/evals/README.md@@ -185,7 +185,17 @@ output with no high-signal term, or past the excerpt cap of a pastedprompt). Where a fact sits is a choice, so this session's`projection_loss` describes the fixture, not real sessions; what itsupports is a before/after comparison. `report.json` records the seed-and the sha256 of the generated JSON next to the numbers.+and the sha256 of the generated JSON next to the numbers; a test pins the+sha256 of seed 1215, so a generator change that alters it has to update the+pin and say so beside any republished number.++Two differences from a live server: fixtures and the generated session reach+the projection without passing through the capture sanitizer (they are+synthetic, and the injection fixture keeps its planted strings on purpose),+and they are projected at the harness's 300,000-character default budget,+not the server's 240,000-character one (100,000 tokens × 3 characters ×+the 0.8 safety margin). Compare runs against each other, not against a+server's prompts.Zero-LLM, no provider:diff --git a/evals/src/generate.rs b/evals/src/generate.rsindex 3f7635f5..fc228d98 100644--- a/evals/src/generate.rs+++ b/evals/src/generate.rs@@ -420,6 +420,19 @@ mod tests {assert_ne!(a, c, "the seed must reach the output");}+ /// Published runs cite the sha256 of `session(1215)`. A change to the+ /// generator (a template, a formatting tweak, the RNG) that alters those+ /// bytes must update this pin and say so next to any republished number,+ /// rather than silently changing what the old hash names.+ #[test]+ fn the_published_seed_keeps_its_sha256() {+ let (_, sha) = dump(&session(DEFAULT_SEED)).unwrap();+ assert_eq!(+ sha,+ "f25fbc436edf70647ff54b910dfa5ab72259e1b45f1c072db63b9e23d01879d4"+ );+ }+#[test]fn every_fact_is_planted_exactly_once_and_nowhere_in_the_prompt() {let fx = session(DEFAULT_SEED);