Reverse engineer anything with agents, from app behavior down to native binaries.
TypeScript agent-skillsai-agentsbinary-analysisclaude-codeclicodexcordisdecompilerdeveloper-toolsdshdsh-pluginghidrahopperllmmcpmodel-context-protocolreareverse-engineeringstatic-analysistouhou
feat(environment): extend evidence-gated write coverage
Edgar Arakelyan5375ff0f9aad
diff --git a/docs/agent-environments.md b/docs/agent-environments.mdindex 359fa0a4c..4112bd87d 100644--- a/docs/agent-environments.md+++ b/docs/agent-environments.md@@ -424,7 +424,7 @@ Each read tool answers according to its view in the state:| View | Answer || ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error; a body is never invented. |+| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. || `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, or a parameter without a role, replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. || `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. || `recorded` | The recording, as replay answers it. |@@ -433,7 +433,7 @@ A list or document call that names no checked line, including a line recordedafter a write or with other query values, answers from the recording. Everyanswer has the replay shape plus `state`: `{"source": "state", "view": …}`naming the entity, collection or document path, or `{"source": "recording",-"reason": …}` with the view's reason, `no-view`, `unrecorded-request` or `page-exceeded` (a call naming a page of a whole list that the collection has outgrown). A+"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown) or `projection` (a call to an item tool whose recordings asked for some fields only, naming such a parameter for an entity the state holds; the answer is a recording, so it does not show served writes). Astate answer carries the base record's status, content type and `replay`provenance with `truncated: false`; one whose JSON would exceed 256 KiB isrefused as a `resource_constraint` tool error rather than cut. Extraction doesdiff --git a/docs/specs/agent-environments/manifest-view/spec.md b/docs/specs/agent-environments/manifest-view/spec.mdindex 873685bb4..fc2009262 100644--- a/docs/specs/agent-environments/manifest-view/spec.md+++ b/docs/specs/agent-environments/manifest-view/spec.md@@ -37,7 +37,11 @@ manifest are later increments.value anywhere in the result.- List records as pointers (`ordinal`, `method`, `path`, `status`,`content_type`) and excluded groups with reason `other-origin`,- `ui-page-or-asset` or `non-json`.+ `ui-page-or-asset` or `non-json`. A write group (`POST`, `PUT`, `PATCH`,+ `DELETE`) whose every response had no body (a 204 or 205, an empty text or a+ recorded size of zero) and at least one succeeded is kept whatever its content+ type: the empty reply is its answer. A missing body text alone does not count,+ as it may be a body the capture did not keep.- Mark the Evidence inferred whenever any tool or excluded group is templated,derived otherwise; keep the parent's authority.- Admit the capture clone, the inline envelope, every retained pointer, group,diff --git a/docs/specs/agent-environments/state-view/spec.md b/docs/specs/agent-environments/state-view/spec.mdindex 18c0e46f7..24b919820 100644--- a/docs/specs/agent-environments/state-view/spec.md+++ b/docs/specs/agent-environments/state-view/spec.md@@ -29,6 +29,17 @@ later increments.- Prove an item type's identity field from the requested path value itsresponses repeat; otherwise infer it from a fixed list of identity names thatevery listed item carries distinctly, which needs at least one listed item.+ An item read that asked for some fields only (`fields`, `fields[...]`,+ `select`, `$select`) and does not return the item is left out of the proof+ and of the view's checks; the view lists it in `projected_record_ordinals`,+ and serving replays any call to that tool that names a projection parameter+ for an entity the state holds. The write gate also leaves out any read that+ names one and leaves out a field its view requires.+- A field every recording of a view carried is required: serving refuses an+ entity lacking it. Once a successful read the write gate checks, naming no+ projection parameter, shows the application leaves the field out of an entity+ the state holds (APIs that omit empty or false members), no view of the type+ requires it.Never treat an envelope that is itself an entity as a list.- Merge observations into one entity per collection and identity: item views'latest values win over list views', differing views keep their own values,diff --git a/docs/specs/agent-environments/state-writes/spec.md b/docs/specs/agent-environments/state-writes/spec.mdindex 68e37dda6..d1b2b5411 100644--- a/docs/specs/agent-environments/state-writes/spec.md+++ b/docs/specs/agent-environments/state-writes/spec.md@@ -81,7 +81,11 @@ identity the client sends is an echo:format is kept: ISO-8601 with its separator, fraction digits and zone, orepoch seconds or milliseconds.7. **Sequence.** An integer one greater than the largest value of the field in- the collection, else in the type.+ the collection, else in the type. The integer may be written as `-`-separated+ digit groups of one width of at least two digits (an asset number `000-005`;+ not a date, version or address); a served value keeps the recorded value's+ groups and zero padding, and digits that outgrow them lead. A later call must+ record the value spelled that way.8. **Unique.** A UUID-shaped string that no other entity of the type shares.9. **Template.** A string containing a generated value as a delimited token(for example a URL that ends in the new id). It is rebuilt around the newdiff --git a/docs/specs/agent-environments/write-coverage/plan.md b/docs/specs/agent-environments/write-coverage/plan.mdnew file mode 100644index 000000000..d18be7a89--- /dev/null+++ b/docs/specs/agent-environments/write-coverage/plan.md@@ -0,0 +1,12 @@+# Implementation plan++1. **Bodiless writes.** The manifest accumulator tracks, per write group, whether every response was empty, and keeps+ the group when one succeeded.+2. **Omitted fields.** The write simulation records required fields its successful reads show missing, and the+ extracted state's views drop them.+3. **Projected reads.** Item planning sets aside unlocated projected reads and the view records their ordinals; the gate+ skips them, and serving replays projected calls to such a view.+4. **Sequences.** A values helper reads a digit-group value as an integer and fills the next one into its mask; the+ write rules, the gate and serving number sequences through it.+5. **Schema, tests and docs.** The view schema, manifest, gate, serving and unit tests, and the manifest, capture,+ state-view and state-writes docs.diff --git a/docs/specs/agent-environments/write-coverage/spec.md b/docs/specs/agent-environments/write-coverage/spec.mdnew file mode 100644index 000000000..cfcb5094e--- /dev/null+++ b/docs/specs/agent-environments/write-coverage/spec.md@@ -0,0 +1,49 @@+# Wider write coverage++Issue: https://github.com/morluto/rea/issues/4933++## Outcome++Four recording shapes kept write tools out of the manifest or left their writes unserved, although the recordings showed+what the writes did:++- **Bodiless replies.** A write answered `204` with no body (Jellyfin's and Shlink's deletes, Miniflux's and Keycloak's+ updates) had no JSON-like content type, so its group was excluded as `non-json` and never became a tool.+- **Omitted fields.** A field every recording of a view carried was required, so an application that leaves empty or+ false members out of some entities (Go's `omitempty`) made the write gate refuse its own read-backs.+- **Projected reads.** An item read asking for some fields only (Directus's `/items/provisions/2?fields=title`) does not+ repeat the requested id, so it blocked the identity proof of the whole item view, and the view's writes had no state+ target.+- **Digit-group sequences.** A sequence spelled in digit groups (Homebox's asset number `000-005`) was not an integer, so+ a create that numbers it had no rule for it and was contradicted.++## Behaviour++- **Manifest.** A write group (`POST`, `PUT`, `PATCH`, `DELETE`) is kept as a tool when every response had no body (a+ `204` or `205`, an empty text, or a recorded size of zero) and one succeeded. A missing text alone does not count.+- **Omitted fields.** The write gate notes required fields missing from the entities the state holds that successful+ reads it checks return, when the read names no projection parameter: the item whose id the path names, or the listed+ items. No view of the type requires them any more. A read that checks nothing, such as one begun while a modelled+ write to its targets still ran or a page the collection has outgrown, notes none.+ Established omissions apply when classifying subsequent reads and choosing later write observations, not just+ when serializing final views. A complete `fields=*` read is not partial solely for lacking a known optional field.+- **Projected reads.** An item read naming `fields`, `fields[...]`, `select` or `$select` that does not return the item+ is left out of the identity proof and the view's checks. The view lists it in `projected_record_ordinals`, and serving+ replays a call to that tool naming such a parameter, for an entity the state holds, with the reason `projection`;+ the replay shows the recording, not served writes. The write gate neither checks nor learns a write's effect from a+ successful read that names such a parameter and leaves out a field its view requires; a 400, 404 or 410 such a read+ answers is still the absence it checks.+ Identity-free projection evidence after the initial-state cutoff is retained too, without seeding later values+ into initial entities. Failed responses are classified before the projection-replay shortcut, so a projected+ absence can confirm a delete or contradict an unexplained disappearance.+- **Sequences.** A sequence value may be `-`-separated digit groups of one width of at least two digits, not a date,+ version or address. The next value keeps the exemplar's groups and zero padding; digits that outgrow them lead. A+ later create must record its value spelled that way. A held value whose digits outgrew the mask (`100-00` after+ `99-99`) counts only when it is the exemplar's spelling of its number, so a reloaded state numbers on from it.++## Acceptance++- Over the 50 published captures, no write group is excluded as `non-json`, served write tools rise from 35 to 39, and+ no write tool is served on the base and unserved here.+- The re-recorded Homebox capture's create is served.+- Every served write tool of those captures applies.diff --git a/docs/specs/agent-environments/write-coverage/tasks.md b/docs/specs/agent-environments/write-coverage/tasks.mdnew file mode 100644index 000000000..1df7c8418--- /dev/null+++ b/docs/specs/agent-environments/write-coverage/tasks.md@@ -0,0 +1,10 @@+# Tasks++- [x] Keep write groups answered with no body as tools.+- [x] Stop requiring fields a successful read shows the application leaves out.+- [x] Set projected item reads aside from the identity proof and replay projected calls.+- [x] Number sequences spelled in digit groups.+- [x] Add manifest, gate, serving and unit tests, failing on the previous head.+- [x] Document the manifest rule, the view member, the replay reason and the sequence spelling.+- [x] Measure the 50 published captures, the re-recorded Homebox capture and the served smoke.+- [x] Run static, formatting, duplication, documentation and test gates.diff --git a/docs/specs/agent-environments/write-coverage/verification.md b/docs/specs/agent-environments/write-coverage/verification.mdnew file mode 100644index 000000000..f80493df6--- /dev/null+++ b/docs/specs/agent-environments/write-coverage/verification.md@@ -0,0 +1,74 @@+# Verification++Executed on macOS with Node v22.23.1 on this branch, whose base is the write-plan branch's head `ad8f674d`.++## Gates++- `npm run build:cached`, `npm run check:fast` (typecheck and lint, no new warnings in the changed files),+ `npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and+ `npm run mcp-catalog:check`: passed.+- `npm run jscpd`: 3.45% duplicated lines over 3776 files.+- `npx vitest run --project domain --project services`: 7808 passed, 5 pre-existing skips. Two unrelated heavy tests+ (keyed-archive traversal, a 125,000-dependent Evidence graph) passed only with a longer timeout on a loaded machine.+- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve+ and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and+ tool schema validity: 8 files, 79 passed; the analysis-view MCP file needed a longer timeout on the loaded machine.++## Tests++- Manifest: a `204` write with no body is kept as a tool whatever its content type, and a write only redirected is not.+- Gate:+ - a field a read-back leaves out stops being required, and a create lacking it is served only when its own read+ leaves it out too;+ - an item view proves its identity past a read that asked for some fields only, which is left to replay;+ - creates numbered in digit groups are served.+- Serving: a created entity is served without the omitted field while seeded ones keep it; a call naming `fields`+ replays with the reason `projection` while other calls answer from the state; creates take the next number in the+ held value's digit groups.+- Unit: `-`-separated digit groups of one width read as a sequence, and a date, version, address or decimal does not;+ the next value keeps the groups, overflow leading.+- The gate, serving and manifest tests fail on `ad8f674d`.++Review fixes, each with a test that fails on `a4c4a23b`:++- Gate:+ - a read naming `fields`, or of a book the state no longer holds, notes no omission, and a create read back only+ through `?fields=id,name` is not served;+ - an omission an item read shows stops every view of the type requiring the field;+ - a read naming `fields` that leaves a required field out neither resynchronises an entity nor serves as a write's+ observation, so a `204` update read back first through `?fields=id,name` learns its effect from the whole read;+ - a later create whose sequence value serving would spell otherwise (`00-04` or `4` after `000-003`) withholds the+ plan.+- Serving: an empty projection argument replays; a projection of a deleted book answers as missing; a created book+ without the omitted field lists.++## Published captures++Over the 50 published captures, against `ad8f674d`:++- Write groups excluded as `non-json` go from 13 in five captures to none. Directus, Jellyfin, Keycloak, Miniflux and+ Shlink gain those tools, so their manifests change.+- Served write tools go from 35 to 39; none is lost:+ - Directus's provisions item view proves its identity past `GET /items/provisions/2?fields=title`, so its create and+ update are served;+ - Jellyfin's `DELETE /Items/{id}` and Shlink's `DELETE /rest/v3/short-urls/{slug}` are served.+- Airbyte's job item view stops requiring two fields its reads leave out; its verdicts are unchanged.+- Six states change: those five captures and Airbyte.+- A served smoke with manifests rebuilt by each tree applies all 39 served write tools here and all 35 on the base.++## Re-recorded captures++Captures re-recorded with a read-back after each first write and two scratch objects per write tool:++- A Homebox pilot recording numbers assets in digit groups (`000-001`, `000-002`, …). Its create goes from+ `contradicted` to served with a `sequence` rule on `assetId`, and the smoke applies it and the delete.+- Five published environments rebuilt from such recordings (Navidrome, Lychee, Homebox, Directus, Linkding) go from 9+ served write tools to 11, and the smoke applies all 11:+ - Navidrome's playlist views stop requiring `imageAbsent`, which its new playlists leave out, so its playlist update is+ served;+ - Homebox's create is served, as in the pilot recording.+- Jellyfin and Memos, re-recorded and rebuilt end to end with this change (manifest, bundle, smoke and both evaluation+ models on 12 retrieval tasks each), serve no new write:+ - Jellyfin's four bodiless write groups become tools, all withheld. Its playlists are created through+ `POST /Playlists` but read back only through `/Items/{id}`, which stops that item view rendering from the state.+ - Memos has none of the four shapes; it names items `memos/<id>`, which the identity proof does not match.diff --git a/docs/web-network-captures.md b/docs/web-network-captures.mdindex 305bdff1c..a998083c8 100644--- a/docs/web-network-captures.md+++ b/docs/web-network-captures.md@@ -410,7 +410,8 @@ agent environment later serves from retained Evidence. The view reuses theapi-surface grouping, then keeps the groups on the primary origin, the originwith the most JSON-like records (`application/json`, `application/*+json`,`text/plain`, `text/markdown`), whose path starts with `/api/` or `/apis/` or-whose content types are JSON-like. Every other group is listed under+whose content types are JSON-like, and the write groups answered with no body+(a bodiless `204 DELETE`). Every other group is listed under`excluded` with reason `other-origin`, `ui-page-or-asset` or `non-json`, andretained records without a usable request line are counted in a limitation.diff --git a/src/application/environment/environmentStateAnswer.ts b/src/application/environment/environmentStateAnswer.tsindex 134ae8b20..408db4ff0 100644--- a/src/application/environment/environmentStateAnswer.ts+++ b/src/application/environment/environmentStateAnswer.ts@@ -1,7 +1,9 @@+import { environmentToolArguments } from "./environmentTools.js";import { stateBodyJson } from "../../domain/analysisView/EnvironmentStateAccumulator.js";import {collectionMatch,collectionPathOf,+ isProjectionParameter,decodedSegment,allLines,listItems,@@ -63,7 +65,8 @@ export type StateProvenance =| RecordedViewReason| "no-view"| "unrecorded-request"- | "page-exceeded";+ | "page-exceeded"+ | "projection";}| {readonly source: "state";@@ -144,6 +147,18 @@ const fromRecording = (: replayed;};+/** Whether a call names a projection parameter of an item view with projected reads, which the state's whole entities cannot answer. */+const asksProjection = (+ view: StateItemView,+ tool: EnvironmentManifestTool,+ args: ReplayArguments,+): boolean =>+ view.projected_record_ordinals !== undefined &&+ [...environmentToolArguments(tool).query].some(+ ([name, argument]) =>+ isProjectionParameter(name) && args[argument] !== undefined,+ );+/** A recorded body the state renders into, usable exactly when extraction could use it. */export const recordedJson = (exchange: RecordedExchange,@@ -291,6 +306,9 @@ const answerItem = (collection: `/${call.segments.map(({ text }) => text).join("/")}`,id: call.id,});+ // A projection of an entity the state holds replays as the capture's projected reads did.+ if (asksProjection(view, tool, args))+ return fromRecording(source, tool, args, "projection");const shown = shownEntity(tool,view,diff --git a/src/application/environment/environmentStateCoverage.test.ts b/src/application/environment/environmentStateCoverage.test.tsnew file mode 100644index 000000000..ea5240c5c--- /dev/null+++ b/src/application/environment/environmentStateCoverage.test.ts@@ -0,0 +1,127 @@+import { describe, expect, it } from "vitest";+import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js";+import type { TimedCaptureRow } from "../../../tests/fixtures/analysisView.js";++const first = { id: 1, name: "First", detail: "seeded" };+const second = { id: 2, name: "Second", detail: "seeded" };+const initial: readonly TimedCaptureRow[] = [+ ["GET", "/api/books", [first, second]],+ ["GET", "/api/books/1", first],+ ["GET", "/api/books/2", second],+];++describe("projection and omission coverage across writes", () => {+ it("records a late identity-free projection without seeding its values into initial state", () => {+ const renamed = { ...first, name: "Renamed" };+ const books = servedFrom([+ ...initial,+ ["PATCH", "/api/books/1", renamed, 200, { name: "Renamed" }],+ ["GET", "/api/books/1", renamed],+ ["GET", "/api/books/1?fields=name", { name: "Renamed" }],+ ]);+ expect(books.store.state.views.get("get_books_by_id")).toMatchObject({+ shape: "item",+ checked_record_ordinals: [1, 2],+ projected_record_ordinals: [5],+ });+ expect(+ books.store+ .snapshot()+ .rea_state.entities.find((entity) => entity.id === "1")?.fields.name,+ ).toBe("First");+ books.answer("patch_books_by_id", { id: "1", body: { name: "Changed" } });+ expect(+ books.answer("get_books_by_id", { id: "1", fields: "name" }),+ ).toMatchObject({+ body: { name: "Renamed" },+ replay: { record_ordinal: 5 },+ state: { source: "recording", reason: "projection" },+ });+ });++ it("uses established omissions when a full projection is the only update observation", () => {+ const third = { id: 3, name: "Third" };+ const books = servedFrom([+ ...initial,+ ["POST", "/api/books", third, 201, { name: "Third" }],+ ["GET", "/api/books/3", third],+ [+ "PATCH",+ "/api/books/3",+ { ...third, name: "Renamed" },+ 200,+ { name: "Renamed" },+ ],+ ["GET", "/api/books/3?fields=*", { ...third, name: "Renamed" }],+ ]);+ expect(books.store.state.writes.get("patch_books_by_id")).toMatchObject({+ served: true,+ checked_record_ordinals: [6],+ });+ expect(books.store.state.views.get("get_books_by_id")).not.toMatchObject({+ required_fields: expect.arrayContaining(["detail"]),+ });+ books.answer("post_books", { body: { name: "New" } });+ books.answer("patch_books_by_id", { id: "3", body: { name: "Changed" } });+ expect(+ books.answer("get_books_by_id", { id: "3", fields: "*" }),+ ).toMatchObject({+ body: { id: 3, name: "Changed" },+ state: { source: "state" },+ });+ });++ it("uses established omissions when observing a later bodiless update through a full projection", () => {+ const third = { id: 3, name: "Third" };+ const renamed = { ...third, name: "Renamed" };+ const again = { ...third, name: "Again" };+ const books = servedFrom([+ ...initial,+ ["POST", "/api/books", third, 201, { name: "Third" }],+ ["GET", "/api/books/3", third],+ ["PATCH", "/api/books/3", renamed, 200, { name: "Renamed" }],+ ["GET", "/api/books/3", renamed],+ ["PATCH", "/api/books/3", "", 204, { name: "Again" }],+ ["GET", "/api/books/3?fields=*", again],+ ["GET", "/api/books", [first, second, again]],+ ]);+ expect(books.store.state.writes.get("patch_books_by_id")).toMatchObject({+ served: true,+ checked_record_ordinals: [6, 9],+ });+ });++ it("does not hide an absence contradicting a confirmed update behind a projection", () => {+ const renamed = { ...first, name: "Renamed" };+ const books = servedFrom([+ ...initial,+ ["GET", "/api/books/2?fields=name", { name: "Second" }],+ ["PATCH", "/api/books/1", renamed, 200, { name: "Renamed" }],+ ["GET", "/api/books/1", renamed],+ [+ "PATCH",+ "/api/books/1",+ { ...first, name: "Again" },+ 200,+ { name: "Again" },+ ],+ ["GET", "/api/books/1?fields=name", { detail: "Not found." }, 404],+ ]);+ expect(books.store.state.writes.get("patch_books_by_id")).toMatchObject({+ served: false,+ });+ });++ it("confirms a delete through a projected absence without needing a body", () => {+ const books = servedFrom([+ ...initial,+ ["GET", "/api/books/2?fields=name", { name: "Second" }],+ ["DELETE", "/api/books/1", "", 204],+ ["GET", "/api/books/1?fields=name", { detail: "Not found." }, 404],+ ]);+ expect(books.store.state.writes.get("delete_books_by_id")).toMatchObject({+ served: true,+ checked_record_ordinals: [5],+ });+ });+});diff --git a/src/application/environment/environmentStateWriteAnswer.test.ts b/src/application/environment/environmentStateWriteAnswer.test.tsindex e12b2f49e..a04f134da 100644--- a/src/application/environment/environmentStateWriteAnswer.test.ts+++ b/src/application/environment/environmentStateWriteAnswer.test.ts@@ -5,17 +5,23 @@ import {type Served,} from "../../../tests/fixtures/environmentStateServing.js";import {+ ASSET_ROWS,BOOK_INIT_ROWS,BOOK_NOT_FOUND,BOOK_ROWS,DELETE_NOT_FOUND,+ DETAILED_LIST_ROWS,FIRST_BOOK,+ OMITTED_FIELD_ROWS,+ PROJECTED_ROWS,LIST_LINE_ROWS,QUERY_ROWS,SECOND_BOOK,THIRD_BOOK,+ assetBook,book,bookClock,+ detailedBook,bookPage,renamedBook,type Book,@@ -254,6 +260,108 @@ describe("served updates and deletes", () => {});});+describe("wider write coverage", () => {+ it("serves a created book without the field the application leaves out of it, and keeps it on the seeded ones", () => {+ const books = servedFrom(OMITTED_FIELD_ROWS);+ const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;+ expect(created).not.toHaveProperty("detail");+ expect(books.answer("get_books_by_id", { id: "3" }).body).toEqual(created);+ expect(books.answer("get_books_by_id", { id: "1" }).body).toEqual(+ detailedBook(FIRST_BOOK),+ );+ });++ it("replays a call that asks for some fields only, and answers the rest from the state", () => {+ const books = servedFrom(PROJECTED_ROWS);+ expect(+ books.answer("get_books_by_id", { id: "2", fields: "name" }),+ ).toMatchObject({+ body: { name: "Second Book" },+ state: { source: "recording", reason: "projection" },+ });+ expect(books.answer("get_books_by_id", { id: "2" })).toMatchObject({+ body: SECOND_BOOK,+ state: { source: "state", view: "item" },+ });+ });++ it("numbers created books after the largest held asset number, in its digit groups", () => {+ const books = servedFrom(ASSET_ROWS);+ const create = (name: string) =>+ books.answer("post_books", { body: { name } }).body as Book;+ expect(create("Fifth Book")).toMatchObject({ asset: "000-003" });+ expect(create("Sixth Book")).toMatchObject({ asset: "000-004" });+ expect(books.answer("get_books_by_id", { id: "1" }).body).toEqual(+ assetBook(FIRST_BOOK, "000-001"),+ );+ });+});++describe("write coverage limits", () => {+ it("numbers past a mask's width again after the state is saved and loaded", () => {+ const books = servedFrom([+ [+ "GET",+ "/api/books",+ [assetBook(FIRST_BOOK, "99-97"), assetBook(SECOND_BOOK, "99-98")],+ ],+ ["GET", "/api/books/1", assetBook(FIRST_BOOK, "99-97")],+ ["GET", "/api/books/2", assetBook(SECOND_BOOK, "99-98")],+ [+ "POST",+ "/api/books",+ assetBook(THIRD_BOOK, "99-99"),+ 201,+ { name: "Third Book" },+ ],+ ["GET", "/api/books/3", assetBook(THIRD_BOOK, "99-99")],+ ]);+ const create = (name: string) =>+ books.answer("post_books", { body: { name } }).body;+ expect(create("Fifth Book")).toMatchObject({ asset: "99-99" });+ expect(create("Sixth Book")).toMatchObject({ asset: "100-00" });+ // Loading what data/get wrote clears the sequence counter, so the held numbers alone carry it on.+ const { rea_state } = books.store.snapshot();+ expect(books.store.replace({ rea_state }).ok).toBe(true);+ expect(create("Seventh Book")).toMatchObject({ asset: "100-01" });+ });++ it("replays a call naming a projection parameter, even an empty one", () => {+ expect(+ servedFrom(PROJECTED_ROWS).answer("get_books_by_id", {+ id: "2",+ fields: "",+ }),+ ).toMatchObject({ state: { source: "recording", reason: "projection" } });+ });++ it("answers a projection of a book the state no longer holds as a missing one", () => {+ const books = servedFrom([+ ...PROJECTED_ROWS,+ ...BOOK_ROWS.slice(BOOK_INIT_ROWS.length),+ ]);+ books.answer("delete_books_by_id", { id: "2" });+ expect(+ books.answer("get_books_by_id", { id: "2", fields: "name" }),+ ).toMatchObject({+ status: 404,+ body: BOOK_NOT_FOUND,+ state: { source: "state", view: "missing", id: "2" },+ });+ });++ it("lists a created book without the field an item read showed the application leaves out", () => {+ const books = servedFrom(DETAILED_LIST_ROWS);+ const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;+ expect(created).not.toHaveProperty("detail");+ expect(books.answer("get_books").body).toEqual([+ detailedBook(FIRST_BOOK),+ detailedBook(SECOND_BOOK),+ created,+ ]);+ });+});+describe("refusals", () => {it("refuses a body the capture does not support and leaves the state unchanged", () => {const books = servedFrom();@@ -1258,3 +1366,82 @@ describe("paged lines", () => {);});});++describe("omissions on query-backed read-backs", () => {+ it("confirms a create through a filtered list and serves later omissions in that query", () => {+ const alpha = { id: 1, name: "Alpha", detail: "x" };+ const beta = { id: 2, name: "Beta", detail: "x" };+ const gamma = { id: 3, name: "Gamma" };+ const books = servedFrom([+ ["GET", "/api/books", [alpha, beta]],+ ["GET", "/api/books?name=Alpha", [alpha]],+ ["GET", "/api/books/1", alpha],+ ["GET", "/api/books/2", beta],+ ["POST", "/api/books", gamma, 201, { name: "Gamma" }],+ ["GET", "/api/books?name=Gamma", [gamma]],+ ]);+ const created = books.answer("post_books", {+ body: { name: "Delta" },+ }).body;+ expect(created).toMatchObject({ id: 3, name: "Delta" });+ expect(created).not.toHaveProperty("detail");+ expect(books.answer("get_books", { name: "Delta" })).toMatchObject({+ body: [created],+ state: { source: "state", membership: "query", listed: 1 },+ });+ expect(books.answer("get_books", { name: "Alpha" }).body).toEqual([alpha]);+ });+});++describe("own JSON members in update bodies", () => {+ it("accepts constructor added by a later recorded update without inheriting a function kind", () => {+ const alpha = { id: 1, name: "Alpha" };+ const beta = { id: 2, name: "Beta", constructor: "known" };+ const renamed = { ...alpha, name: "Renamed" };+ const extended = { ...renamed, constructor: "stored" };+ const books = servedFrom([+ ["GET", "/api/books", [alpha, beta]],+ ["GET", "/api/books/1", alpha],+ ["GET", "/api/books/2", beta],+ ["PATCH", "/api/books/1", renamed, 200, { name: "Renamed" }],+ ["GET", "/api/books/1", renamed],+ ["PATCH", "/api/books/1", extended, 200, { constructor: "stored" }],+ ["GET", "/api/books/1", extended],+ ]);+ expect(+ books.answer("patch_books_by_id", {+ id: "1",+ body: { constructor: "value" },+ }).body,+ ).toMatchObject({+ id: 1,+ name: "Alpha",+ });+ expect(books.answer("get_books_by_id", { id: "1" }).body).toMatchObject({+ constructor: "value",+ });+ });+});++describe("asset sequence inference", () => {+ it("answers the recorded first create consistently when a held asset has outgrown its mask", () => {+ const alpha = { id: 1, name: "Alpha", asset: "99-98" };+ const beta = { id: 2, name: "Beta", asset: "100-00" };+ const gamma = { id: 3, name: "Gamma", asset: "99-99" };+ const books = servedFrom([+ ["GET", "/api/books", [alpha, beta]],+ ["GET", "/api/books/1", alpha],+ ["GET", "/api/books/2", beta],+ ["POST", "/api/books", gamma, 201, { name: "Gamma" }],+ ["GET", "/api/books/3", gamma],+ ]);+ expect(books.state.writes.get("post_books")?.rules).not.toContainEqual({+ field: "asset",+ rule: "sequence",+ scope: "collection",+ });+ expect(+ books.answer("post_books", { body: { name: "Gamma" } }).body,+ ).toEqual(gamma);+ });+});diff --git a/src/application/environment/environmentStateWriteAnswer.ts b/src/application/environment/environmentStateWriteAnswer.tsindex 98fc683f4..39ac804de 100644--- a/src/application/environment/environmentStateWriteAnswer.ts+++ b/src/application/environment/environmentStateWriteAnswer.ts@@ -128,11 +128,16 @@ const observedEntity = (return isObject(object) ? object : undefined;};-/** One past the largest integer these entities hold in the field, or undefined when none holds one. */-const nextHeld = (entities: readonly StateEntity[], field: string) =>+/** One past the largest integer these entities hold in the field, read as the exemplar spells it, or undefined when none holds one. */+const nextHeld = (+ entities: readonly StateEntity[],+ field: string,+ exemplar: JsonValue,+) =>nextInteger(entities.map((entity) => entity.fields),field,+ exemplar,);/**@@ -197,6 +202,7 @@ const generator = (? [...target.collection, ...startedIn]: [...ofType, ...started],rule.field,+ exemplar,),),),@@ -212,7 +218,11 @@ const generator = (exemplar,sequence(`${request.plan.target ?? ""} ${rule.field} ${target.name}`,- nextHeld([...target.collection, ...startedIn], rule.field),+ nextHeld(+ [...target.collection, ...startedIn],+ rule.field,+ exemplar,+ ),),),);diff --git a/src/domain/analysisView/EnvironmentManifestAccumulator.ts b/src/domain/analysisView/EnvironmentManifestAccumulator.tsindex 5d6991018..8a9900bf2 100644--- a/src/domain/analysisView/EnvironmentManifestAccumulator.ts+++ b/src/domain/analysisView/EnvironmentManifestAccumulator.ts@@ -91,8 +91,36 @@ interface GroupState extends ToolGroup {readonly queryNames: Set<string>;readonly apiPath: boolean;jsonLike: boolean;+ /** Every response of the write group had no body, as a 204 has. */+ bodiless: boolean;}+const responseTextSchema = z.object({+ response: z.object({+ status: z.unknown().optional(),+ content: z+ .object({ text: z.unknown().optional(), size: z.unknown().optional() })+ .nullable()+ .optional(),+ }),+});+/** Whether a record's response had no body: a 204 or 205, an empty text, or a recorded size of zero; a missing text alone may be a body not kept. */+const emptyResponse = (reported: unknown): boolean => {+ const parsed = responseTextSchema.safeParse(reported);+ if (!parsed.success) return false;+ const { status, content } = parsed.data.response;+ return (+ status === 204 ||+ status === 205 ||+ content?.text === "" ||+ ((content?.text === undefined || content.text === null) &&+ content?.size === 0)+ );+};+/** A write group every response of which had no body, at least one of them a success. */+const answeredEmpty = (group: GroupState): boolean =>+ group.bodiless &&+ [...group.statuses].some((status) => status >= 200 && status < 300);const postDataSchema = z.object({request: z.object({postData: z@@ -118,6 +146,8 @@ const ASSET_PREFIXES = ["image/", "font/", "audio/", "video/"];const ASSET_PATH =/\.(?:m?js|css|map|html?|png|jpe?g|gif|svg|webp|ico|woff2?|ttf|otf)$/iu;const WRITE_METHODS = new Set(["POST", "PUT", "PATCH"]);+/** A write whose every response is empty is a tool whatever its content type: the empty reply is its answer. */+const BODILESS_WRITE_METHODS = new Set([...WRITE_METHODS, "DELETE"]);/** Quotes, separators and brackets a four-part composite key adds around its texts. */const KEY_OVERHEAD_CHARS = 32;const UNKNOWNS: Manifest["unknowns"] = [@@ -262,7 +292,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumularoute,queryNames,{ key, group },- bodyShape.value,+ { shape: bodyShape.value, empty: emptyResponse(record.reported) },);} finally {this.releaseTemporary();@@ -304,8 +334,9 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumularoute: RouteTemplate,queryNames: ReadonlySet<string> | undefined,bucket: { readonly key: string; readonly group: GroupState | undefined },- bodyShape: BodyShape | null,+ exchange: { readonly shape: BodyShape | null; readonly empty: boolean },): Result<void, AnalysisError> {+ const bodyShape = exchange.shape;const mime = mediaTypeEssence(line.rawMime);let group = bucket.group;// One retained record pointer, including its projected output.@@ -373,6 +404,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumulabodyShape,apiPath: isApiPath(route.template),jsonLike: false,+ bodiless: BODILESS_WRITE_METHODS.has(line.method),};this.#groups.set(bucket.key, group);} else {@@ -383,6 +415,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula}group.positions.push(position);group.jsonLike ||= jsonLike;+ group.bodiless &&= exchange.empty;if (line.status !== null) group.statuses.add(line.status);if (mime !== null) group.contentTypes.add(mime);for (const name of queryNames ?? []) group.queryNames.add(name);@@ -457,7 +490,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumulaconst reason: Exclusion["reason"] | null =group.origin !== primary? "other-origin"- : group.apiPath || group.jsonLike+ : group.apiPath || group.jsonLike || answeredEmpty(group)? null: isAsset(group)? "ui-page-or-asset"diff --git a/src/domain/analysisView/EnvironmentStateExtraction.ts b/src/domain/analysisView/EnvironmentStateExtraction.tsindex 610ef6e19..a05402a3e 100644--- a/src/domain/analysisView/EnvironmentStateExtraction.ts+++ b/src/domain/analysisView/EnvironmentStateExtraction.ts@@ -22,6 +22,7 @@ import {locateItem,locateList,normalizedPath,+ projectsFields,rootOf,same,scalarText,@@ -230,14 +231,22 @@ export class EnvironmentStateExtraction {return first;}- /** Successful recordings that finished before the first related write began. */- #initRecordings(tool: StateToolFacts, template: string): StateRecordFacts[] {- const limit = this.#initLimit(template);- const before = (record: StateRecordFacts): boolean =>+ /** Initial-state admission is shared with later projection-evidence selection. */+ #beforeInitLimit(+ record: StateRecordFacts,+ limit: StateRecordFacts | undefined,+ ): boolean {+ return (limit === undefined ||(this.#position.get(record.ordinal)! <this.#position.get(limit.ordinal)! &&- (this.#timeline === "record-order" || record.end <= limit.start));+ (this.#timeline === "record-order" || record.end <= limit.start))+ );+ }++ /** Successful recordings that finished before the first related write began. */+ #initRecordings(tool: StateToolFacts, template: string): StateRecordFacts[] {+ const limit = this.#initLimit(template);return tool.record_ordinals.map((ordinal) => this.#byOrdinal.get(ordinal)).filter(@@ -246,7 +255,7 @@ export class EnvironmentStateExtraction {record.status !== null &&record.status >= 200 &&record.status < 300 &&- before(record),+ this.#beforeInitLimit(record, limit),).sort((left, right) =>@@ -276,15 +285,50 @@ export class EnvironmentStateExtraction {candidate: Candidate,pointer: string,recordings: readonly StateRecordFacts[],+ projected: readonly number[] = [],): void {this.#plans.push({...candidate,pointer,recordings,latest: this.#latestPerLine(recordings),+ projected,});}+ /** Later identity-free projections are evidence of replay, never initial entities. Read only those query bodies. */+ #addLateProjections(+ tool: StateToolFacts,+ type: string,+ projected: number[],+ ): void {+ const limit = this.#initLimit(type);+ if (limit === undefined) return;+ for (const ordinal of tool.record_ordinals) {+ const record = this.#byOrdinal.get(ordinal);+ if (+ record === undefined ||+ record.status === null ||+ record.status < 200 ||+ record.status >= 300 ||+ !projectsFields(record.query) ||+ this.#beforeInitLimit(record, limit)+ )+ continue;+ if (+ this.#withBody(+ ordinal,+ (body) =>+ body !== undefined &&+ locateItem(body, lastSegmentValue(record.path)) === undefined,+ )+ ) {+ this.#retain(ENTRY_BYTES);+ projected.push(ordinal);+ }+ }+ }+/** Item tools prove a type's identity field by the path value their bodies repeat. */#planItem(candidate: Candidate): void {const { tool, type } = candidate;@@ -292,6 +336,8 @@ export class EnvironmentStateExtraction {if (recordings === undefined) return;let pointer: string | undefined;let keys: string[] | undefined;+ const kept: StateRecordFacts[] = [];+ const projected: number[] = [];for (const record of recordings) {const located = this.#withBody(record.ordinal, (body) =>body === undefined@@ -302,6 +348,13 @@ export class EnvironmentStateExtraction {this.#recorded(tool, "body-unusable");return;}+ // A read that asked for some fields only may leave the identity out; replay answers it.+ if (located === undefined && projectsFields(record.query)) {+ this.#retain(ENTRY_BYTES);+ projected.push(record.ordinal);+ continue;+ }+ kept.push(record);if (located === undefined ||(pointer ?? located.pointer) !== located.pointer@@ -328,7 +381,11 @@ export class EnvironmentStateExtraction {return;}this.#identity.set(type, { field, basis: "path-parameter" });- this.#addPlan(candidate, pointer, recordings);+ this.#addLateProjections(tool, type, projected);+ projected.sort(+ (left, right) => this.#position.get(left)! - this.#position.get(right)!,+ );+ this.#addPlan(candidate, pointer, kept, projected);}/**diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.test.tsindex b6600afc5..362b96a36 100644--- a/src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts+++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.test.ts@@ -1,3 +1,4 @@+import { OmittedFields } from "./environmentStateWriteOmitted.js";import { checkedLines, servedLine } from "./environmentStateCheckedLines.js";import { describe, expect, it } from "vitest";@@ -19,7 +20,11 @@ import {} from "../../../tests/fixtures/environmentStateWrites.js";import type { JsonValue } from "../jsonValue.js";import { ok } from "../result.js";-import { itemTarget, SimulatedState } from "./environmentStateWriteChecks.js";+import {+ checkItemRead,+ itemTarget,+ SimulatedState,+} from "./environmentStateWriteChecks.js";import {laterObservation,type ObservationSearch,@@ -639,16 +644,23 @@ describe("reads the served answer no longer matches", () => {});});- it("confirms no create whose entity lacks a field every recording of its item view carried", () => {+ it("confirms a create that lacks a field every item recording carried only when its own read leaves the field out too", () => {const detailed = (entry: Book): Book => ({ ...entry, detail: "x" });- const rows: TimedCaptureRow[] = [+ const rows = (readBack: Book): TimedCaptureRow[] => [["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],["GET", "/api/books/1", detailed(FIRST_BOOK)],["GET", "/api/books/2", detailed(SECOND_BOOK)],["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],- ["GET", "/api/books/3", THIRD_BOOK],+ ["GET", "/api/books/3", readBack],];- expect(gated(rows, "post_books")).toMatchObject({ served: false });+ // The application omits the field for this book, so serving need not invent it.+ expect(gated(rows(THIRD_BOOK), "post_books")).toMatchObject({+ served: true,+ });+ // The read shows the field the create's own answer left out: the state cannot render it.+ expect(gated(rows(detailed(THIRD_BOOK)), "post_books")).toMatchObject({+ served: false,+ });});});@@ -2813,3 +2825,82 @@ describe("path fields of a create tied to its collection", () => {});});});++it("withholds a create when a sibling read through an encoded collection reveals an extra change", () => {+ const alpha = { id: 1, name: "Alpha" };+ const beta = { id: 2, name: "Beta" };+ const gamma = { id: 3, name: "Gamma" };+ const delta = { id: 4, name: "Delta" };+ const rows: TimedCaptureRow[] = [+ ["GET", "/api/orgs/%61cme1/books", [alpha]],+ ["GET", "/api/orgs/beta2/books", [beta]],+ ["GET", "/api/orgs/%61cme1/books/1", alpha],+ ["GET", "/api/orgs/beta2/books/2", beta],+ ["POST", "/api/orgs/acme1/books", gamma, 201, { name: "Gamma" }],+ ["GET", "/api/orgs/acme1/books/3", gamma],+ ["GET", "/api/orgs/%61cme1/books/1", { ...alpha, name: "Changed" }],+ ["POST", "/api/orgs/beta2/books", delta, 201, { name: "Delta" }],+ ];+ const state = environmentStateViewOf(timedCaptureEvidence(rows), [+ "sibling-variance",+ ]);+ expect(state.writes.find((plan) => plan.effect === "create")).toMatchObject({+ served: false,+ reason: "side-effect",+ });+});++it("learns omitted fields using the raw ID resolved by the item owner", () => {+ const state = new SimulatedState(+ [+ {+ type: "/api/books",+ collection: "/api/books",+ id: "%63",+ fields: { id: "%63", name: "Created" },+ view_fields: [],+ record_ordinals: [],+ },+ ],+ [],+ );+ const view = {+ tool: "get_books_by_id",+ type: "/api/books",+ shape: "item" as const,+ pointer: "",+ fields: ["id", "name", "detail"],+ required_fields: ["id", "name", "detail"],+ checked_record_ordinals: [0],+ envelope_ordinal: 0,+ };+ const record = {+ ordinal: 1,+ method: "GET",+ path: "/api/books/%63",+ query: undefined,+ status: 200,+ start: 1,+ end: 1,+ };+ const target = itemTarget(state, view.type, record);+ expect(target.id).toBe("%63");+ const read = {+ body: { id: "%63", name: "Created" },+ base: { id: "seed", name: "Seed", detail: "x" },+ volatile: new Set<string>(),+ idField: "id",+ collection: target.collection,+ itemId: target.id,+ };+ const omitted = new OmittedFields(() => {});+ const outcome = omitted.check(state, view, record, read, (loosened) =>+ checkItemRead(state, loosened, record, read),+ );+ expect(outcome).toMatchObject({ pass: true, omitted: ["detail"] });+ if (outcome === undefined) throw new Error("Expected a counted read");+ omitted.keep(view, outcome);+ expect(omitted.views([view])[0]).toMatchObject({+ required_fields: ["id", "name"],+ });+});diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.tsindex 4f59913ad..971e6d9cc 100644--- a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts+++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts@@ -13,6 +13,8 @@ import {type StateView,} from "./environmentStateRules.js";import { updatedEntity } from "./environmentStateWriteApply.js";+import { projectedRead } from "./environmentStateWriteObserve.js";+import { OmittedFields } from "./environmentStateWriteOmitted.js";import {checkDocumentRead,checkItemRead,@@ -92,6 +94,8 @@ export class EnvironmentStateWriteSimulation {readonly #taint: WriteTaint;readonly #touched: TouchedTargets = new Map();readonly #effects = new WriteEffects();+ /** Fields reads show the application leaves out, which serving then does not require. */+ readonly omitted = new OmittedFields((bytes) => this.#input.io.retain(bytes));readonly #lastCheck = new Map<string, number>();/** When the latest modelled write to each target since its last check ended. */readonly #touchedSince: PendingTargets = new Map();@@ -178,8 +182,12 @@ export class EnvironmentStateWriteSimulation {io: this.#input.io,records: this.#input.records,timed: this.#input.timed,- viewOf: (ordinal) =>- this.#input.views.get(this.#toolOf.get(ordinal)?.name ?? ""),+ viewOf: (ordinal) => {+ const view = this.#input.views.get(+ this.#toolOf.get(ordinal)?.name ?? "",+ );+ return view === undefined ? undefined : this.omitted.view(view);+ },idField: (type) => this.#types.get(type)!.id_field,observed: (ordinal) =>this.#observations.set(@@ -311,9 +319,11 @@ export class EnvironmentStateWriteSimulation {checkDocumentRead(this.#state, view.tool, record, body),);const item = view.shape === "item";- const collection = item- ? itemTarget(this.#state, view.type, record).collection- : line!.collection;+ const target = item+ ? itemTarget(this.#state, view.type, record)+ : undefined;+ const collection =+ target === undefined ? line!.collection : target.collection;if (this.#taint.under(collection)) return undefined;if (!item) {const ids = this.#state@@ -336,10 +346,16 @@ export class EnvironmentStateWriteSimulation {idField: this.#types.get(view.type)?.id_field ?? "",volatile: this.#input.volatileOf(view.type),collection,+ ...(target === undefined ? {} : { itemId: target.id }),};- return item- ? checkItemRead(this.#state, view, record, check)- : checkListRead(this.#state, view, record, { ...check, line: line! });+ return this.omitted.check(this.#state, view, record, check, (loosened) =>+ loosened.shape === "item"+ ? checkItemRead(this.#state, loosened, record, check)+ : checkListRead(this.#state, loosened, record, {+ ...check,+ line: line!,+ }),+ );});}@@ -349,10 +365,12 @@ export class EnvironmentStateWriteSimulation {}#read(record: StateRecordFacts, index: number, tool: StateToolFacts): void {- const view = this.#input.views.get(tool.name);+ const original = this.#input.views.get(tool.name);+ if (original === undefined) return;+ const view = this.omitted.view(original);+ if (projectedRead(view, record, this.#input.io)) return;const borrowers = this.#observations.get(record.ordinal);- if (view === undefined || (!borrowers && this.#resynced(view, record)))- return;+ if (!borrowers && this.#resynced(view, record)) return;let outcome = this.#check(view, record);if (outcome === undefined) return;let keys = [...outcome.targets, ...outcome.unheld];@@ -365,6 +383,7 @@ export class EnvironmentStateWriteSimulation {if (blocked(keys)) return;let exercised = exercisedBy(this.#touched, keys, borrowers);if (borrowers !== undefined) {+ this.omitted.keep(view, outcome);if (!outcome.pass)this.#contradictOthers(exercised,@@ -384,6 +403,8 @@ export class EnvironmentStateWriteSimulation {exercised = exercisedBy(this.#touched, keys, borrowers);}}+ // Retain omissions only after the final check's coverage passes the ownership guards.+ this.omitted.keep(view, outcome);const interferes = interference(this.#history, outcome);const unseen = outcome.pass? this.#confirm(view, record, outcome, exercised)diff --git a/src/domain/analysisView/environmentManifestView.test.ts b/src/domain/analysisView/environmentManifestView.test.tsindex 0910f6e3f..4c5088659 100644--- a/src/domain/analysisView/environmentManifestView.test.ts+++ b/src/domain/analysisView/environmentManifestView.test.ts@@ -1062,6 +1062,46 @@ it("treats a +json suffix as JSON only on application media types", () => {expect(projected.tools[1]?.body_shape).toBeNull();});+it("keeps a write answered with no body as a tool whatever its content type, but not one only redirected", () => {+ const capture = syntheticCapture([+ ["GET", "https://s.test/items/1", "application/json"],+ ["GET", "https://s.test/items/2", "application/json"],+ ["DELETE", "https://s.test/items/1", "text/html"],+ ["DELETE", "https://s.test/items/2", "text/html"],+ ["POST", "https://s.test/forms/1", "text/html"],+ ["PUT", "https://s.test/pages/1", "text/html"],+ ]);+ const answer = (index: number, status: number, text?: string): void => {+ const record = capture.records[index]!;+ record.reported = {+ ...(record.reported as Record<string, unknown>),+ response: {+ status,+ content: {+ mimeType: "text/html",+ ...(text === undefined ? {} : { text }),+ },+ },+ };+ };+ answer(2, 204);+ answer(3, 200, "");+ // A redirect is no answer, and a missing text may be a body the capture did not keep.+ answer(4, 302, "");+ answer(5, 200);+ const projected = manifest([], capture);+ expect(projected.tools.map(line)).toEqual([+ "DELETE /items/{numeric}",+ "GET /items/{numeric}",+ ]);+ expect(+ projected.excluded.map((entry) => [entry.path_template, entry.reason]),+ ).toEqual([+ ["/forms/{numeric}", "ui-page-or-asset"],+ ["/pages/{numeric}", "ui-page-or-asset"],+ ]);+});+it("refuses a parent whose subject is absent or names another path or digest", () => {const parent = parentOf();const refusal = {diff --git a/src/domain/analysisView/environmentStateGate.ts b/src/domain/analysisView/environmentStateGate.tsindex 1acec652c..317f9b090 100644--- a/src/domain/analysisView/environmentStateGate.ts+++ b/src/domain/analysisView/environmentStateGate.ts@@ -42,6 +42,8 @@ export interface StatePlan {readonly recordings: readonly StateRecordFacts[];/** The latest of them per request line, in timeline order. */readonly latest: readonly StateRecordFacts[];+ /** Item reads left to replay because they asked for some fields only. */+ readonly projected?: readonly number[];}/** What the gate reads from an extraction; `withBody` drops each body's temporary work after `use`. */@@ -166,6 +168,9 @@ export const checkItemPlan = (required_fields: union.required,envelope_ordinal: plan.latest[0]!.ordinal,checked_record_ordinals: plan.latest.map((record) => record.ordinal),+ ...(plan.projected === undefined || plan.projected.length === 0+ ? {}+ : { projected_record_ordinals: [...plan.projected] }),};};diff --git a/src/domain/analysisView/environmentStateRules.ts b/src/domain/analysisView/environmentStateRules.tsindex 1bc627deb..9e0afd7be 100644--- a/src/domain/analysisView/environmentStateRules.ts+++ b/src/domain/analysisView/environmentStateRules.ts@@ -104,8 +104,26 @@ interface EntityViewPlan {export type StateItemView = EntityViewPlan & {readonly shape: "item";readonly envelope_ordinal: number;+ /** Recorded reads that asked for some fields only and did not return the item, which replay answers. */+ readonly projected_record_ordinals?: number[] | undefined;};+/** A query parameter that asks for some fields only (`fields`, `fields[...]`, `select`, `$select`). */+export const isProjectionParameter = (name: string): boolean => {+ const lower = name.toLowerCase();+ return (+ lower === "fields" ||+ lower.startsWith("fields[") ||+ lower === "select" ||+ lower === "$select"+ );+};++/** Whether a recorded query string names a projection parameter. */+export const projectsFields = (query: string | undefined): boolean =>+ query !== undefined &&+ [...new URLSearchParams(query).keys()].some(isProjectionParameter);+export type StateListView = EntityViewPlan & {readonly shape: "list";readonly membership: "all" | "recorded";diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.tsindex 5ca962a3a..754a0a26e 100644--- a/src/domain/analysisView/environmentStateView.ts+++ b/src/domain/analysisView/environmentStateView.ts@@ -115,6 +115,12 @@ const viewSchema = z.discriminatedUnion("shape", [.describe("Record whose response, with the entity object replaced, every item response shares.",),+ projected_record_ordinals: z+ .array(z.number().int().nonnegative())+ .optional()+ .describe(+ "Recorded reads that asked for some fields only (`fields`, `select`) and did not return the item; calls naming a projection replay.",+ ),}),z.strictObject({...entityViewFields,diff --git a/src/domain/analysisView/environmentStateWriteApply.test.ts b/src/domain/analysisView/environmentStateWriteApply.test.tsindex 3ea99ac31..185efdfa5 100644--- a/src/domain/analysisView/environmentStateWriteApply.test.ts+++ b/src/domain/analysisView/environmentStateWriteApply.test.ts@@ -4,6 +4,7 @@ import { err, ok } from "../result.js";import type { JsonObject, StateEntity } from "./environmentStateRules.js";import {bodyProblem,+ storedKinds,createdFields,updatedEntity,updatedFields,@@ -532,3 +533,10 @@ it("drops view-specific values of the fields an update changed and cites no recoentity.view_fields,);});++it("learns own stored JSON member kinds without inheriting prototype members", () => {+ expect(storedKinds(["constructor"], {}, { constructor: "value" })).toEqual([+ { member: "constructor", kind: "string" },+ ]);+ expect(storedKinds(["constructor", "toString"], {}, {})).toEqual([]);+});diff --git a/src/domain/analysisView/environmentStateWriteApply.ts b/src/domain/analysisView/environmentStateWriteApply.tsindex 92eaf9a64..64ad47cb9 100644--- a/src/domain/analysisView/environmentStateWriteApply.ts+++ b/src/domain/analysisView/environmentStateWriteApply.ts@@ -37,6 +37,13 @@ export type GenerateValue = (const kindOf = (value: JsonValue | undefined): string =>value === null ? "null" : Array.isArray(value) ? "array" : typeof value;+/** JSON objects contain only their own members, even when a name also exists on Object.prototype. */+const ownMember = (+ object: JsonObject,+ member: string,+): JsonValue | undefined =>+ Object.hasOwn(object, member) ? object[member] : undefined;+/** The JSON types `stored` holds members in, for those `observed` shows no value of. */export const storedKinds = (members: readonly string[],@@ -44,8 +51,8 @@ export const storedKinds = (stored: JsonObject,): WriteBody["kinds"] =>members.flatMap((member) => {- const value = stored[member];- return (observed[member] ?? null) === null &&+ const value = ownMember(stored, member);+ return (ownMember(observed, member) ?? null) === null &&value !== undefined &&value !== null? [{ member, kind: kindOf(value) as JsonKind }]@@ -100,7 +107,7 @@ export const bodyProblem = (}if (!body.accepted.includes(member))return `Member ${member} is not one the capture showed stored (${bodyContract(body)}).`;- const recorded = shapes.entity[member];+ const recorded = ownMember(shapes.entity, member);const kind =recorded === undefined || recorded === null? kinds.get(member)@@ -361,7 +368,7 @@ export const updatedEntity = (): StateEntity => {const changed = new Set(Object.keys(fields).filter(- (field) => !same(entity.fields[field], fields[field]),+ (field) => !same(ownMember(entity.fields, field), fields[field]),),);// Written values are no longer the ones the cited records show.diff --git a/src/domain/analysisView/environmentStateWriteChecks.ts b/src/domain/analysisView/environmentStateWriteChecks.tsindex 40dbb942f..7526160d6 100644--- a/src/domain/analysisView/environmentStateWriteChecks.ts+++ b/src/domain/analysisView/environmentStateWriteChecks.ts@@ -192,6 +192,8 @@ export interface CheckOutcome {readonly unheld: readonly string[];/** For a failed list read that differs only in some members' fields, those members' targets. */readonly differing?: readonly string[];+ /** Required fields the read shows the application leaves out of entities the state holds. */+ readonly omitted?: readonly string[];}/** Whether a passing check shows where a create placed its entity: a list's does unless a sort orders that entity, leaving it tied with none, or it matched alone. */@@ -208,7 +210,7 @@ export const showsPlacement = ());/** What a check compares: the read's recorded body and the served base recording it would answer on. */-interface ReadBodies {+export interface ReadBodies {readonly body: JsonValue | undefined;/** The recording whose envelope the served answer keeps: an item view's envelope, a list line's latest init recording. */readonly base: JsonValue | undefined;diff --git a/src/domain/analysisView/environmentStateWriteCreate.ts b/src/domain/analysisView/environmentStateWriteCreate.tsindex 09bde41ea..511e03447 100644--- a/src/domain/analysisView/environmentStateWriteCreate.ts+++ b/src/domain/analysisView/environmentStateWriteCreate.ts@@ -51,10 +51,11 @@ import {type WriteCall,} from "./environmentStateWriteRules.js";import {- integerOf,isUuid,nextSequence,readClock,+ sequenceNumber,+ spelledLike,} from "./environmentStateWriteValues.js";/** The id a response names alone: a scalar body, or an object's id field or single scalar member. */@@ -180,7 +181,7 @@ const sequencesOf = (: [],);-/** Whether a later call's sequences hold what serving generates: one past the largest value their scope holds or started with, and past the plan's last. */+/** Whether a later call's sequences hold what serving generates: one past the largest value their scope holds or started with and past the plan's last, spelled as the exemplar spells it. */const followsSequences = (context: EffectContext,draft: WriteDraft,@@ -195,10 +196,13 @@ const followsSequences = (rule.scope === "type" || entity.collection === created.collection,);const next = nextSequence(- nextInteger(fieldsOf(scope), rule.field),+ nextInteger(fieldsOf(scope), rule.field, draft.observed?.[rule.field]),draft.sequences.get(key),);- return integerOf(created.fields[rule.field]) === next;+ return (+ created.fields[rule.field] ===+ spelledLike(draft.observed?.[rule.field] ?? "", next)+ );});/** Whether a later call's generated values take the forms serving generates them in: clocks in the learned format, unique values and UUID identities as UUIDs, and sequences in order. */@@ -231,7 +235,10 @@ const takeSequences = (created: Pick<StateEntity, "collection" | "fields">,): void => {for (const { rule, key } of sequencesOf(draft, created.collection)) {- const value = integerOf(created.fields[rule.field]);+ const value = sequenceNumber(+ created.fields[rule.field],+ draft.observed?.[rule.field],+ );if (value !== undefined) draft.sequences.set(key, value);}};diff --git a/src/domain/analysisView/environmentStateWriteEffects.ts b/src/domain/analysisView/environmentStateWriteEffects.tsindex dc34ab6be..33b7528f3 100644--- a/src/domain/analysisView/environmentStateWriteEffects.ts+++ b/src/domain/analysisView/environmentStateWriteEffects.ts@@ -477,7 +477,9 @@ export const applyUpdate = ((member) => member !== draft.plan.body?.identity,);const derived = Object.keys(written.value).filter(- (field) => !same(written.value[field], entity.fields[field]),+ (field) =>+ !Object.hasOwn(entity.fields, field) ||+ !same(written.value[field], entity.fields[field]),);const changed = {fields: [...new Set([...sent, ...derived])],diff --git a/src/domain/analysisView/environmentStateWriteExplain.test.ts b/src/domain/analysisView/environmentStateWriteExplain.test.tsindex 0ab067888..19f629824 100644--- a/src/domain/analysisView/environmentStateWriteExplain.test.ts+++ b/src/domain/analysisView/environmentStateWriteExplain.test.ts@@ -576,6 +576,10 @@ it.each<[string, string, boolean]>([["/users/42", "/users/43", false],["/users", "/teams/{id}", false],["/{tenant}/users", "/acme/teams", false],+ ["/orgs/%61cme/books", "/orgs/acme/books/1", true],+ ["/orgs/%2561cme/books", "/orgs/%61cme/books/1", true],+ ["/orgs/%2561cme/books", "/orgs/acme/books/1", false],+ ["/orgs/a%2Fb/books", "/orgs/a/b/books/1", false],])("relates %s and %s by prefix: %s", (left, right, related) => {expect(prefixRelated(left, right)).toBe(related);expect(prefixRelated(right, left)).toBe(related);diff --git a/src/domain/analysisView/environmentStateWriteExplain.ts b/src/domain/analysisView/environmentStateWriteExplain.tsindex 6cafd4522..d41acaa0a 100644--- a/src/domain/analysisView/environmentStateWriteExplain.ts+++ b/src/domain/analysisView/environmentStateWriteExplain.ts@@ -23,6 +23,7 @@ import {integerOf,isUuid,readClock,+ sequenceNumber,slugOf,} from "./environmentStateWriteValues.js";@@ -100,10 +101,11 @@ const isClockNear = (export const nextInteger = (members: readonly JsonObject[],field: string,+ exemplar?: JsonValue,): number | undefined => {let largest: number | undefined;for (const member of members) {- const value = integerOf(member[field]);+ const value = sequenceNumber(member[field], exemplar);if (value !== undefined && (largest === undefined || value > largest))largest = value;}@@ -156,11 +158,11 @@ const generatedRule = (const clock = readClock(value);if (clock !== undefined && isClockNear(value, evidence.startMs))return { field, rule: "clock", format: clock.format };- const integer = integerOf(value);+ const integer = sequenceNumber(value);if (integer !== undefined) {- if (nextInteger(evidence.collection, field) === integer)+ if (nextInteger(evidence.collection, field, value) === integer)return { field, rule: "sequence", scope: "collection" };- if (nextInteger(evidence.type, field) === integer)+ if (nextInteger(evidence.type, field, value) === integer)return { field, rule: "sequence", scope: "type" };}if (diff --git a/src/domain/analysisView/environmentStateWriteObserve.ts b/src/domain/analysisView/environmentStateWriteObserve.tsindex ac79d7e01..d7bf89768 100644--- a/src/domain/analysisView/environmentStateWriteObserve.ts+++ b/src/domain/analysisView/environmentStateWriteObserve.ts@@ -1,11 +1,13 @@import type { JsonValue } from "../jsonValue.js";import { overlapsWrite } from "./environmentStateWriteAttribution.js";import {+ at,collectionPathOf,isObject,lastSegmentValue,listItems,pointerTo,+ projectsFields,scalarText,type JsonObject,type StateEntity,@@ -92,6 +94,36 @@ export interface ObservationSearch {) => T;}+/** Whether a read asked for some fields only and shows no whole entity: an item view replays such reads, or the read leaves out a field its view requires. */+export const projectedRead = (+ view: StateView,+ record: StateRecordFacts,+ io: Pick<ObservationSearch, "withBody">,+): boolean => {+ if (+ (view.shape !== "item" && view.shape !== "list") ||+ !projectsFields(record.query)+ )+ return false;+ // An absence or error holds no entity, so it is never a partial one.+ if (!succeeded(record)) return false;+ if (view.shape === "item" && view.projected_record_ordinals !== undefined)+ return true;+ return io.withBody(record.ordinal, (body) => {+ const objects =+ body === undefined || body === null+ ? []+ : view.shape === "item"+ ? [at(body, view.pointer)]+ : (listItems(body, view.pointer) ?? []);+ return objects.some(+ (object) =>+ !isObject(object) ||+ view.required_fields.some((field) => !Object.hasOwn(object, field)),+ );+ });+};+const succeeded = (record: StateRecordFacts): boolean =>record.status !== null && record.status >= 200 && record.status < 300;@@ -126,7 +158,8 @@ export const laterObservation = (if (!succeeded(record) ||(view?.shape !== "item" && view?.shape !== "list") ||- view.type !== target.type+ view.type !== target.type ||+ projectedRead(view, record, search))continue;if (target.covers !== undefined && !target.covers(view)) continue;diff --git a/src/domain/analysisView/environmentStateWriteOmitted.ts b/src/domain/analysisView/environmentStateWriteOmitted.tsnew file mode 100644index 000000000..10dc019bd--- /dev/null+++ b/src/domain/analysisView/environmentStateWriteOmitted.ts@@ -0,0 +1,146 @@+import {+ at,+ isObject,+ listItems,+ projectsFields,+ scalarText,+ type JsonObject,+ type StateItemView,+ type StateListView,+ type StateRecordFacts,+ type StateView,+} from "./environmentStateRules.js";+import {+ succeeded,+ type CheckOutcome,+ type ReadBodies,+ type SimulatedState,+} from "./environmentStateWriteChecks.js";++/**+ * Fields a type's recordings always carried that a later successful whole read shows the+ * application leaves out of an entity the state holds, as APIs that omit empty or false members+ * do: no entity of the type must carry them, in any of its views.+ */+export class OmittedFields {+ readonly #omitted = new Map<string, Set<string>>();++ constructor(private readonly retain: (bytes: number) => void) {}++ /** Check an item or list read with the fields reads showed omitted, this one's included, no longer required, naming this one's in the outcome. */+ check<View extends StateItemView | StateListView>(+ state: SimulatedState,+ view: View,+ record: StateRecordFacts,+ read: ReadBodies & {+ readonly idField: string;+ readonly collection: string;+ readonly itemId?: string;+ },+ compare: (view: View) => CheckOutcome | undefined,+ ): CheckOutcome | undefined {+ const omitted = lackedFields(state, view, record, read);+ const noted = this.#omitted.get(view.type);+ const loosened = relaxed(+ view,+ omitted.length === 0 ? noted : new Set([...(noted ?? []), ...omitted]),+ );+ const outcome = compare(loosened);+ return outcome === undefined ? undefined : { ...outcome, omitted };+ }++ /** Note the fields a read the gate counts shows left out, so no view of its type requires them. */+ keep(view: StateView, outcome: CheckOutcome): void {+ if (view.shape !== "item" && view.shape !== "list") return;+ for (const field of outcome.omitted ?? []) this.#note(view.type, field);+ }++ /** The current required-field facts, shared by observations, checks and final output. */+ view(view: StateView): StateView {+ return view.shape === "item" || view.shape === "list"+ ? relaxed(view, this.#omitted.get(view.type))+ : view;+ }++ /** The views with the fields reads showed omitted no longer required. */+ views(views: readonly StateView[]): StateView[] {+ return views.map((view) => this.view(view));+ }++ #note(type: string, field: string): void {+ let fields = this.#omitted.get(type);+ if (fields === undefined) {+ fields = new Set();+ this.#omitted.set(type, fields);+ }+ if (!fields.has(field)) {+ this.retain(64 + field.length * 2);+ fields.add(field);+ }+ }+}++/** The required fields a successful whole read's held entities lack. */+const lackedFields = (+ state: SimulatedState,+ view: StateItemView | StateListView,+ record: StateRecordFacts,+ read: Pick<ReadBodies, "body"> & {+ readonly idField: string;+ readonly collection: string;+ readonly itemId?: string;+ },+): string[] => {+ if (+ !succeeded(record) ||+ read.body === undefined ||+ projectsFields(record.query)+ )+ return [];+ const lacked = new Set<string>();+ for (const entity of heldEntities(state, view, read))+ for (const field of view.required_fields)+ if (!Object.hasOwn(entity, field)) lacked.add(field);+ return [...lacked];+};++/** The entities a read shows that the state holds: an item read's object naming the path's id, a list read's items. */+const heldEntities = (+ state: SimulatedState,+ view: StateItemView | StateListView,+ read: Pick<ReadBodies, "body"> & {+ readonly idField: string;+ readonly collection: string;+ readonly itemId?: string;+ },+): JsonObject[] => {+ const object = view.shape === "item" ? at(read.body!, view.pointer) : null;+ const shown =+ view.shape === "list"+ ? (listItems(read.body!, view.pointer) ?? [])+ : isObject(object) &&+ read.itemId !== undefined &&+ scalarText(object[read.idField]) === read.itemId+ ? [object]+ : [];+ return shown.filter((entity) => {+ const id = scalarText(entity[read.idField]);+ return (+ id !== undefined &&+ state.entity(view.type, read.collection, id) !== undefined+ );+ });+};++const relaxed = <View extends StateItemView | StateListView>(+ view: View,+ omitted: ReadonlySet<string> | undefined,+): View =>+ omitted === undefined+ ? view+ : {+ ...view,+ required_fields: view.required_fields.filter(+ (field) => !omitted.has(field),+ ),+ };diff --git a/src/domain/analysisView/environmentStateWriteRules.ts b/src/domain/analysisView/environmentStateWriteRules.tsindex 58a598a66..6881dab31 100644--- a/src/domain/analysisView/environmentStateWriteRules.ts+++ b/src/domain/analysisView/environmentStateWriteRules.ts@@ -3,6 +3,7 @@ import {decodedSegment,isParam,normalizedPath,+ pathSegmentMatches,segmentsOf,type JsonObject,} from "./environmentStateRules.js";@@ -193,13 +194,17 @@ export const pathValuesOf = (return values;};-/** Whether two paths are segment prefixes of one another, a parameter on either side matching any segment. */+/** Whether two paths are segment prefixes of one another, a parameter on either side matching any segment, with the shared single-decode spelling rule. */export const prefixRelated = (left: string, right: string): boolean => {const a = segmentsOf(left);const b = segmentsOf(right);const shared = Math.min(a.length, b.length);for (let index = 0; index < shared; index += 1)- if (!isParam(a[index]!) && !isParam(b[index]!) && a[index] !== b[index])+ if (+ !isParam(a[index]!) &&+ !isParam(b[index]!) &&+ !pathSegmentMatches(a[index]!, b[index]!)+ )return false;return true;};diff --git a/src/domain/analysisView/environmentStateWriteValues.test.ts b/src/domain/analysisView/environmentStateWriteValues.test.tsindex acf0175a9..6bb54dab6 100644--- a/src/domain/analysisView/environmentStateWriteValues.test.ts+++ b/src/domain/analysisView/environmentStateWriteValues.test.ts@@ -10,6 +10,7 @@ import {replaceDelimited,seededToken,seededUuid,+ sequenceNumber,slugOf,spelledLike,type ClockFormat,@@ -254,3 +255,30 @@ it("draws tokens from the whole alphabet the recorded token uses", () => {seededToken("seed-b", "k9x2m7q4z1w8"),);});++it("reads a sequence written as digit groups of one width, but not a date, version or address", () => {+ expect(sequenceNumber("000-005")).toBe(5);+ expect(sequenceNumber("00-01-12")).toBe(112);+ expect(sequenceNumber(7)).toBe(7);+ for (const other of ["2026-10-21", "2026-08", "12.04", "1.2.3", "10.0.0.9"])+ expect(sequenceNumber(other)).toBeUndefined();+ expect(sequenceNumber("A-005")).toBeUndefined();+ expect(sequenceNumber("000-")).toBeUndefined();+});++it("reads a count that outgrew its mask only as the exemplar's spelling of it", () => {+ expect(sequenceNumber("100-00", "99-99")).toBe(10_000);+ expect(sequenceNumber("1000-000", "000-005")).toBe(1_000_000);+ expect(sequenceNumber("100-00")).toBeUndefined();+ for (const other of ["2026-10-21", "20261-02", "100-0"])+ expect(sequenceNumber(other, "000-005")).toBeUndefined();+ expect(sequenceNumber("100-00", "2026-10")).toBeUndefined();+});++it("spells a sequence value in the recorded value's digit groups, the overflow leading", () => {+ expect(spelledLike("000-005", 6)).toBe("000-006");+ expect(spelledLike("000-999", 1000)).toBe("001-000");+ expect(spelledLike("999-999", 1_000_000)).toBe("1000-000");+ expect(spelledLike("12", 13)).toBe("13");+ expect(spelledLike(12, 13)).toBe(13);+});diff --git a/src/domain/analysisView/environmentStateWriteValues.ts b/src/domain/analysisView/environmentStateWriteValues.tsindex 10e814f22..b415ad0ff 100644--- a/src/domain/analysisView/environmentStateWriteValues.ts+++ b/src/domain/analysisView/environmentStateWriteValues.ts@@ -120,7 +120,58 @@ export const nextSequence = (/** An integer spelled as the recorded value spells integers: a number, or a string of digits. */export const spelledLike = (exemplar: JsonValue, value: number): JsonValue =>- typeof exemplar === "string" ? String(value) : value;+ maskedCount(exemplar) !== undefined+ ? fillMask(exemplar as string, value)+ : typeof exemplar === "string"+ ? String(value)+ : value;++const MASKED_DIGITS = /^\d{2,}(?:-\d{2,})+$/u;++/** The count `-`-separated digit groups of one width spell, such as an asset number `000-005`; dates, versions and addresses are not counts. */+const maskedCount = (value: JsonValue | undefined): number | undefined => {+ if (typeof value !== "string" || !MASKED_DIGITS.test(value)) return undefined;+ const groups = value.split("-");+ if (groups.some((group) => group.length !== groups[0]!.length))+ return undefined;+ const digits = groups.join("");+ return digits.length <= 15 ? Number(digits) : undefined;+};++/** A count that outgrew the exemplar's mask, read only when it is spelled as serving spells it, so dates never match. */+const outgrownMask = (+ value: JsonValue | undefined,+ exemplar: JsonValue | undefined,+): number | undefined => {+ if (typeof value !== "string" || maskedCount(exemplar) === undefined)+ return undefined;+ const digits = value.replaceAll("-", "");+ if (!/^\d{1,15}$/u.test(digits)) return undefined;+ const count = Number(digits);+ return fillMask(exemplar as string, count) === value ? count : undefined;+};++/** A sequence's number: an integer, digit groups that count, or with the sequence's exemplar a count that outgrew its mask. */+export const sequenceNumber = (+ value: JsonValue | undefined,+ exemplar?: JsonValue,+): number | undefined =>+ integerOf(value) ?? maskedCount(value) ?? outgrownMask(value, exemplar);++/** A number written into a mask's digit positions, zero-padded; digits that outgrow the mask lead it. */+const fillMask = (mask: string, value: number): string => {+ const padded = String(value).padStart(mask.replace(/\D/gu, "").length, "0");+ let next = padded.length - 1;+ let out = "";+ for (let at = mask.length - 1; at >= 0; at -= 1) {+ const character = mask[at]!;+ if (/\d/u.test(character)) {+ out = padded[next]! + out;+ next -= 1;+ } else out = character + out;+ }+ return padded.slice(0, next + 1) + out;+};/** Lower case, with every run of other characters joined by one `-` and none at either end. */export const slugOf = (text: string): string =>diff --git a/src/domain/analysisView/environmentStateWrites.test.ts b/src/domain/analysisView/environmentStateWrites.test.tsindex 1c3ba4982..43f40c1fe 100644--- a/src/domain/analysisView/environmentStateWrites.test.ts+++ b/src/domain/analysisView/environmentStateWrites.test.ts@@ -6,17 +6,24 @@ import {type TimedCaptureRow,} from "../../../tests/fixtures/analysisView.js";import {+ ASSET_ROWS,BOOK_INIT_ROWS,BOOK_NOT_FOUND,BOOK_ROWS,BOOK_WRITE_CLOCK,+ DETAILED_LIST_ROWS,FIRST_BOOK,+ OMITTED_FIELD_ROWS,+ PROJECTED_ROWS,+ RENAMED_BOOK,LIST_LINE_ROWS,QUERY_ROWS,SECOND_BOOK,THIRD_BOOK,+ assetBook,book,bookClock,+ detailedBook,environmentStateViewOf,rawCaptureEvidence,renamedBook,@@ -173,6 +180,216 @@ describe("served write plans", () => {});});+describe("wider write coverage", () => {+ it("drops a field from an item view's required ones once a read shows the application leaves it out", () => {+ const state = stateOf(OMITTED_FIELD_ROWS);+ expect(planOf(state, "post_books")).toMatchObject({ served: true });+ expect(+ state.views.find((view) => view.tool === "get_books_by_id"),+ ).toMatchObject({+ shape: "item",+ fields: expect.arrayContaining(["detail"]),+ });+ expect(+ state.views.find((view) => view.tool === "get_books_by_id"),+ ).not.toMatchObject({+ required_fields: expect.arrayContaining(["detail"]),+ });+ });++ it("proves an item's identity past a read that asked for some fields only, leaving that read to replay", () => {+ expect(+ stateOf(PROJECTED_ROWS).views.find(+ (view) => view.tool === "get_books_by_id",+ ),+ ).toMatchObject({+ shape: "item",+ checked_record_ordinals: [1, 2],+ projected_record_ordinals: [3],+ });+ });++ it("serves creates whose number follows the largest held one in its digit groups", () => {+ expect(planOf(stateOf(ASSET_ROWS), "post_books")).toMatchObject({+ served: true,+ rules: expect.arrayContaining([+ { field: "asset", rule: "sequence", scope: "collection" },+ ]),+ });+ });+});++describe("write coverage limits", () => {+ const WHOLE_BOOK = ["slug", "created_at", "updated_at"];+ const itemView = (rows: readonly TimedCaptureRow[]) =>+ stateOf(rows).views.find((view) => view.tool === "get_books_by_id");++ it("notes no omission from a read that asked for some fields, nor from one of a book the state no longer holds", () => {+ const projected: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["PATCH", "/api/books/1", RENAMED_BOOK, 200, { name: "Renamed Book" }],+ ["GET", "/api/books/1?fields=id,name", { id: 1, name: "Renamed Book" }],+ ];+ const tombstone: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],+ ["GET", "/api/books/3", THIRD_BOOK],+ ["DELETE", "/api/books/3", "", 204],+ ["GET", "/api/books/3", { id: 3, deleted: true }],+ ];+ for (const rows of [projected, tombstone])+ expect(itemView(rows)).toMatchObject({+ required_fields: expect.arrayContaining(WHOLE_BOOK),+ });+ const partial = { id: 3, name: "Third Book" };+ const created: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["POST", "/api/books", partial, 201, { name: "Third Book" }],+ ["GET", "/api/books/3?fields=id,name", partial],+ ];+ expect(planOf(stateOf(created), "post_books")).toMatchObject({+ served: false,+ });+ });++ it("stops requiring a field in every view of the type once a read shows it left out", () => {+ const state = stateOf(DETAILED_LIST_ROWS);+ expect(planOf(state, "post_books")).toMatchObject({ served: true });+ const list = state.views.find((view) => view.tool === "get_books");+ expect(list).toMatchObject({ fields: expect.arrayContaining(["detail"]) });+ expect(list).not.toMatchObject({+ required_fields: expect.arrayContaining(["detail"]),+ });+ });++ it("leaves a read that asked for some fields only out of the gate, even where it would resynchronise a book", () => {+ const rows: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["GET", "/api/books/2?fields=name", { name: "Second Book" }],+ [+ "PATCH",+ "/api/books/2",+ renamedBook(SECOND_BOOK, "Renamed", 5),+ 200,+ { name: "Renamed" },+ ],+ // A redirected write leaves book 2 unknown.+ ["PATCH", "/api/books/2", "", 302, { name: "Renamed" }],+ ["GET", "/api/books/2?fields=slug", { slug: "renamed" }],+ ];+ expect(planOf(stateOf(rows), "patch_books_by_id")).toMatchObject({+ contradicted_by: null,+ });+ });++ it("learns no update's effect from a read-back that asked for some fields only", () => {+ const rows: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["PATCH", "/api/books/1", "", 204, { name: "Renamed Book" }],+ ["GET", "/api/books/1?fields=id,name", { id: 1, name: "Renamed Book" }],+ ["GET", "/api/books/1", RENAMED_BOOK],+ ["GET", "/api/books", [RENAMED_BOOK, SECOND_BOOK]],+ ];+ expect(planOf(stateOf(rows), "patch_books_by_id")).toMatchObject({+ served: true,+ observation: { ordinal: 5 },+ });+ });++ it("confirms a delete from a read that asked for some fields and found no book", () => {+ const rows: TimedCaptureRow[] = [+ ...BOOK_INIT_ROWS,+ ["DELETE", "/api/books/2", "", 204],+ ["GET", "/api/books/2?fields=id,name", BOOK_NOT_FOUND, 404],+ ];+ expect(planOf(stateOf(rows), "delete_books_by_id")).toMatchObject({+ served: true,+ checked_record_ordinals: [4],+ });+ });++ it("follows a sequence past its mask's width, as serving spells it", () => {+ const numbered = (id: number, asset: string): Book =>+ assetBook(book(id, `Book ${String(id)}`, bookClock(id + 2)), asset);+ const rows: TimedCaptureRow[] = [+ ["GET", "/api/books", [numbered(1, "99-97"), numbered(2, "99-98")]],+ ["GET", "/api/books/1", numbered(1, "99-97")],+ ["GET", "/api/books/2", numbered(2, "99-98")],+ ];+ for (const [id, asset] of [+ [3, "99-99"],+ [4, "100-00"],+ [5, "100-01"],+ ] as const)+ rows.push(+ [+ "POST",+ "/api/books",+ numbered(id, asset),+ 201,+ { name: `Book ${String(id)}` },+ ],+ ["GET", `/api/books/${String(id)}`, numbered(id, asset)],+ );+ expect(planOf(stateOf(rows), "post_books")).toMatchObject({+ served: true,+ rules: expect.arrayContaining([+ { field: "asset", rule: "sequence", scope: "collection" },+ ]),+ });+ });++ it("withholds creates whose later call spells its sequence otherwise than serving would", () => {+ const spelled = (asset: JsonValue): TimedCaptureRow[] => {+ const fourth = { ...book(4, "Fourth Book", bookClock(5)), asset };+ return [+ ...ASSET_ROWS.slice(0, 5),+ ["POST", "/api/books", fourth, 201, { name: "Fourth Book" }],+ ["GET", "/api/books/4", fourth],+ ];+ };+ for (const asset of ["00-04", 4])+ expect(planOf(stateOf(spelled(asset)), "post_books")).toMatchObject({+ served: false,+ });+ expect(planOf(stateOf(spelled("000-004")), "post_books")).toMatchObject({+ served: true,+ });+ });+});++describe("omissions shown by reads the gate ignores", () => {+ it("notes no omission from a read begun while the create still ran, nor from a page the collection outgrew", () => {+ const first = detailedBook(FIRST_BOOK);+ const second = detailedBook(SECOND_BOOK);+ const running: TimedCaptureRow[] = [+ ["GET", "/api/books", [first, second]],+ ["GET", "/api/books/1", first],+ ["GET", "/api/books/2", second],+ // The create runs from second 3 to 4.5, so the read at second 4 may show the book before its detail.+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }, 1500],+ ["GET", "/api/books/3", THIRD_BOOK],+ ];+ const outgrown: TimedCaptureRow[] = [+ ["GET", "/api/books?_start=0&_end=2", [second, first]],+ ["GET", "/api/books/1", first],+ ["GET", "/api/books/2", second],+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],+ // Serving answers the range from its recording once the collection outgrows it.+ ["GET", "/api/books?_start=0&_end=2", [THIRD_BOOK, second]],+ ];+ const requiresDetail = {+ required_fields: expect.arrayContaining(["detail"]),+ };+ const itemView = (rows: readonly TimedCaptureRow[]) =>+ stateOf(rows).views.find((view) => view.tool === "get_books_by_id");+ expect([running, outgrown].map(itemView)).toMatchObject([+ requiresDetail,+ requiresDetail,+ ]);+ });+});+describe("unserved write plans", () => {it("names a write to no state type and a tool with no successful call", () => {expect(diff --git a/src/domain/analysisView/environmentStateWrites.ts b/src/domain/analysisView/environmentStateWrites.tsindex b0c04eebd..1628104a4 100644--- a/src/domain/analysisView/environmentStateWrites.ts+++ b/src/domain/analysisView/environmentStateWrites.ts@@ -24,20 +24,28 @@ const clockStart = (: null;};-/** The write plans a capture's own timeline confirms against an extracted state, and the served clock's start. */+/**+ * The write plans a capture's own timeline confirms against an extracted state, the served clock's+ * start, and the views without the fields reads showed the application leaves out.+ */export const extractWrites = (state: Pick<ExtractedState, "views" | "types" | "entities" | "documents">,input: Omit<WriteSimulationInput,"views" | "types" | "entities" | "documents">,-): Pick<ExtractedState, "writes" | "write_clock"> => ({- writes: new EnvironmentStateWriteSimulation({+): Pick<ExtractedState, "writes" | "write_clock" | "views"> => {+ const simulation = new EnvironmentStateWriteSimulation({...input,views: new Map(state.views.map((view) => [view.tool, view])),types: state.types,entities: state.entities,documents: state.documents,- }).run(),- write_clock: clockStart(input.records, input.timed),-});+ });+ const writes = simulation.run();+ return {+ writes,+ write_clock: clockStart(input.records, input.timed),+ views: simulation.omitted.views(state.views),+ };+};diff --git a/tests/fixtures/environmentStateWrites.ts b/tests/fixtures/environmentStateWrites.tsindex d646ca46b..bc05618f1 100644--- a/tests/fixtures/environmentStateWrites.ts+++ b/tests/fixtures/environmentStateWrites.ts@@ -94,6 +94,62 @@ export const BOOK_ROWS: readonly TimedCaptureRow[] = [["POST", "/api/exports", { queued: true }, 202, { format: "csv" }],];+/** A book whose item reads carry a `detail` member, which the application leaves out of books that have none. */+export const detailedBook = (entity: Book): Book => ({+ ...entity,+ detail: "x",+});++/** Seeded books read with `detail`; a created book read back without it. */+export const OMITTED_FIELD_ROWS: readonly TimedCaptureRow[] = [+ ["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],+ ["GET", "/api/books/1", detailedBook(FIRST_BOOK)],+ ["GET", "/api/books/2", detailedBook(SECOND_BOOK)],+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],+ ["GET", "/api/books/3", THIRD_BOOK],+];++/** Seeded books listed and read with `detail`; a created book read back by item alone, without it. */+export const DETAILED_LIST_ROWS: readonly TimedCaptureRow[] = [+ ["GET", "/api/books", [detailedBook(FIRST_BOOK), detailedBook(SECOND_BOOK)]],+ ["GET", "/api/books/1", detailedBook(FIRST_BOOK)],+ ["GET", "/api/books/2", detailedBook(SECOND_BOOK)],+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],+ ["GET", "/api/books/3", THIRD_BOOK],+];++/** Item reads of both books, one of them asking for its name only, which leaves its id out. */+export const PROJECTED_ROWS: readonly TimedCaptureRow[] = [+ ["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],+ ["GET", "/api/books/1", FIRST_BOOK],+ ["GET", "/api/books/2", SECOND_BOOK],+ ["GET", "/api/books/2?fields=name", { name: "Second Book" }],+];++/** A book numbered as an asset, in digit groups. */+export const assetBook = (entity: Book, asset: string): Book => ({+ ...entity,+ asset,+});++const ASSET_THIRD = assetBook(THIRD_BOOK, "000-003");+const ASSET_FOURTH = assetBook(book(4, "Fourth Book", bookClock(5)), "000-004");++/** Two creates whose asset number follows the largest held one, in its digit groups. */+export const ASSET_ROWS: readonly TimedCaptureRow[] = [+ [+ "GET",+ "/api/books",+ [assetBook(FIRST_BOOK, "000-001"), assetBook(SECOND_BOOK, "000-002")],+ ],+ ["GET", "/api/books/1", assetBook(FIRST_BOOK, "000-001")],+ ["GET", "/api/books/2", assetBook(SECOND_BOOK, "000-002")],+ ["POST", "/api/books", ASSET_THIRD, 201, { name: "Third Book" }],+ ["GET", "/api/books/3", ASSET_THIRD],+ ["POST", "/api/books", ASSET_FOURTH, 201, { name: "Fourth Book" }],+ ["GET", "/api/books/4", ASSET_FOURTH],+];+/** A page of books as a paging API wraps it, counting what it lists. */export const bookPage = (books: readonly Book[]) => ({content: [...books],