Skip to content

armgit / rea

Public Imported from GitHub
0

Reverse engineer anything with agents, from app behavior down to native binaries.

feat(environment): list every entity on a list's whole lines (#5787)

Edgar Arakelyan4043e71f6cb9
diff --git a/docs/agent-environments.md b/docs/agent-environments.md
index 431566f5f..778f43f43 100644
--- a/docs/agent-environments.md
+++ b/docs/agent-environments.md
@@ -425,7 +425,7 @@ Each read tool answers according to its view in the state:
| View | Answer |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error; a body is never invented. |
-| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A call with no query argument to a `membership: "all"` view answers from a checked line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
+| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
| `recorded` | The recording, as replay answers it. |
@@ -441,6 +441,12 @@ not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
stay served and answer from their recordings with the reason `query-body`.
+A line establishes whole-collection membership only from the beginning:
+`offset`, `start`, `_start` and `page` must be zero when present. Positive
+page numbers have an unknown zero-/one-based convention and stay recorded,
+as do malformed paging values. An empty later page therefore cannot start
+returning earlier entities after a write or `data/add`.
+
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
[write plan](web-network-captures.md#environment-state) for them is served:
the capture's later reads confirmed the effect the plan models. Other write
diff --git a/docs/specs/agent-environments/list-lines/plan.md b/docs/specs/agent-environments/list-lines/plan.md
new file mode 100644
index 000000000..53046a4bd
--- /dev/null
+++ b/docs/specs/agent-environments/list-lines/plan.md
@@ -0,0 +1,12 @@
+# Implementation plan
+
+1. **Gate.** Judge each latest list recording on its own for paging-only query, totals, next page and state order;
+ keep `all` when every line passes and otherwise name the passing lines in `all_record_ordinals`; reproduce each
+ line with its own membership.
+2. **Counters.** Count `number`-of-items members as item totals.
+3. **Consumers.** One helper says whether a view lists all on a line; serving, the write gate's list checks and the
+ insert retry use it, and an argument-less call may answer on any such line.
+4. **Absences.** When a read shows an entity absent, drop it from every write's claim on its collection, so a write
+ left with none no longer counts there.
+5. **Tests and docs.** Domain and serving tests as the acceptance lists; the membership row of the environment-state
+ docs and the list answer row of the agent-environment docs.
diff --git a/docs/specs/agent-environments/list-lines/spec.md b/docs/specs/agent-environments/list-lines/spec.md
new file mode 100644
index 000000000..156c8d416
--- /dev/null
+++ b/docs/specs/agent-environments/list-lines/spec.md
@@ -0,0 +1,49 @@
+# Decide a list's whole-collection membership per line
+
+Issue: https://github.com/morluto/rea/issues/4933
+
+## Outcome
+
+A list view reproduces its lines from the state with one of two memberships: `all`, every entity of the collection in
+state order, or `recorded`, the entities each recording listed. The view was `all` only when every checked line listed
+its whole collection. A list tool often also has a filtered line (`?library_id=…`, `?name=…`), and that one line kept
+every line of the tool to its recorded members. An entity the environment created then never appeared in a list,
+though the unfiltered line of the same tool had shown the application listing every entity.
+
+Each checked line is now judged on its own. A recorded-membership view names, in `all_record_ordinals`, the checked
+lines that each meet the conditions `all` asks of every line: the line has no query or only paging parameters starting
+at zero, lists
+exactly the entities of its collection in state order, and shows no total that differs from the number listed and no
+next page. A call answered on such a line lists every entity of the collection, the write gate checks a read on it
+against the whole collection, and a create's insert rule may be retried on it, as for an `all` view. Other lines keep
+their recorded members. Without such a line, or when every line is one, the view is unchanged and the member is absent,
+so existing states keep their values.
+
+Positive or malformed `offset`, `start`, `_start`, and `page` positions do not establish whole membership, even for an
+empty collection. Positive page numbers are ambiguous without a recorded zero-/one-based convention. These lines keep
+recorded membership rather than inventing earlier items after a write or `data/add`; all paging values must be safe
+nonnegative integers. Repeated positions must all be zero.
+
+A whole line's read checks the collection, so it exercises every write that changed the collection since it was last
+read. A write whose entity a read has since shown absent, for example a create an unmodelled delete removed and a 404
+then showed gone, no longer counts there, as when a modelled delete replaces it. The same holds when a failed item check
+takes the read's answer and that answer holds no entity. Otherwise a whole list taken after the entity was gone would
+confirm a create it cannot show.
+
+`numberOfElements` and similar number-of-items members (`number` with `elements`, `entries`, `items` or `results`)
+count the listed entities, so a whole line's counter follows the state as `totalElements` does. A view with whole lines
+takes the counters that equal the number listed on each of them, since a partial page's total counts more than the page;
+a view without one keeps those that do on every line. A line is answered with each counter set to the number listed
+only when its own recording showed that counter equal to the number it listed, so a partial page keeps its recorded
+total.
+
+## Acceptance
+
+- Domain: a tool with a whole line and a filtered line names the whole one; a create the whole line later lists is
+ confirmed; a tool whose lines are all whole, or none, names none; a whole list taken after a read showed the created
+ entity gone confirms no create.
+- Serving: after a served create, the whole line lists the created entity with its counters, and the filtered line
+ keeps its recorded members; beside a partial page whose total counts the collection, the whole line's total follows
+ the state and the page keeps its recorded total.
+- Measured: the six published environments compared with the application before and after, on the probes of the
+ fidelity comparison and the held-out sessions recorded since.
diff --git a/docs/specs/agent-environments/list-lines/tasks.md b/docs/specs/agent-environments/list-lines/tasks.md
new file mode 100644
index 000000000..cac18cf2b
--- /dev/null
+++ b/docs/specs/agent-environments/list-lines/tasks.md
@@ -0,0 +1,10 @@
+# Tasks
+
+- [x] Judge whole-collection membership per checked line.
+- [x] Count number-of-items members.
+- [x] Serve, check and place creates per line.
+- [x] Drop a write from its collection once a read shows its entity absent.
+- [x] Add domain and serving tests.
+- [x] Document per-line membership.
+- [x] Measure the six published environments before and after.
+- [x] Run static, formatting, duplication, documentation and test gates.
diff --git a/docs/specs/agent-environments/list-lines/verification.md b/docs/specs/agent-environments/list-lines/verification.md
new file mode 100644
index 000000000..9c9c16328
--- /dev/null
+++ b/docs/specs/agent-environments/list-lines/verification.md
@@ -0,0 +1,103 @@
+# Verification
+
+Verified commit: 40800631 (this branch with the write-plan branch's head `dc5e2aad` merged in). Executed on macOS with
+Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was not installed); the probes were recorded earlier
+against the live applications on six disposable Linux VMs.
+
+## Gates
+
+- `npm run build:cached`, `npm run check:fast` (typecheck and lint), `npm run format:check`, `npx knip`,
+ `npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and `npm run mcp-catalog:check`: passed.
+- `npm run jscpd`: 3.45% duplicated lines over 3646 files, under the 3.5% gate.
+- `npx vitest run --project domain --project services`: 7616 passed, 5 pre-existing skips.
+- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
+ and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
+ tool schema validity: 8 files, 78 passed.
+
+New tests: three domain tests in `environmentStateWrites` (a tool with a whole line and a filtered line names the whole
+one, with `numberOfElements` and `totalElements` as counters, and a create the whole line lists is confirmed; a whole
+list taken after a read showed the created entity gone confirms no create, which fails without the absence rule; a
+tool whose lines are all whole, or none, names none) and one serving test in `environmentStateWriteAnswer` (after a served
+create, the whole line lists the created book with its counters, and the filtered line keeps its recorded member).
+
+## Fifty published captures
+
+The environment-state view ran over the 50 published captures with each manifest's template rules, on builds with and
+without this change. Both merged the prefix-variance branch, which some manifests need.
+
+- 47 list views in 20 environments gain a whole line, among them Komga's books, series, collections and read lists,
+ BookStack's books, chapters and pages, Mealie's cookbooks, shopping lists, categories and tags, WordPress's posts and
+ comments, Vikunja's labels and Paperless's groups. On that line a call now lists the state's entities, including
+ those a task loads with `data/add` and those served writes create or delete, where it listed only the recorded
+ members. Three of them belong to a type with a served write: Airbyte's tags, BookStack's books and Linkding's tags.
+- Served write tools stay at 31, with two changes:
+ - Airbyte's tag delete is now served. A whole list after the delete shows the tag gone.
+ - Sentry's release create is now unconfirmed, in the pool's earlier manifest, where the release's update and delete
+ are literal tools and not modelled. Its only confirmation had been a whole list taken after the delete and a 404
+ showed the release gone. The write-plan branch already refuses such a list after a modelled delete; this change
+ refuses it after an observed absence too.
+- Komga's list views also count `numberOfElements`. No other view or write plan changed.
+- Every write verdict is unchanged for the six environments republished with newer manifests (NetBox, Komga, Mealie,
+ n8n, Shlink, Sentry). Komga gains 6 whole-line lists, Mealie 4 and Sentry 1.
+
+## Six environments against their applications
+
+The six environments of the fidelity comparison (NetBox, Komga, Mealie, n8n, Shlink, self-hosted Sentry) were compared
+with their applications by `environment-compare` with and without this change. The probes were the comparison's capture
+re-run, create/read/update/delete scenario and read of every seeded entity, and held-out sessions recorded later with
+values no capture sent. Both builds merged the environment-compare and prefix-variance branches.
+
+| Environment | Steps | Before | After | Ignoring order, before | After |
+| ----------- | ----: | -----: | ----: | ---------------------: | ----: |
+| NetBox | 479 | 98.5% | 98.5% | 98.5% | 98.5% |
+| Komga | 600 | 91.5% | 91.5% | 91.5% | 91.5% |
+| Mealie | 449 | 94.0% | 94.0% | 94.9% | 94.9% |
+| n8n | 278 | 71.6% | 71.6% | 84.2% | 84.2% |
+| Shlink | 202 | 87.1% | 87.1% | 90.1% | 90.1% |
+| Sentry | 372 | 24.5% | 24.5% | 47.6% | 47.6% |
+
+No step changed. This Komga environment's capture creates, updates and deletes a collection before any whole list, so
+with the write-plan branch's head its create and update are unconfirmed and serve nothing a list could show. Before
+that head (at `f2b95351`), when a list taken after the delete still confirmed them, the same comparison moved Komga
+from 94.7% to 95.2%: three list reads after creates agreed and two more differed only in where the created collection
+was placed. None of the other five environments has a list tool with both kinds of line and a served write on its type.
+
+## Review round
+
+The branch merged the write-plan branch at 73c0e9b5 and 6d28aa78 and main at 4dfa1c67. Six of Greptile's eight
+findings were in the write-plan branch's code and were fixed there. This branch fixed the other two:
+
+- a view with whole lines takes its counters from them, and each line updates only the counters its own recording
+ counted, so a partial page no longer drops a whole line's total;
+- a failed item check that takes an answer with no entity drops the writes that claimed it from its collection, as an
+ absence does, so a whole list cannot confirm the create.
+
+Both new tests fail on the previous head. Over the 50 published captures, the merges changed no view or write plan,
+and these two fixes add one counter each to BookStack's pages (`total`) and Linkding's tags (`count`), both views with
+a whole line beside partial ones. Served write tools stay at 31. The six environments compare exactly as in the table
+above, step for step. The gates passed again.
+
+A second round merged the write-plan branch at 43d98ea9, which carries main d8a9673f and that branch's fixes for the
+four findings Greptile raised here against its code. Its page cap applies on every line that lists its whole
+collection: a call naming such a page that the collection has outgrown answers from its recording, and the gate skips
+such reads. Over the 50 captures no view or write plan changed and served write tools stay at 31; the six environments
+compare exactly as before.
+
+A third round merged the write-plan branch at 264ce410. Greptile's finding here, that a `_start`–`_end` range escaped
+the page cap, was in that branch's code and was fixed there: the range's width bounds a page as `limit` does, so a
+whole line naming a range the collection has outgrown answers from its recording, and the gate checks nothing by it.
+The merge also brings that branch's other fixes: unseen writes stay pending, later creates follow their sequences, and
+later document writes stay at their object. Over the 50 captures no view or write plan changed and served write tools
+stay at 31; the six environments compare exactly as before.
+
+It then merged that branch at 0ae64400 and 6257de93: an update's identity member never takes a rule, a page size of
+zero bounds nothing, and the paging rules move to `environmentStatePaging.ts`. Over the 50 captures again no view or
+write plan changed, and the six environments compare exactly as before.
+
+A fourth round merged the write-plan branch at 466ed8b4, which fixes four findings Greptile raised here against its
+code: an update whose observation drops a field is refused, entities are named as serving names them, a later
+create's generated values must keep their forms, and a derived state too large to load back is not served. The page
+cap here now uses that branch's `pageBound`. Over the 50 captures no view or write plan changed and served write tools
+stay at 31; the six environments compare exactly as before. It then merged 7cef2fb2, whose fixes settle two more findings Greptile raised here against that branch's code: zero
+page sizes erasing positive bounds, and interference timed by a tool's call on another collection. Over the 50
+captures again no view or write plan changed, and the six environments compare exactly as before.
diff --git a/docs/web-network-captures.md b/docs/web-network-captures.md
index b25ab0e7e..91cb9d860 100644
--- a/docs/web-network-captures.md
+++ b/docs/web-network-captures.md
@@ -595,12 +595,12 @@ Every entity cites its `record_ordinals`.
`views` lists one plan per read tool in tool order:
-| Shape | Plan |
-| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
-| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query or only paging parameters (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`, `pagesize`, `size`, `start`, `_start`, `_end`), no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
-| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
-| `recorded` | A `reason` the tool replays its recordings instead |
+| Shape | Plan |
+| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
+| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query or only paging parameters (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`, `pagesize`, `size`, `start`, `_start`, `_end`), no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, `numberOfElements`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed, except on the lines `all_record_ordinals` names, which each meet those conditions on their own and list every entity), and `counters` that equal the number listed |
+| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
+| `recorded` | A `reason` the tool replays its recordings instead |
An item or list plan is kept only when it reproduces the latest initial
recording of each request line from the state, comparing JSON values without
@@ -651,8 +651,9 @@ with its recording as the read plans are, as the whole response it would serve:
the entity or list rendered into the envelope of the item view's or the line's
base recording, with the members the state still holds. A 400, 404 or 410 (or a
`null` body) agrees with an entity the copy does not hold. A match confirms
-the writes that changed its entity or collection since it was last read; a
-mismatch contradicts them, or, when an unmodelled write came after them, is
+the writes that changed its entity or collection since it was last read, less
+those whose entity a read since showed absent, which no longer count for the
+collection; a mismatch contradicts them, or, when an unmodelled write came after them, is
checked only on the members they sent; a mismatch no write explains marks the related writes since as
`side-effect`. A plan is `served` when a read confirmed it and none
contradicted it; otherwise its `reason` is the first of its build reason
diff --git a/src/application/environment/environmentStateAnswer.ts b/src/application/environment/environmentStateAnswer.ts
index cfd673966..5032668b2 100644
--- a/src/application/environment/environmentStateAnswer.ts
+++ b/src/application/environment/environmentStateAnswer.ts
@@ -2,7 +2,9 @@ import { stateBodyJson } from "../../domain/analysisView/EnvironmentStateAccumul
import {
collectionPathOf,
decodedSegment,
+ allLines,
listItems,
+ listsAll,
missingRequiredFields,
renderStateEntity,
renderStateList,
@@ -427,27 +429,27 @@ const answerList = (
return fromRecording(source, tool, args, "body-unusable");
const collection = collectionPathOf(base, false);
const type = store.type(view.type);
+ const all = listsAll(view, base.ordinal);
// A requested page the collection has outgrown is answered as recorded, never with more items than it asked for.
const size = exact
? pageBound(base.query ?? undefined, recorded.length)
: undefined;
if (
- view.membership === "all" &&
+ all &&
size !== undefined &&
store.inCollection(view.type, collection).length > size
)
return fromRecording(source, tool, args, "page-exceeded");
- const members: readonly StateEntity[] =
- view.membership === "all"
- ? store.inCollection(view.type, collection)
- : recorded.flatMap((item) => {
- const id = scalarText(item[type?.id_field ?? ""]);
- const entity =
- id === undefined
- ? undefined
- : store.entity(view.type, collection, id);
- return entity === undefined ? [] : [entity];
- });
+ const members: readonly StateEntity[] = all
+ ? store.inCollection(view.type, collection)
+ : recorded.flatMap((item) => {
+ const id = scalarText(item[type?.id_field ?? ""]);
+ const entity =
+ id === undefined
+ ? undefined
+ : store.entity(view.type, collection, id);
+ return entity === undefined ? [] : [entity];
+ });
const volatile = new Set(type?.volatile_fields);
const items: JsonObject[] = [];
for (const entity of members) {
@@ -464,7 +466,7 @@ const answerList = (
view: "list",
type: view.type,
collection,
- membership: view.membership,
+ membership: all ? "all" : "recorded",
listed: items.length,
},
);
@@ -501,8 +503,8 @@ const answerCheckedLine = (
);
const base =
named ??
- (view.shape === "list" && view.membership === "all"
- ? unfilteredLine(serving.source, tool, view.checked_record_ordinals, args)
+ (view.shape === "list"
+ ? unfilteredLine(serving.source, tool, allLines(view), args)
: undefined);
if (base === undefined)
return fromRecording(serving.source, tool, args, "unrecorded-request");
diff --git a/src/application/environment/environmentStateWriteAnswer.test.ts b/src/application/environment/environmentStateWriteAnswer.test.ts
index 1df8da53f..c7f249574 100644
--- a/src/application/environment/environmentStateWriteAnswer.test.ts
+++ b/src/application/environment/environmentStateWriteAnswer.test.ts
@@ -10,10 +10,12 @@ import {
BOOK_ROWS,
DELETE_NOT_FOUND,
FIRST_BOOK,
+ LIST_LINE_ROWS,
SECOND_BOOK,
THIRD_BOOK,
book,
bookClock,
+ bookPage,
renamedBook,
type Book,
} from "../../../tests/fixtures/environmentStateWrites.js";
@@ -539,6 +541,97 @@ describe("paged list lines", () => {
});
});
+describe("list lines", () => {
+ it("keeps an empty later page recorded after data/add, while a first line lists the loaded entity", () => {
+ for (const [query, whole] of [
+ ["offset=0", true],
+ ["start=0", true],
+ ["_start=0", true],
+ ["page=0", true],
+ ["offset=2", false],
+ ["start=2", false],
+ ["_start=2", false],
+ ["page=1", false],
+ ["offset=-1", false],
+ ["offset=unknown", false],
+ ] as const) {
+ const alpha = { id: 1, name: "Alpha", org: "1" };
+ const beta = { id: 2, name: "Beta", org: "2" };
+ const books = servedFrom([
+ ["GET", "/api/orgs/1/books", [alpha]],
+ ["GET", "/api/orgs/1/books?name=Alpha", [alpha]],
+ ["GET", `/api/orgs/2/books?${query}&limit=50`, []],
+ ["GET", "/api/orgs/1/books/1", alpha],
+ ]);
+ const held = books.store.snapshot().rea_state;
+ const loaded = books.store.replace({
+ rea_state: {
+ ...held,
+ entities: [
+ ...held.entities,
+ {
+ type: held.entities[0]!.type,
+ collection: "/api/orgs/2/books",
+ id: "2",
+ fields: beta,
+ view_fields: [],
+ record_ordinals: [],
+ },
+ ],
+ },
+ });
+ if (!loaded.ok) throw loaded.error;
+ expect(
+ books.answer("get_orgs_books_by_id", {
+ id: "2",
+ ...Object.fromEntries(new URLSearchParams(query)),
+ limit: "50",
+ }),
+ ).toMatchObject({
+ body: whole ? [beta] : [],
+ state: {
+ source: "state",
+ membership: whole ? "all" : "recorded",
+ listed: whole ? 1 : 0,
+ },
+ });
+ }
+ });
+
+ it("lists a created book on the line that lists every book, while a filtered line keeps its recorded members", () => {
+ const books = servedFrom(LIST_LINE_ROWS);
+ const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;
+ expect(books.answer("get_books")).toMatchObject({
+ body: bookPage([FIRST_BOOK, SECOND_BOOK, created]),
+ state: { source: "state", membership: "all", listed: 3 },
+ });
+ expect(books.answer("get_books", { name: "First Book" })).toMatchObject({
+ body: bookPage([FIRST_BOOK]),
+ state: { source: "state", membership: "recorded", listed: 1 },
+ });
+ });
+
+ it("updates on each line only the counters its recording counted, so a partial page keeps its total", () => {
+ const partial = {
+ content: [FIRST_BOOK],
+ totalElements: 2,
+ numberOfElements: 1,
+ };
+ const books = servedFrom([
+ ["GET", "/api/books", bookPage([FIRST_BOOK, SECOND_BOOK])],
+ ["GET", "/api/books?page=1&size=1", partial],
+ ...LIST_LINE_ROWS.slice(2),
+ ]);
+ const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;
+ expect(books.answer("get_books")).toMatchObject({
+ body: bookPage([FIRST_BOOK, SECOND_BOOK, created]),
+ });
+ expect(books.answer("get_books", { page: "1", size: "1" })).toMatchObject({
+ body: partial,
+ });
+ });
+});
+
describe("path values", () => {
const orgBook = (id: number, org: string, name: string): Book => ({
id,
diff --git a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts
index 098e849a2..ed140338e 100644
--- a/src/domain/analysisView/EnvironmentStateWriteSimulation.ts
+++ b/src/domain/analysisView/EnvironmentStateWriteSimulation.ts
@@ -3,6 +3,7 @@ import {
changesState,
isObject,
isStateWrite,
+ listsAll,
normalizedPath,
type JsonObject,
type StateEntity,
@@ -24,7 +25,6 @@ import {
servedLine,
succeeded,
withBodies,
- writtenFieldChecks,
SimulatedState,
type CheckOutcome,
type CheckedLines,
@@ -42,6 +42,7 @@ import {
WriteEffects,
type PendingTargets,
type TouchedTargets,
+ writtenFieldChecks,
} from "./environmentStateWriteAttribution.js";
import { applyCreate } from "./environmentStateWriteCreate.js";
import { applyDocument } from "./environmentStateWriteDocument.js";
@@ -54,7 +55,6 @@ import {
type WriteSimulationInput,
type WriteStep,
} from "./environmentStateWriteEffects.js";
-import { insertIndex } from "./environmentStateWriteExplain.js";
import {
classifyWrite,
prefixRelated,
@@ -270,7 +270,7 @@ export class EnvironmentStateWriteSimulation {
this.#taint.known(entity.type, entity.collection, entity.id);
this.#touched.delete(target);
if (isAbsence(record.status)) {
- this.#state.remove(entity);
+ this.#removeEntity(entity);
return;
}
const object = this.#checkFields(view, record, entity, writes);
@@ -336,7 +336,10 @@ export class EnvironmentStateWriteSimulation {
};
return item
? checkItemRead(this.#state, view, record, check)
- : checkListRead(this.#state, view, record, check);
+ : checkListRead(this.#state, view, record, {
+ ...check,
+ all: listsAll(view, base),
+ });
});
}
@@ -484,7 +487,13 @@ export class EnvironmentStateWriteSimulation {
);
if (isObject(object))
this.#state.put(updatedEntity(entity, { ...entity.fields, ...object }));
- else this.#state.remove(entity);
+ else this.#removeEntity(entity);
+ }
+
+ /** A read showed the entity gone: the state drops it, and no write keeps a claim on its collection through it. */
+ #removeEntity(entity: StateEntity): void {
+ this.#state.remove(entity);
+ this.#effects.removed(this.#touched, entity);
}
/** Contradict the writes a failed check exercised, or, when none, the related writes since its targets were last checked; a failed list stops being checked. */
@@ -521,9 +530,9 @@ export class EnvironmentStateWriteSimulation {
record: StateRecordFacts,
exercised: ReadonlySet<string>,
): boolean {
- const collection = this.#line(view, record)?.collection;
- if (view.membership !== "all" || collection === undefined) return false;
- const members = this.#state.members(view.type, collection);
+ const line = this.#line(view, record);
+ if (line === undefined || !listsAll(view, line.ordinal)) return false;
+ const members = this.#state.members(view.type, line.collection);
const key = (entity: StateEntity) =>
entityTarget(entity.type, entity.collection, entity.id);
const inserted = members.filter((entity) =>
@@ -538,13 +547,7 @@ export class EnvironmentStateWriteSimulation {
// An item read confirms the entity, not where a list places it.
if (!exercised.has(name!) || this.#placed.has(name!)) return false;
for (const candidate of draft.candidates.slice(1)) {
- for (const entity of inserted) this.#state.remove(entity);
- for (const entity of inserted) {
- const order = this.#state
- .members(view.type, collection)
- .map((member) => member.fields);
- this.#state.put(entity, insertIndex(candidate, order, entity.fields));
- }
+ this.#state.reinsert(view.type, line.collection, inserted, candidate);
if (this.#check(view, record)?.pass === true) {
draft.plan.insert = candidate;
return true;
diff --git a/src/domain/analysisView/environmentStateGate.ts b/src/domain/analysisView/environmentStateGate.ts
index 0e0b17509..ce7152354 100644
--- a/src/domain/analysisView/environmentStateGate.ts
+++ b/src/domain/analysisView/environmentStateGate.ts
@@ -19,7 +19,7 @@ import {
type StateToolFacts,
type StateView,
} from "./environmentStateRules.js";
-import { pagesOnly } from "./environmentStatePaging.js";
+import { startsAtBeginning } from "./environmentStatePaging.js";
import { ENTRY_BYTES, textBytes } from "./webNetworkRouteTemplates.js";
/** A read tool whose recordings locate entities of one type at one pointer. */
@@ -159,44 +159,53 @@ export const checkItemPlan = (
};
};
-/** Whether every latest recording lists its whole collection in state order, and which counters equal the item count. */
+/** The latest recordings that list their whole collection in state order, and the counters that equal the item count on each of them, or on every line when none does. */
const listFacts = (
context: StateGateContext,
plan: StatePlan,
union: FieldUnion,
-): { readonly all: boolean; readonly counters: string[] } => {
+): { readonly whole: ReadonlySet<number>; readonly counters: string[] } => {
const field = context.idField(plan.type);
- let all = true;
- let counters: string[] | undefined;
+ const whole = new Set<number>();
+ const counting = new Map<number, string[]>();
for (const record of plan.latest)
context.withBody(record.ordinal, (body) => {
const items =
body === undefined ? undefined : listItems(body, plan.pointer);
if (body === undefined || items === undefined) return;
items.forEach((item) => union.add(item));
- // Once a list cannot be the whole collection, its membership is not compared again; a line
- // that only pages its list can still hold all of it.
- if (all && pagesOnly(record.query)) {
- const ids = items.map((item) => scalarText(item[field]) ?? "");
- const expected = context
- .inCollection(plan.type, collectionPathOf(record, false))
- .map((entity) => entity.id);
- if (!same(ids, expected)) all = false;
- } else all = false;
const parent = plan.pointer === "" || !isObject(body) ? {} : body;
const counted = Object.keys(parent).filter(
(name) => isItemTotal(name) && countOf(parent[name]) !== undefined,
);
- // A total that differs from the number listed, or a next page, shows the list is partial.
+ counting.set(
+ record.ordinal,
+ counted.filter((name) => parent[name] === items.length),
+ );
+ // A later (or ambiguously positioned) page cannot establish whole membership,
+ // even when its collection is empty. Totals and continuation also disprove it.
if (
+ !startsAtBeginning(record.query) ||
counted.some((name) => countOf(parent[name]) !== items.length) ||
Object.keys(parent).some((name) => continues(name, parent[name]))
)
- all = false;
- const names = counted.filter((name) => parent[name] === items.length);
- counters = (counters ?? names).filter((name) => names.includes(name));
+ return;
+ const ids = items.map((item) => scalarText(item[field]) ?? "");
+ const expected = context
+ .inCollection(plan.type, collectionPathOf(record, false))
+ .map((entity) => entity.id);
+ if (same(ids, expected)) whole.add(record.ordinal);
});
- return { all, counters: (counters ?? []).sort(compare) };
+ // A whole line's counters count the collection, which a partial page's total does not, so the whole lines decide.
+ const lines = [...counting].filter(
+ ([ordinal]) => whole.size === 0 || whole.has(ordinal),
+ );
+ const counters = lines.reduce<string[] | undefined>(
+ (kept, [, names]) =>
+ kept === undefined ? names : kept.filter((name) => names.includes(name)),
+ undefined,
+ );
+ return { whole, counters: (counters ?? []).sort(compare) };
};
const nameTokens = (name: string): string[] =>
@@ -204,17 +213,21 @@ const nameTokens = (name: string): string[] =>
.split(/[_\-\s]|(?<=[a-z0-9])(?=[A-Z])/u)
.map((token) => token.toLowerCase());
-/** Item totals, including compact `totalcount`/`itemcount`; page counts and account IDs are excluded. */
+const LISTED_NOUNS = new Set(["elements", "entries", "items", "results"]);
+
+/** Item totals, including compact `totalcount`/`itemcount` and `numberOfElements`; page counts and account IDs are excluded. */
const isItemTotal = (name: string): boolean => {
const tokens = nameTokens(name);
return (
- tokens.some(
+ (tokens.some(
(token) =>
token === "total" ||
token === "count" ||
token === "totalcount" ||
token === "itemcount",
- ) &&
+ ) ||
+ (tokens.includes("number") &&
+ tokens.some((token) => LISTED_NOUNS.has(token)))) &&
!tokens.some(
(token) => token === "page" || token === "pages" || token === "per",
)
@@ -255,7 +268,7 @@ const listReproduces = (
readonly fields: string[];
readonly pointer: string;
readonly counters: string[];
- readonly all: boolean;
+ readonly whole: ReadonlySet<number>;
},
record: StateRecordFacts,
): boolean => {
@@ -267,7 +280,7 @@ const listReproduces = (
const observed = new Map(
items.map((item) => [scalarText(item[field]) ?? "", item]),
);
- const members = view.all
+ const members = view.whole.has(record.ordinal)
? context.inCollection(plan.type, collectionPathOf(record, false))
: items.flatMap((item) => {
const entity = context.entity(
@@ -295,15 +308,16 @@ export const checkListPlan = (
plan: StatePlan,
): StateView => {
const union = new FieldUnion();
- const { all, counters } = listFacts(context, plan, union);
+ const { whole, counters } = listFacts(context, plan, union);
context.retain(union.bytes);
const view = {
tool: plan.tool.name,
fields: union.fields,
pointer: plan.pointer,
counters,
- all,
+ whole,
};
+ const all = plan.latest.every((record) => whole.has(record.ordinal));
if (
!plan.latest.every((record) => listReproduces(context, plan, view, record))
)
@@ -316,6 +330,13 @@ export const checkListPlan = (
fields: union.fields,
required_fields: union.required,
membership: all ? "all" : "recorded",
+ ...(all || whole.size === 0
+ ? {}
+ : {
+ all_record_ordinals: plan.latest
+ .map((record) => record.ordinal)
+ .filter((ordinal) => whole.has(ordinal)),
+ }),
counters,
checked_record_ordinals: plan.latest.map((record) => record.ordinal),
};
diff --git a/src/domain/analysisView/environmentStatePaging.test.ts b/src/domain/analysisView/environmentStatePaging.test.ts
new file mode 100644
index 000000000..e679086ba
--- /dev/null
+++ b/src/domain/analysisView/environmentStatePaging.test.ts
@@ -0,0 +1,18 @@
+import { expect, it } from "vitest";
+import { startsAtBeginning } from "./environmentStatePaging.js";
+
+it("requires every recorded position to establish the beginning, including repeated names", () => {
+ expect(startsAtBeginning(undefined)).toBe(true);
+ expect(startsAtBeginning("?offset=0&offset=0&limit=50")).toBe(true);
+ for (const query of [
+ "?offset=0&offset=2&limit=50",
+ "?page=1&size=50",
+ "?start=1&_end=50",
+ "?_start=2&_end=50",
+ "?offset=9007199254740992",
+ "?offset=NaN",
+ "?limit=unknown",
+ "?name=Alpha",
+ ])
+ expect(startsAtBeginning(query), query).toBe(false);
+});
diff --git a/src/domain/analysisView/environmentStatePaging.ts b/src/domain/analysisView/environmentStatePaging.ts
index 92b17e9ca..ecab67ccc 100644
--- a/src/domain/analysisView/environmentStatePaging.ts
+++ b/src/domain/analysisView/environmentStatePaging.ts
@@ -25,6 +25,20 @@ export const pagesOnly = (query: string | undefined): boolean =>
PAGING_PARAMETERS.has(name.toLowerCase()),
);
+const PAGE_POSITION_PARAMETERS = new Set(["offset", "start", "_start", "page"]);
+
+/** A paging-only line that can start at the collection's beginning. Positive page numbers are ambiguous (zero- or one-based); malformed or conflicting positions do not establish whole membership. */
+export const startsAtBeginning = (query: string | undefined): boolean => {
+ if (!pagesOnly(query)) return false;
+ for (const [name, value] of iterateUrlQueryParameters(query ?? "")) {
+ const number = Number(value);
+ if (!/^\d+$/u.test(value) || !Number.isSafeInteger(number)) return false;
+ if (PAGE_POSITION_PARAMETERS.has(name.toLowerCase()) && number !== 0)
+ return false;
+ }
+ return true;
+};
+
/** Paging parameters that bound how many items a page holds. */
const PAGE_SIZE_PARAMETERS = new Set([
"limit",
diff --git a/src/domain/analysisView/environmentStateRules.ts b/src/domain/analysisView/environmentStateRules.ts
index d67e9edf3..ada8c2f24 100644
--- a/src/domain/analysisView/environmentStateRules.ts
+++ b/src/domain/analysisView/environmentStateRules.ts
@@ -108,9 +108,22 @@ export type StateItemView = EntityViewPlan & {
export type StateListView = EntityViewPlan & {
readonly shape: "list";
readonly membership: "all" | "recorded";
+ /** With recorded membership, the checked lines that each list their whole collection. */
+ readonly all_record_ordinals?: number[] | undefined;
readonly counters: string[];
};
+/** Whether a list view answers on a checked line with every entity of the collection. */
+export const listsAll = (view: StateListView, ordinal: number): boolean =>
+ view.membership === "all" ||
+ (view.all_record_ordinals?.includes(ordinal) ?? false);
+
+/** The checked lines a list view answers with every entity of the collection. */
+export const allLines = (view: StateListView): readonly number[] =>
+ view.membership === "all"
+ ? view.checked_record_ordinals
+ : (view.all_record_ordinals ?? []);
+
export type StateView =
| StateItemView
| StateListView
@@ -530,13 +543,21 @@ export const missingRequiredFields = (
return view.required_fields.filter((name) => !Object.hasOwn(shown, name));
};
-/** A list response rendered from `base`: its array replaced by `items` and its counters set to their number. */
+/**
+ * A list response rendered from `base`: its array replaced by `items`, and set to their number each counter that
+ * counted the items `base` listed; one that did not, such as a partial page's total, keeps its recorded value.
+ */
export const renderStateList = (
view: Pick<StateListView, "pointer" | "counters">,
base: JsonValue,
items: readonly JsonObject[],
-): JsonValue =>
- view.counters.reduce(
- (body, name) => replaceAt(body, pointerTo(name), items.length),
+): JsonValue => {
+ const listed = listItems(base, view.pointer)?.length;
+ return view.counters.reduce(
+ (body, name) =>
+ at(base, pointerTo(name)) === listed
+ ? replaceAt(body, pointerTo(name), items.length)
+ : body,
replaceAt(base, view.pointer, [...items]),
);
+};
diff --git a/src/domain/analysisView/environmentStateView.ts b/src/domain/analysisView/environmentStateView.ts
index dcaf204c9..619111b57 100644
--- a/src/domain/analysisView/environmentStateView.ts
+++ b/src/domain/analysisView/environmentStateView.ts
@@ -58,7 +58,13 @@ const viewSchema = z.discriminatedUnion("shape", [
membership: z
.enum(["all", "recorded"])
.describe(
- "all: every entity of the collection in state order; recorded: the entities each recorded response listed.",
+ "all: every entity of the collection in state order; recorded: the entities each recorded response listed, except on the lines all_record_ordinals names.",
+ ),
+ all_record_ordinals: z
+ .array(z.number().int().nonnegative())
+ .optional()
+ .describe(
+ "With recorded membership, the checked lines that each listed every entity of their collection in state order, with only paging parameters starting at zero; a call answered on one lists the whole collection. Absent when none does.",
),
counters: z
.array(z.string())
diff --git a/src/domain/analysisView/environmentStateWriteAttribution.ts b/src/domain/analysisView/environmentStateWriteAttribution.ts
index 7b20d4ea1..9df901921 100644
--- a/src/domain/analysisView/environmentStateWriteAttribution.ts
+++ b/src/domain/analysisView/environmentStateWriteAttribution.ts
@@ -1,4 +1,15 @@
-import type { StateView } from "./environmentStateRules.js";
+import {
+ renderStateEntity,
+ same,
+ type JsonObject,
+ type StateEntity,
+ type StateItemView,
+ type StateView,
+} from "./environmentStateRules.js";
+import {
+ collectionTarget,
+ entityTarget,
+} from "./environmentStateWriteChecks.js";
/** The fields a read of a view shows of its entities, less volatile ones it does not compare; all of a document's. */
export const shownFields = (
@@ -178,6 +189,27 @@ export class WriteEffects {
if (!unseen.has(tool)) byTool.get(key)!.delete(tool);
}
+ /** A read showed the entity absent: no write's effect on it remains, so a write left with no entity drops from the collection. */
+ removed(
+ touched: TouchedTargets,
+ gone: {
+ readonly type: string;
+ readonly collection: string;
+ readonly id: string;
+ },
+ ): void {
+ const entity = entityTarget(gone.type, gone.collection, gone.id);
+ const collection = collectionTarget(gone.type, gone.collection);
+ this.#changed.delete(entity);
+ const byTool = this.#members.get(collection);
+ if (byTool === undefined) return;
+ for (const [tool, entities] of byTool) {
+ if (!entities.delete(entity) || entities.size > 0) continue;
+ byTool.delete(tool);
+ touched.get(collection)?.delete(tool);
+ }
+ }
+
/** Whether a read showing `fields` of a target shows anything of a tool's effect there: a create's or delete's always, an update's through a field it changed; unknown claims count as shown. */
shows(
tool: string,
@@ -348,3 +380,35 @@ export const forgetChecked = (
}
pending.effects.checked(key, unseen);
};
+
+/**
+ * Check, field by field, the writes that changed an entity an unmodelled write also changed:
+ * each write's own fields must show as the state holds them, but for volatile ones, which take
+ * the read's value and so show nothing of a write. A create wrote every field, so it cannot be
+ * told apart from the unmodelled change and is not checked.
+ */
+export const writtenFieldChecks = (
+ entity: StateEntity,
+ view: StateItemView,
+ observed: JsonObject,
+ check: {
+ readonly volatile: ReadonlySet<string>;
+ readonly writes: ReadonlyMap<string, readonly string[] | null>;
+ },
+): Map<string, boolean> => {
+ const shown = renderStateEntity(entity, view, check.volatile, observed);
+ const outcomes = new Map<string, boolean>();
+ for (const [tool, fields] of check.writes) {
+ const compared = (fields ?? []).filter(
+ (field) =>
+ !check.volatile.has(field) &&
+ (Object.hasOwn(observed, field) || Object.hasOwn(shown, field)),
+ );
+ if (fields === null || compared.length === 0) continue;
+ outcomes.set(
+ tool,
+ compared.every((field) => same(shown[field], observed[field])),
+ );
+ }
+ return outcomes;
+};
diff --git a/src/domain/analysisView/environmentStateWriteChecks.ts b/src/domain/analysisView/environmentStateWriteChecks.ts
index 590397910..689c70c36 100644
--- a/src/domain/analysisView/environmentStateWriteChecks.ts
+++ b/src/domain/analysisView/environmentStateWriteChecks.ts
@@ -9,6 +9,7 @@ import {
decodedSegment,
isObject,
isParam,
+ listsAll,
listItems,
missingRequiredFields,
renderStateEntity,
@@ -27,6 +28,8 @@ import {
type StateView,
} from "./environmentStateRules.js";
import { pageBound, pagesOnly } from "./environmentStatePaging.js";
+import { insertIndex } from "./environmentStateWriteExplain.js";
+import type { InsertRule } from "./environmentStateWriteRules.js";
/** A query as replay compares a call's: decoded values by name, in any order; undefined when it repeats a name with another value, which no call's arguments can carry. */
const queryKey = (query: string | undefined): string | undefined => {
@@ -49,7 +52,7 @@ interface CheckedLine {
export interface CheckedLines {
readonly params: ReadonlySet<number>;
readonly byQuery: ReadonlyMap<string, readonly CheckedLine[]>;
- /** A whole list's lines that only page it, which serving answers a call supplying no query from. */
+ /** The lines listing their whole collection that only page it, which serving answers a call supplying no query from. */
readonly unfiltered: readonly CheckedLine[];
}
@@ -76,14 +79,18 @@ export const checkedLines = (input: {
const checked = new Set(view.checked_record_ordinals);
const byQuery = new Map<string, CheckedLine[]>();
const unfiltered: CheckedLine[] = [];
- const whole = view.shape === "list" && view.membership === "all";
for (const ordinal of tool.record_ordinals) {
const record = byOrdinal.get(ordinal);
const key = record === undefined ? undefined : queryKey(record.query);
if (!checked.has(ordinal) || key === undefined) continue;
const line = { ordinal, segments: segmentsOf(record!.path) };
byQuery.set(key, [...(byQuery.get(key) ?? []), line]);
- if (whole && pagesOnly(record!.query)) unfiltered.push(line);
+ if (
+ view.shape === "list" &&
+ listsAll(view, ordinal) &&
+ pagesOnly(record!.query)
+ )
+ unfiltered.push(line);
}
const params = new Set(
segmentsOf(tool.path_template).flatMap((segment, index) =>
@@ -234,6 +241,22 @@ export class SimulatedState {
);
}
+ /** Place created entities again as a create under another insert rule would have. */
+ reinsert(
+ type: string,
+ collection: string,
+ entities: readonly StateEntity[],
+ rule: InsertRule,
+ ): void {
+ for (const entity of entities) this.remove(entity);
+ for (const entity of entities) {
+ const order = this.members(type, collection).map(
+ (member) => member.fields,
+ );
+ this.put(entity, insertIndex(rule, order, entity.fields));
+ }
+ }
+
document(tool: string): JsonValue | undefined {
return this.#documents.get(tool);
}
@@ -504,12 +527,17 @@ const checkRecordedList = (
return { ...listCheck(view, check, members), targets, unheld };
};
-/** A list read on a checked line, compared with the answer the served environment renders on that line's base recording from the collection it names. */
+/** A list read on a checked line, compared with the answer the served environment renders on that line's base recording from the collection it names: every entity when that line lists them all. */
export const checkListRead = (
state: SimulatedState,
view: StateListView,
record: StateRecordFacts,
- check: ReadBodies & { readonly idField: string; readonly collection: string },
+ check: ReadBodies & {
+ readonly idField: string;
+ readonly collection: string;
+ /** Whether the base line lists every entity of its collection. */
+ readonly all: boolean;
+ },
): CheckOutcome | undefined => {
if (!succeeded(record)) return undefined;
const { body, base } = check;
@@ -520,7 +548,7 @@ export const checkListRead = (
// Serving answers a page too small for the collection from its recording, so such a read checks nothing.
const size = pageBound(record.query, baseItems.length);
if (
- view.membership === "all" &&
+ check.all &&
size !== undefined &&
state.members(view.type, check.collection).length > size
)
@@ -528,10 +556,9 @@ export const checkListRead = (
// A usable body that no longer holds the list is not what serving answers on this line.
const items = listItems(body, view.pointer);
const bodies = { ...check, body, base, items: items ?? [], baseItems };
- const outcome =
- view.membership === "all"
- ? checkAllList(state, view, bodies)
- : checkRecordedList(state, view, bodies);
+ const outcome = check.all
+ ? checkAllList(state, view, bodies)
+ : checkRecordedList(state, view, bodies);
return items === undefined ? { ...outcome, pass: false } : outcome;
};
@@ -551,35 +578,3 @@ export const checkDocumentRead = (
unheld: [],
};
};
-
-/**
- * Check, field by field, the writes that changed an entity an unmodelled write also changed:
- * each write's own fields must show as the state holds them, but for volatile ones, which take
- * the read's value and so show nothing of a write. A create wrote every field, so it cannot be
- * told apart from the unmodelled change and is not checked.
- */
-export const writtenFieldChecks = (
- entity: StateEntity,
- view: StateItemView,
- observed: JsonObject,
- check: {
- readonly volatile: ReadonlySet<string>;
- readonly writes: ReadonlyMap<string, readonly string[] | null>;
- },
-): Map<string, boolean> => {
- const shown = renderStateEntity(entity, view, check.volatile, observed);
- const outcomes = new Map<string, boolean>();
- for (const [tool, fields] of check.writes) {
- const compared = (fields ?? []).filter(
- (field) =>
- !check.volatile.has(field) &&
- (Object.hasOwn(observed, field) || Object.hasOwn(shown, field)),
- );
- if (fields === null || compared.length === 0) continue;
- outcomes.set(
- tool,
- compared.every((field) => same(shown[field], observed[field])),
- );
- }
- return outcomes;
-};
diff --git a/src/domain/analysisView/environmentStateWrites.test.ts b/src/domain/analysisView/environmentStateWrites.test.ts
index e813071ae..66d45592e 100644
--- a/src/domain/analysisView/environmentStateWrites.test.ts
+++ b/src/domain/analysisView/environmentStateWrites.test.ts
@@ -11,6 +11,7 @@ import {
BOOK_ROWS,
BOOK_WRITE_CLOCK,
FIRST_BOOK,
+ LIST_LINE_ROWS,
SECOND_BOOK,
THIRD_BOOK,
book,
@@ -541,3 +542,65 @@ describe("observation from a later read", () => {
});
});
});
+
+describe("list lines", () => {
+ it("lists every entity on a line that lists its whole collection, though another line of the tool is filtered", () => {
+ const state = stateOf(LIST_LINE_ROWS);
+ expect(state.views.find((view) => view.tool === "get_books")).toMatchObject(
+ {
+ shape: "list",
+ pointer: "/content",
+ membership: "recorded",
+ all_record_ordinals: [0],
+ counters: ["numberOfElements", "totalElements"],
+ checked_record_ordinals: [0, 1],
+ },
+ );
+ // The whole list after the create shows the created book, which confirms it.
+ expect(planOf(state, "post_books")).toMatchObject({
+ served: true,
+ checked_record_ordinals: [5, 6],
+ });
+ });
+
+ it("confirms no create by a whole list taken after a read showed its entity gone", () => {
+ const rows: TimedCaptureRow[] = [
+ ...BOOK_INIT_ROWS,
+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
+ // A redirected delete is not modelled; the read after it shows the book gone.
+ ["DELETE", "/api/books/3", "", 302],
+ ["GET", "/api/books/3", BOOK_NOT_FOUND, 404],
+ ["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],
+ ];
+ expect(planOf(stateOf(rows), "post_books")).toMatchObject({
+ served: false,
+ reason: "unconfirmed",
+ });
+ });
+
+ it("confirms no create by a whole list after an item read showed nothing of its entity", () => {
+ const rows: TimedCaptureRow[] = [
+ ...BOOK_INIT_ROWS,
+ ["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
+ // An unmodelled write elsewhere may explain the next read, which the state then takes.
+ ["POST", "/api/exports", { queued: true }, 202, { format: "csv" }],
+ ["GET", "/api/books/3", null],
+ ["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK]],
+ ];
+ expect(planOf(stateOf(rows), "post_books")).toMatchObject({
+ served: false,
+ checked_record_ordinals: [],
+ });
+ });
+
+ it("names no line when every line lists its whole collection, or none does", () => {
+ const listOf = (rows: readonly TimedCaptureRow[]) =>
+ stateOf(rows).views.find((view) => view.tool === "get_books");
+ const whole = listOf(LIST_LINE_ROWS.filter((_, index) => index !== 1));
+ expect(whole).toMatchObject({ membership: "all" });
+ expect(whole).not.toHaveProperty("all_record_ordinals");
+ expect(listOf(LIST_LINE_ROWS.slice(1))).not.toHaveProperty(
+ "all_record_ordinals",
+ );
+ });
+});
diff --git a/tests/fixtures/environmentStateWrites.ts b/tests/fixtures/environmentStateWrites.ts
index 0880edbd0..f89a56ad3 100644
--- a/tests/fixtures/environmentStateWrites.ts
+++ b/tests/fixtures/environmentStateWrites.ts
@@ -60,6 +60,8 @@ export const FIRST_BOOK = book(1, "First Book", "2026-10-01T09:00:00Z");
export const SECOND_BOOK = book(2, "Second Book", "2026-10-01T10:00:00Z");
/** The book the recorded create at exchange 3 made. */
export const THIRD_BOOK = book(3, "Third Book", bookClock(3));
+/** The book a create recorded at exchange 4 made. */
+const THIRD_BOOK_AT_4 = book(3, "Third Book", bookClock(4));
/** The first book after the recorded update at exchange 5. */
export const RENAMED_BOOK = renamedBook(FIRST_BOOK, "Renamed Book", 5);
@@ -92,6 +94,26 @@ export const BOOK_ROWS: readonly TimedCaptureRow[] = [
["POST", "/api/exports", { queued: true }, 202, { format: "csv" }],
];
+/** A page of books as a paging API wraps it, counting what it lists. */
+export const bookPage = (books: readonly Book[]) => ({
+ content: [...books],
+ totalElements: books.length,
+ numberOfElements: books.length,
+});
+
+/**
+ * One list tool with a line that lists every book and a line filtered by name, item reads, then a
+ * create the whole list confirms.
+ */
+export const LIST_LINE_ROWS: readonly TimedCaptureRow[] = [
+ ["GET", "/api/books", bookPage([FIRST_BOOK, SECOND_BOOK])],
+ ["GET", "/api/books?name=First%20Book", bookPage([FIRST_BOOK])],
+ ...BOOK_INIT_ROWS.slice(1),
+ ["POST", "/api/books", THIRD_BOOK_AT_4, 201, { name: "Third Book" }],
+ ["GET", "/api/books/3", THIRD_BOOK_AT_4],
+ ["GET", "/api/books", bookPage([FIRST_BOOK, SECOND_BOOK, THIRD_BOOK_AT_4])],
+];
+
/** The end of the last exchange of BOOK_ROWS, to the next whole second: where the served clock starts. */
export const BOOK_WRITE_CLOCK = "2026-10-17T18:00:12.000Z";