A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.
TypeScript 2d-game-engine3d3d-game-engineandroidcppecsgame-editorgame-enginegamedeviostypescriptwebassemblywebglwebgpuwechat-minigame
fix(ecs): removal history is kept for the readers that still need it
esengine2bed0668446d
diff --git a/docs/MUTATION_CONTRACT.md b/docs/MUTATION_CONTRACT.mdindex c0f9c81dc..4d0637c11 100644--- a/docs/MUTATION_CONTRACT.md+++ b/docs/MUTATION_CONTRACT.md@@ -175,6 +175,42 @@ shadow scan 看得见,ChangeTracker 看不见。** 只要还有一条这样的判据现在先查 `sdk/src → sdk/dist → template` 这条链,陈旧就报 `did NOT run`(exit 2)。(`.generated.ts` 不计入,原生构建会在打包之后重写它们。)+## 3.7 PR6d:removal 历史有主了++`cleanRemovedBuffer` 是一个**没有归属权**的全局 destructive API,而且它有一个调用者:+`App.tick` 每帧末尾 `cleanRemovedBuffer(worldTick - 2)`。所以这不是内存问题,是**正确性缺陷**——+任何超过 2 帧没运行的 `Removed(C)` system(`runIf` 关着、FixedUpdate 没到步、条件系统)+会静默丢失它本该看到的 removal。判据 `sdk/tests/removed-retention.test.ts` 第一条先红在这里。++现在是 **per-component 的 reader watermark**:++```+ChangeTracker+├─ 变更观测:Added / Changed / anyChangedSince ← trackedComponents_+└─ removal 历史:每 component 一张 reader 表 ← removedReaders_+ readerId → retainFromTick,prune 到最低的那个+```++- **watermark 按 component 分**,不是全局:一个组件的慢 reader 没有资格钉住另一个组件的历史。+- **cursor 在 `flushSystem_` 推进**,不在 `resetTick`——system 可以 `await` 到一半,+ 在 body 读之前就宣告「读完了」,会让另一个 reader 的 prune 删掉它仍然欠着的行。+- **认领发生在参数提交之后**,不在构造函数里:一组参数中途抛异常时,+ 已经建好的那个 reader 没有任何人能释放它,那个 component 的历史从此永久被钉住。+- **`evict()` / `reset()` 真的 dispose**:丢掉 JS 引用不等于放弃认领。+- **没有 reader 就不产生 removal 行**。`Changed(C)` 需要知道拓扑动了+ (`componentLastChangedTick_` 照常更新,UI 布局依赖它),但不需要知道谁在 tick 738 掉了 C。+ 于是「开着 Changed(C)、没有 Removed(C)、despawn 十万次」从十万行变成 **0 行**。+- **历史从 reader 认领的那一刻开始**(`retainFromTick = worldTick + 1`),+ 新 reader 不认领别人碰巧留下的古老 removal。++慢的**活着的** reader 钉住历史不是 bug——`Removed` 的语义就是「自这个 system 上次运行以来」。+真正的 bug 是已经 evict 掉的 reader 还在钉,所以 ownership 才必须显式。++八条判据,六次 sabotage 各红一条:把 release 挪到 body 之前/改成全局 watermark/+`evict` 不 dispose/`reset` 不 dispose/在构造函数里认领/让任何 tracked 组件都写 removal 行。++`World.cleanRemovedBuffer` 已删——一个谁都能替别人删历史的接缝,在有了归属权之后没有位置。+## 4. 待裁定普查给出的结论是明确的:**`get`/`tryGet`/裸 `Query` 必须收成只读**,而且必须是@@ -188,7 +224,4 @@ shadow scan 看得见,ChangeTracker 看不见。** 只要还有一条这样的同时未判定、留给后续的两件:- ~~**AOT 写回**~~:已实测,见 3.6。-- **`cleanRemovedBuffer` 的归属权**:它是全局 destructive cleanup,而每个- `RemovedQueryInstance` 各自持有 `lastRunTick`。复制层若替所有 `Removed()` 消费者删历史,- 会让 frontier 落后的消费者永久错过那段 removal。这是正确性契约而不是内存优化,- 属于 PR6d(共享 retention watermark),本文只记录事实。+- ~~**`cleanRemovedBuffer` 的归属权**~~:已收掉,见 3.7。diff --git a/sdk/benchmarks/system.bench.ts b/sdk/benchmarks/system.bench.tsindex 9f0cd97d5..632f7bfee 100644--- a/sdk/benchmarks/system.bench.ts+++ b/sdk/benchmarks/system.bench.ts@@ -208,6 +208,5 @@ describe('System - Full frame simulation (10 systems)', () => {for (const sys of systems) {runner.run(sys);}- world.cleanRemovedBuffer(2);});});diff --git a/sdk/benchmarks/world-internals.bench.ts b/sdk/benchmarks/world-internals.bench.tsindex d318c32c1..ba384d2b8 100644--- a/sdk/benchmarks/world-internals.bench.ts+++ b/sdk/benchmarks/world-internals.bench.ts@@ -44,9 +44,6 @@ describe('World internals - per-frame operations', () => {world.resetQueryPool();});- bench('cleanRemovedBuffer (nothing to clean)', () => {- world.cleanRemovedBuffer(2);- });});describe('World internals - getEntitiesWithComponents (cache behavior)', () => {diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.mdindex dd6007fc1..b1b6cfd67 100644--- a/sdk/etc/index.api.md+++ b/sdk/etc/index.api.md@@ -6371,6 +6371,9 @@ send: (data: string | ArrayBuffer) => void## RemovedQueryInstance — class @public```+@internal activateRetention: () => void+@internal dispose: () => void+@internal releaseThrough: (lastRunTick: number) => void@internal resetTick: (tick: number) => void[Symbol.iterator]: () => Iterator<Entity>isEmpty: () => boolean@@ -9489,15 +9492,16 @@ tabIndex: number | undefined```@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void@internal advanceTick: () => void@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean@internal applyEntityOrder: (entities: readonly Entity[]) => void@internal beginIteration: () => void@internal builtin: BuiltinBridge@internal changes_: ChangeTracker-@internal cleanRemovedBuffer: (beforeTick: number) => void@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void@internal disconnectCpp: () => void+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void@internal enableChangeTracking: (component: AnyComponentDef) => void@internal endIteration: () => void@internal getCppRegistry: () => CppRegistry | null@@ -9517,6 +9521,8 @@ tabIndex: number | undefined@internal queries_: QueryCache@internal queryCostEnabled: boolean@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]+@internal registerRemovedReader: (component: AnyComponentDef) => number+@internal removedReaderCount: (component: AnyComponentDef) => number@internal resetIterationDepth: () => void@internal resetQueryCacheStats: () => void@internal resetQueryPool: () => voiddiff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.mdindex 9f4948c6d..85cd92d02 100644--- a/sdk/etc/index.native.api.md+++ b/sdk/etc/index.native.api.md@@ -6564,6 +6564,9 @@ send: (data: string | ArrayBuffer) => void## RemovedQueryInstance — class @public```+@internal activateRetention: () => void+@internal dispose: () => void+@internal releaseThrough: (lastRunTick: number) => void@internal resetTick: (tick: number) => void[Symbol.iterator]: () => Iterator<Entity>isEmpty: () => boolean@@ -9682,15 +9685,16 @@ tabIndex: number | undefined```@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void@internal advanceTick: () => void@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean@internal applyEntityOrder: (entities: readonly Entity[]) => void@internal beginIteration: () => void@internal builtin: BuiltinBridge@internal changes_: ChangeTracker-@internal cleanRemovedBuffer: (beforeTick: number) => void@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void@internal disconnectCpp: () => void+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void@internal enableChangeTracking: (component: AnyComponentDef) => void@internal endIteration: () => void@internal getCppRegistry: () => CppRegistry | null@@ -9710,6 +9714,8 @@ tabIndex: number | undefined@internal queries_: QueryCache@internal queryCostEnabled: boolean@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]+@internal registerRemovedReader: (component: AnyComponentDef) => number+@internal removedReaderCount: (component: AnyComponentDef) => number@internal resetIterationDepth: () => void@internal resetQueryCacheStats: () => void@internal resetQueryPool: () => voiddiff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.mdindex cdf9af7cc..65e451031 100644--- a/sdk/etc/index.node.api.md+++ b/sdk/etc/index.node.api.md@@ -6366,6 +6366,9 @@ send: (data: string | ArrayBuffer) => void## RemovedQueryInstance — class @public```+@internal activateRetention: () => void+@internal dispose: () => void+@internal releaseThrough: (lastRunTick: number) => void@internal resetTick: (tick: number) => void[Symbol.iterator]: () => Iterator<Entity>isEmpty: () => boolean@@ -9484,15 +9487,16 @@ tabIndex: number | undefined```@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void@internal advanceTick: () => void@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean@internal applyEntityOrder: (entities: readonly Entity[]) => void@internal beginIteration: () => void@internal builtin: BuiltinBridge@internal changes_: ChangeTracker-@internal cleanRemovedBuffer: (beforeTick: number) => void@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void@internal disconnectCpp: () => void+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void@internal enableChangeTracking: (component: AnyComponentDef) => void@internal endIteration: () => void@internal getCppRegistry: () => CppRegistry | null@@ -9512,6 +9516,8 @@ tabIndex: number | undefined@internal queries_: QueryCache@internal queryCostEnabled: boolean@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]+@internal registerRemovedReader: (component: AnyComponentDef) => number+@internal removedReaderCount: (component: AnyComponentDef) => number@internal resetIterationDepth: () => void@internal resetQueryCacheStats: () => void@internal resetQueryPool: () => voiddiff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.mdindex c84c22335..64a53fa74 100644--- a/sdk/etc/minigame.api.md+++ b/sdk/etc/minigame.api.md@@ -6612,6 +6612,9 @@ send: (data: string | ArrayBuffer) => void## RemovedQueryInstance — class @public```+@internal activateRetention: () => void+@internal dispose: () => void+@internal releaseThrough: (lastRunTick: number) => void@internal resetTick: (tick: number) => void[Symbol.iterator]: () => Iterator<Entity>isEmpty: () => boolean@@ -9736,15 +9739,16 @@ tabIndex: number | undefined```@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void@internal advanceTick: () => void@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean@internal applyEntityOrder: (entities: readonly Entity[]) => void@internal beginIteration: () => void@internal builtin: BuiltinBridge@internal changes_: ChangeTracker-@internal cleanRemovedBuffer: (beforeTick: number) => void@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void@internal disconnectCpp: () => void+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void@internal enableChangeTracking: (component: AnyComponentDef) => void@internal endIteration: () => void@internal getCppRegistry: () => CppRegistry | null@@ -9764,6 +9768,8 @@ tabIndex: number | undefined@internal queries_: QueryCache@internal queryCostEnabled: boolean@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]+@internal registerRemovedReader: (component: AnyComponentDef) => number+@internal removedReaderCount: (component: AnyComponentDef) => number@internal resetIterationDepth: () => void@internal resetQueryCacheStats: () => void@internal resetQueryPool: () => voiddiff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.mdindex e6d50228a..d450f2648 100644--- a/sdk/etc/wechat.api.md+++ b/sdk/etc/wechat.api.md@@ -6373,6 +6373,9 @@ send: (data: string | ArrayBuffer) => void## RemovedQueryInstance — class @public```+@internal activateRetention: () => void+@internal dispose: () => void+@internal releaseThrough: (lastRunTick: number) => void@internal resetTick: (tick: number) => void[Symbol.iterator]: () => Iterator<Entity>isEmpty: () => boolean@@ -9512,15 +9515,16 @@ tabIndex: number | undefined```@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void@internal advanceTick: () => void@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean@internal applyEntityOrder: (entities: readonly Entity[]) => void@internal beginIteration: () => void@internal builtin: BuiltinBridge@internal changes_: ChangeTracker-@internal cleanRemovedBuffer: (beforeTick: number) => void@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void@internal disconnectCpp: () => void+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void@internal enableChangeTracking: (component: AnyComponentDef) => void@internal endIteration: () => void@internal getCppRegistry: () => CppRegistry | null@@ -9540,6 +9544,8 @@ tabIndex: number | undefined@internal queries_: QueryCache@internal queryCostEnabled: boolean@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]+@internal registerRemovedReader: (component: AnyComponentDef) => number+@internal removedReaderCount: (component: AnyComponentDef) => number@internal resetIterationDepth: () => void@internal resetQueryCacheStats: () => void@internal resetQueryPool: () => voiddiff --git a/sdk/src/app/app.ts b/sdk/src/app/app.tsindex f13b26659..d8df71890 100644--- a/sdk/src/app/app.ts+++ b/sdk/src/app/app.ts@@ -1233,9 +1233,6 @@ export class App {}}- const REMOVED_BUFFER_RETENTION = 2;- this.world_.cleanRemovedBuffer(this.world_.getWorldTick() - REMOVED_BUFFER_RETENTION);-for (const observe of this.frameObservers_) {try { observe(delta * 1000); } catch (e) {log.error('app', 'Frame observer error', e);diff --git a/sdk/src/diagnostics/censusProbes.ts b/sdk/src/diagnostics/censusProbes.tsindex fc0f54e14..8f6b1c3c2 100644--- a/sdk/src/diagnostics/censusProbes.ts+++ b/sdk/src/diagnostics/censusProbes.ts@@ -67,8 +67,9 @@ export function installBuiltinCensusProbes(): void {counter('ecs.despawnCallbacks', s.despawnCallbacks, 'conserved'),counter('ecs.changeRows.added', s.changes.addedRows, 'conserved'),counter('ecs.changeRows.changed', s.changes.changedRows, 'conserved'),- // Drained by cleanRemovedBuffer, not by despawn — so it is legitimately- // non-zero mid-frame and must still not climb from cycle to cycle.+ // Held for registered history readers and pruned to their lowest+ // claim, so it is legitimately non-zero mid-frame and must still+ // not climb from cycle to cycle.counter('ecs.changeRows.removed', s.changes.removedRows, 'bounded'),counter('ecs.nameKeys', s.names.names, 'bounded'),counter('ecs.scriptStorages', s.scripts.storages, 'bounded'),diff --git a/sdk/src/ecs/ChangeTracker.ts b/sdk/src/ecs/ChangeTracker.tsindex 0bdc1fc69..eb7659662 100644--- a/sdk/src/ecs/ChangeTracker.ts+++ b/sdk/src/ecs/ChangeTracker.ts@@ -17,6 +17,11 @@ export class ChangeTracker {// The most recent worldTick at which ANY entity changed each component — an// O(1) "did anything change since tick T" gate (vs scanning the per-entity map).private componentLastChangedTick_ = new Map<symbol, number>();+ // Who still needs removal history, and from which tick. Removal rows are+ // produced ONLY for a component in here: `Changed(C)` needs to know the+ // topology moved, never which entity lost C at tick 738.+ private removedReaders_ = new Map<symbol, Map<number, number>>();+ private nextReaderId_ = 1;advanceTick(): void {this.worldTick_++;@@ -62,19 +67,71 @@ export class ChangeTracker {return result;}- cleanRemovedBuffer(beforeTick: number): void {- for (const [id, buffer] of this.componentRemovedBuffer_) {- let writeIdx = 0;- for (let i = 0; i < buffer.length; i++) {- if (buffer[i].tick >= beforeTick) {- buffer[writeIdx++] = buffer[i];- }- }- buffer.length = writeIdx;- if (writeIdx === 0) {- this.componentRemovedBuffer_.delete(id);- }+ /**+ * Take out a claim on `component`'s removal history. History begins HERE:+ * a new reader does not inherit rows another reader happened to leave, so+ * ownership is never ambiguous.+ */+ registerRemovedReader(component: AnyComponentDef): number {+ let readers = this.removedReaders_.get(component._id);+ if (!readers) {+ readers = new Map();+ this.removedReaders_.set(component._id, readers);}+ const id = this.nextReaderId_++;+ readers.set(id, this.worldTick_ + 1);+ return id;+ }++ /**+ * This reader has finished a run whose window ended at `lastRunTick`, so it+ * will never ask for a row at or before it again. `Removed` reads+ * `tick > lastRunTick`, which is why the claim is the tick after.+ */+ advanceRemovedReader(component: AnyComponentDef, readerId: number, lastRunTick: number): void {+ const readers = this.removedReaders_.get(component._id);+ if (!readers?.has(readerId)) return;+ readers.set(readerId, lastRunTick + 1);+ this.pruneRemoved_(component._id);+ }++ /** Give up the claim, and let the watermark move at once — history with no+ * owner is not history anyone can ask for. */+ disposeRemovedReader(component: AnyComponentDef, readerId: number): void {+ const readers = this.removedReaders_.get(component._id);+ if (!readers) return;+ readers.delete(readerId);+ this.pruneRemoved_(component._id);+ }++ /** @internal How many readers hold `component`'s history — for the fixtures+ * that assert a disposed reader stopped pinning it. */+ removedReaderCount(component: AnyComponentDef): number {+ return this.removedReaders_.get(component._id)?.size ?? 0;+ }++ /**+ * Drop what no reader of THIS component can still ask for. Per component,+ * never global: a slow reader of one component has no business pinning+ * another's history.+ */+ private pruneRemoved_(componentId: symbol): void {+ const readers = this.removedReaders_.get(componentId);+ if (!readers || readers.size === 0) {+ this.removedReaders_.delete(componentId);+ this.componentRemovedBuffer_.delete(componentId);+ return;+ }+ const buffer = this.componentRemovedBuffer_.get(componentId);+ if (!buffer) return;+ let safe = Infinity;+ for (const from of readers.values()) if (from < safe) safe = from;+ let writeIdx = 0;+ for (let i = 0; i < buffer.length; i++) {+ if (buffer[i].tick >= safe) buffer[writeIdx++] = buffer[i];+ }+ buffer.length = writeIdx;+ if (writeIdx === 0) this.componentRemovedBuffer_.delete(componentId);}recordAdded(component: AnyComponentDef, entity: Entity): void {@@ -118,13 +175,18 @@ export class ChangeTracker {* recorded for one kind and dropped for the other.*/recordRemovedById(componentId: symbol, entity: Entity): void {- if (!this.trackedComponents_.has(componentId)) return;- let buffer = this.componentRemovedBuffer_.get(componentId);- if (!buffer) {- buffer = [];- this.componentRemovedBuffer_.set(componentId, buffer);+ // A row is stored only for a component someone reads history of. With+ // `Changed(C)` alone this is the whole difference between a buffer that+ // grows with every despawn for the life of the world, and none at all.+ if (this.removedReaders_.has(componentId)) {+ let buffer = this.componentRemovedBuffer_.get(componentId);+ if (!buffer) {+ buffer = [];+ this.componentRemovedBuffer_.set(componentId, buffer);+ }+ buffer.push({ entity, tick: this.worldTick_ });}- buffer.push({ entity, tick: this.worldTick_ });+ if (!this.trackedComponents_.has(componentId)) return;this.componentAddedTicks_.get(componentId)?.delete(entity);this.componentChangedTicks_.get(componentId)?.delete(entity);// Losing a component IS a change to it. Without this, `anyChangedSince`@@ -138,9 +200,9 @@ export class ChangeTracker {/*** @internal Live map sizes for the resource census.*- * `removedRows` is the one that can run away: it is drained only by- * {@link cleanRemovedBuffer}, so a tracked component whose consumer stops- * calling that accumulates a row per despawn for the life of the world.+ * `removedRows` is bounded by what registered readers still owe: rows exist+ * only while a `Removed` reader holds the component, and are pruned to the+ * lowest claim whenever one advances or goes away.*/sizes(): { tracked: number; addedRows: number; changedRows: number; removedRows: number } {const rowsIn = (maps: Map<symbol, Map<Entity, number>>): number => {diff --git a/sdk/src/ecs/query.ts b/sdk/src/ecs/query.tsindex 0d76c362e..560ede913 100644--- a/sdk/src/ecs/query.ts+++ b/sdk/src/ecs/query.ts@@ -816,12 +816,40 @@ export class RemovedQueryInstance<T extends AnyComponentDef> implements Iterableprivate readonly world_: World;private readonly component_: T;private lastRunTick_: number;+ private readerId_: number | null = null;+ /** Constructing one has NO side effect on the world: a runner that builds+ * several params and throws on a later one would otherwise leave a claim+ * on removal history that nothing can ever release. */constructor(world: World, component: T, lastRunTick: number) {this.world_ = world;this.component_ = component;this.lastRunTick_ = lastRunTick;- world.enableChangeTracking(component);+ }++ /** @internal Claim the history, once the whole parameter set is committed. */+ activateRetention(): void {+ if (this.readerId_ !== null) return;+ this.world_.enableChangeTracking(this.component_);+ this.readerId_ = this.world_.registerRemovedReader(this.component_);+ }++ /**+ * @internal This run is over, so rows up to `lastRunTick` will never be+ * asked for again. NOT `resetTick`: a system may await mid-body, and a claim+ * released before the body read would let another reader's prune take rows+ * this one is still owed.+ */+ releaseThrough(lastRunTick: number): void {+ if (this.readerId_ === null) return;+ this.world_.advanceRemovedReader(this.component_, this.readerId_, lastRunTick);+ }++ /** @internal Give up the claim — the runner evicted or reset this system. */+ dispose(): void {+ if (this.readerId_ === null) return;+ this.world_.disposeRemovedReader(this.component_, this.readerId_);+ this.readerId_ = null;}/** @internal Update lastRunTick for reuse across system runs */diff --git a/sdk/src/ecs/system.ts b/sdk/src/ecs/system.tsindex 002d1ea81..82783c826 100644--- a/sdk/src/ecs/system.ts+++ b/sdk/src/ecs/system.ts@@ -462,6 +462,9 @@ export class SystemRunner {this.argsCache_.delete(systemId);this.systemTicks_.delete(systemId);this.queryCache_.delete(systemId);+ // Dropping the reference is not releasing the claim: a reader nothing+ // can reach still pins its component's removal history forever.+ for (const removed of this.removedCache_.get(systemId) ?? []) removed.dispose();this.removedCache_.delete(systemId);}@@ -470,6 +473,9 @@ export class SystemRunner {this.argsCache_.clear();this.systemTicks_.clear();this.queryCache_.clear();+ for (const removeds of this.removedCache_.values()) {+ for (const removed of removeds) removed.dispose();+ }this.removedCache_.clear();}@@ -534,6 +540,9 @@ export class SystemRunner {if (firstRun) {this.queryCache_.set(system._id, queries);this.removedCache_.set(system._id, removeds);+ // After the commit, never during: a param that threw halfway would+ // otherwise leave a claim on history with no owner to release it.+ for (const removed of removeds) removed.activateRetention();}const t0 = this.timings_ ? performance.now() : 0;@@ -597,7 +606,14 @@ export class SystemRunner {this.timings_.set(name, (this.timings_.get(name) ?? 0) + (performance.now() - t0));}this.world_.resetIterationDepth();- this.systemTicks_.set(system._id, this.world_.getWorldTick());+ // The window closes here, not where it opened: a system that awaited+ // mid-body reads its rows after `resetTick`, so releasing there would let+ // another reader prune what this one is still owed.+ const ranAt = this.world_.getWorldTick();+ for (const removed of this.removedCache_.get(system._id) ?? []) {+ removed.releaseThrough(ranAt);+ }+ this.systemTicks_.set(system._id, ranAt);}private harvestQueryCost_(system: SystemDef): void {diff --git a/sdk/src/ecs/world.ts b/sdk/src/ecs/world.tsindex bb5356f25..666f05d3c 100644--- a/sdk/src/ecs/world.ts+++ b/sdk/src/ecs/world.ts@@ -1197,9 +1197,24 @@ export class World {return this.changes_.getRemovedEntitiesSince(component, sinceTick);}- /** @internal */- cleanRemovedBuffer(beforeTick: number): void {- this.changes_.cleanRemovedBuffer(beforeTick);+ /** @internal A `Removed` reader's claim on history, from now on. */+ registerRemovedReader(component: AnyComponentDef): number {+ return this.changes_.registerRemovedReader(component);+ }++ /** @internal That reader is done with everything up to `lastRunTick`. */+ advanceRemovedReader(component: AnyComponentDef, readerId: number, lastRunTick: number): void {+ this.changes_.advanceRemovedReader(component, readerId, lastRunTick);+ }++ /** @internal The reader is gone; its share of the watermark goes with it. */+ disposeRemovedReader(component: AnyComponentDef, readerId: number): void {+ this.changes_.disposeRemovedReader(component, readerId);+ }++ /** @internal How many readers hold `component`'s removal history. */+ removedReaderCount(component: AnyComponentDef): number {+ return this.changes_.removedReaderCount(component);}/** @internal Mark component as changed without writing data (for in-place Mut query) */diff --git a/sdk/tests/query-mut-change-tracking.test.ts b/sdk/tests/query-mut-change-tracking.test.tsindex 0ae8aec77..82790af2c 100644--- a/sdk/tests/query-mut-change-tracking.test.ts+++ b/sdk/tests/query-mut-change-tracking.test.ts@@ -129,7 +129,7 @@ describe('removal is a change', () => {it('still lists the entity as removed since that tick', () => {const world = new World();- world.enableChangeTracking(Transform);+ world.registerRemovedReader(Transform);const e = world.spawn();world.insert(e, Transform, defaultTransform());diff --git a/sdk/tests/query.test.ts b/sdk/tests/query.test.tsindex 1ef3c0906..aa2f40d80 100644--- a/sdk/tests/query.test.ts+++ b/sdk/tests/query.test.ts@@ -537,37 +537,37 @@ describe('Query System', () => {expect(desc._component).toBe(Position);});+ // History begins when the reader claims it: a reader does not inherit+ // removals from before it existed, so the claim comes first.it('should iterate over removed entities', () => {- world.enableChangeTracking(Position);+ const instance = new RemovedQueryInstance(world, Position, -1);+ instance.activateRetention();const e1 = world.spawn();world.insert(e1, Position, { x: 1, y: 1 });world.remove(e1, Position);- const instance = new RemovedQueryInstance(world, Position, -1);- const removed = instance.toArray();- expect(removed).toContain(e1);+ expect(instance.toArray()).toContain(e1);});it('should report isEmpty correctly', () => {- world.enableChangeTracking(Position);- const instance = new RemovedQueryInstance(world, Position, Infinity);+ const instance = new RemovedQueryInstance(world, Position, -1);+ instance.activateRetention();expect(instance.isEmpty()).toBe(true);const e1 = world.spawn();world.insert(e1, Position, { x: 1, y: 1 });world.remove(e1, Position);- const instance2 = new RemovedQueryInstance(world, Position, -1);- expect(instance2.isEmpty()).toBe(false);+ expect(instance.isEmpty()).toBe(false);});it('should be iterable', () => {- world.enableChangeTracking(Position);+ const instance = new RemovedQueryInstance(world, Position, -1);+ instance.activateRetention();const e1 = world.spawn();world.insert(e1, Position, { x: 1, y: 1 });world.remove(e1, Position);- const instance = new RemovedQueryInstance(world, Position, -1);const results: number[] = [];for (const entity of instance) {results.push(entity);diff --git a/sdk/tests/removed-query-symmetry.test.ts b/sdk/tests/removed-query-symmetry.test.tsindex 7af63c289..9b6be559d 100644--- a/sdk/tests/removed-query-symmetry.test.ts+++ b/sdk/tests/removed-query-symmetry.test.ts@@ -7,12 +7,10 @@** Removal reaches the tracker two ways: recordRemoved(def) for an* explicit remove() and for builtins on despawn, recordRemovedById(id)- * for script components on despawn. Only the first checked whether- * anything tracks the component, so an untracked script component- * accumulated a removal record on every despawn — and a Removed() query- * created later reported despawns from before it existed, while the same- * query over a builtin, or over the same component removed explicitly,- * reported nothing.+ * for script components on despawn. Both must answer to the same gate —+ * a component with a registered history reader records, one without+ * records nothing — or the same despawn is history one way and gone the+ * other.*/import { describe, it, expect } from 'vitest';import { World } from '../src/ecs/world';@@ -21,8 +19,11 @@ import { defineComponent } from '../src/ecs/component';const Sc = defineComponent('RQSScript', { v: 0 });-function removedSince(world: World, tick: number): number[] {- return new RemovedQueryInstance(world, Sc, tick).toArray();+/** A reader that holds `Sc`'s history from now on, and what it can see. */+function readerFor(world: World): () => number[] {+ const inst = new RemovedQueryInstance(world, Sc, -1);+ inst.activateRetention();+ return () => inst.toArray();}describe('Removed() records only what something is tracking', () => {@@ -37,13 +38,14 @@ describe('Removed() records only what something is tracking', () => {removed.insert(b, Sc, { v: 1 });removed.remove(b, Sc);- expect(removedSince(despawned, -1)).toEqual([]);- expect(removedSince(removed, -1)).toEqual([]);+ // Nothing registered a reader, so neither road stored a row at all.+ expect(despawned.getRemovedEntitiesSince(Sc, -1)).toEqual([]);+ expect(removed.getRemovedEntitiesSince(Sc, -1)).toEqual([]);});- it('a tracked component records both ways', () => {+ it('a component with a reader records both ways', () => {const world = new World();- world.enableChangeTracking(Sc);+ const read = readerFor(world);const viaDespawn = world.spawn();world.insert(viaDespawn, Sc, { v: 1 });@@ -53,22 +55,23 @@ describe('Removed() records only what something is tracking', () => {world.despawn(viaDespawn);world.remove(viaRemove, Sc);- expect(removedSince(world, -1).sort()).toEqual([viaDespawn, viaRemove].sort());+ expect(read().sort()).toEqual([viaDespawn, viaRemove].sort());});- it('tracking starts when something asks, not retroactively', () => {+ it('history starts when a reader asks, not retroactively', () => {const world = new World();const before = world.spawn();world.insert(before, Sc, { v: 1 });world.despawn(before);- // The query's own ctor is what enables tracking.- expect(removedSince(world, -1)).toEqual([]);+ // A reader does not inherit what happened before it existed.+ const read = readerFor(world);+ expect(read()).toEqual([]);const after = world.spawn();world.insert(after, Sc, { v: 2 });world.despawn(after);- expect(removedSince(world, -1)).toEqual([after]);+ expect(read()).toEqual([after]);});it('losing a tracked component still moves the anyChangedSince watermark', () => {diff --git a/sdk/tests/removed-retention.test.ts b/sdk/tests/removed-retention.test.tsnew file mode 100644index 000000000..8bc1dc670--- /dev/null+++ b/sdk/tests/removed-retention.test.ts@@ -0,0 +1,256 @@+// SPDX-License-Identifier: Apache-2.0+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team+/**+ * @file Who owns removal history, and how long it lives.+ *+ * `Removed(C)` promises the entities that lost C since THIS system last ran.+ * That promise is only as good as the retention: a buffer pruned by anything+ * other than what its readers still need drops rows nobody consented to lose,+ * and the system sees an empty list rather than an error.+ */+import { describe, it, expect } from 'vitest';+import { App } from '../src/app/app';+import { defineComponent } from '../src/ecs/component';+import { defineSystem, Schedule, SystemRunner } from '../src/ecs/system';+import { World } from '../src/ecs/world';+import { ResourceStorage } from '../src/ecs/resource';+import { defineEvent, EventWriter } from '../src/ecs/event';+import { Removed, Query, Changed } from '../src/ecs/query';+import type { Entity } from '../src/types';++const Bullet = defineComponent('RetBullet', { n: 0 });+const Shell = defineComponent('RetShell', { n: 0 });++describe('removal history retention', () => {+ it('a system that runs every few frames still sees what it missed', async () => {+ const app = App.new();+ const world = app.world;++ let armed = false;+ const seen: Entity[][] = [];+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Removed(Bullet)],+ (removed) => { seen.push(removed.toArray()); },+ { name: 'RetReap' },+ ), { runIf: () => armed });++ // Run once so the reader exists and has a frame to be "since".+ armed = true;+ await app.tick(1 / 60);+ armed = false;++ const e = world.spawn();+ world.insert(e, Bullet, { n: 1 });+ await app.tick(1 / 60);+ world.remove(e, Bullet);++ // Long enough that a fixed-window prune would have dropped the row.+ for (let i = 0; i < 6; i++) await app.tick(1 / 60);++ armed = true;+ await app.tick(1 / 60);++ expect(seen.at(-1)).toEqual([e]);+ });+++ /** Everything still in `C`'s buffer, whoever it belongs to. */+ const held = (world: { getRemovedEntitiesSince(c: typeof Bullet, t: number): Entity[] }) =>+ world.getRemovedEntitiesSince(Bullet, -1);++ /** An app with a fast reader and a slow one over the same component. */+ function twoReaders() {+ const app = App.new();+ let slowArmed = false;+ const slowSaw: Entity[][] = [];+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Removed(Bullet)], () => { /* keeps up every frame */ }, { name: 'RetFast' },+ ));+ const slow = defineSystem(+ [Removed(Bullet)],+ (removed) => { slowSaw.push(removed.toArray()); },+ { name: 'RetSlow' },+ );+ app.addSystemToSchedule(Schedule.Update, slow, { runIf: () => slowArmed });+ return { app, slow, slowSaw, arm: (on: boolean) => { slowArmed = on; } };+ }++ it('holds history to the LOWEST claim, and lets go when it rises', async () => {+ const { app, slowSaw, arm } = twoReaders();+ const world = app.world;+ arm(true);+ await app.tick(1 / 60);+ arm(false);++ // A tick of its own: a removal recorded at the tick a reader last ran is+ // outside its next window (`tick > lastRunTick`) and owed to nobody.+ await app.tick(1 / 60);+ const e = world.spawn();+ world.insert(e, Bullet, { n: 1 });+ world.remove(e, Bullet);+ // The fast reader runs and releases; the slow one has not, so the row stays.+ for (let i = 0; i < 4; i++) await app.tick(1 / 60);+ expect(held(world)).toEqual([e]);++ arm(true);+ await app.tick(1 / 60);+ expect(slowSaw.at(-1)).toEqual([e]);+ // Both have now read past it, so nobody is owed it.+ await app.tick(1 / 60);+ expect(held(world)).toEqual([]);+ });++ it('a slow reader of one component does not pin another', async () => {+ const app = App.new();+ const world = app.world;+ let armed = false;+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Removed(Bullet)], () => { /* every frame */ }, { name: 'RetA' },+ ));+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Removed(Shell)], () => { /* only when armed */ }, { name: 'RetB' },+ ), { runIf: () => armed });+ armed = true;+ await app.tick(1 / 60);+ armed = false;+ // One frame with only the fast reader running, so the two claims differ.+ await app.tick(1 / 60);++ const a = world.spawn();+ world.insert(a, Bullet, { n: 1 });+ world.remove(a, Bullet);+ const b = world.spawn();+ world.insert(b, Shell, { n: 1 });+ world.remove(b, Shell);+ for (let i = 0; i < 4; i++) await app.tick(1 / 60);++ // A global watermark would hold both, because B's reader is behind.+ expect(world.getRemovedEntitiesSince(Bullet, -1)).toEqual([]);+ expect(world.getRemovedEntitiesSince(Shell, -1)).toEqual([b]);+ });++ it('evicting the slow system lets the watermark rise at once', async () => {+ const { app, slow, arm } = twoReaders();+ const world = app.world;+ arm(true);+ await app.tick(1 / 60);+ arm(false);++ await app.tick(1 / 60);+ const e = world.spawn();+ world.insert(e, Bullet, { n: 1 });+ world.remove(e, Bullet);+ for (let i = 0; i < 3; i++) await app.tick(1 / 60);+ expect(held(world)).toEqual([e]);++ app.removeSystem(slow._id);+ expect(held(world)).toEqual([]);+ expect(world.removedReaderCount(Bullet)).toBe(1);+ });++ it('an async system keeps its rows while it is parked', async () => {+ const app = App.new();+ const world = app.world;+ let saw: Entity[] | null = null;+ let armed = false;+ let other: number | null = null;+ let victim: Entity | null = null;++ // Removed IN-FRAME: the window is `tick > lastRunTick`, so a removal+ // between two frames carries the tick the reader just ran at and is+ // outside the next window by construction.+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [], () => {+ if (!armed || victim === null) return;+ world.remove(victim, Bullet);+ },+ { name: 'RetKiller' },+ ));++ // A second claim, advanced WHILE the reader below is parked mid-body.+ // Its own claim has not moved — it has not read yet — so the row is+ // still owed and the prune must leave it.+ const gate = () => new Promise<void>((resolve) => {+ queueMicrotask(() => {+ world.advanceRemovedReader(Bullet, other!, world.getWorldTick());+ resolve();+ });+ });++ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Removed(Bullet)],+ async (removed) => {+ if (!armed) return;+ await gate();+ saw = removed.toArray();+ },+ { name: 'RetAsync' },+ ));+ await app.tick(1 / 60);+ other = world.registerRemovedReader(Bullet);++ victim = world.spawn();+ world.insert(victim, Bullet, { n: 1 });+ armed = true;+ await app.tick(1 / 60);++ expect(saw).toEqual([victim]);+ });++ it('a reset gives up every claim it was holding', async () => {+ const world = new World();+ const runner = new SystemRunner(world, new ResourceStorage());+ const sys = defineSystem([Removed(Bullet)], () => { /* reads */ }, { name: 'RetReset' });++ runner.run(sys);+ expect(world.removedReaderCount(Bullet)).toBe(1);++ runner.reset();+ expect(world.removedReaderCount(Bullet)).toBe(0);+ // With nobody left to ask, the rows are not history — they are garbage.+ const e = world.spawn();+ world.insert(e, Bullet, { n: 1 });+ world.remove(e, Bullet);+ expect(world.getRemovedEntitiesSince(Bullet, -1)).toEqual([]);+ });++ it('a parameter set that fails to build leaves no claim behind', () => {+ const world = new World();+ // No event registry, so the SECOND parameter throws while the first has+ // already been constructed. A claim taken during construction would have+ // no owner to release it — and would pin this component forever.+ const runner = new SystemRunner(world, new ResourceStorage());+ const sys = defineSystem(+ [Removed(Bullet), EventWriter(defineEvent<{ n: number }>('RetBoom'))],+ () => { /* never runs */ },+ { name: 'RetBadParams' },+ );++ expect(() => runner.run(sys)).toThrow('EventRegistry');+ expect(world.removedReaderCount(Bullet)).toBe(0);+ });++ it('change tracking alone does not accumulate removal rows', async () => {+ const app = App.new();+ const world = app.world;++ // A Changed() reader, and nothing that asks for removal history.+ app.addSystemToSchedule(Schedule.Update, defineSystem(+ [Query(Changed(Bullet))],+ (q) => { for (const _row of q) { /* counted by the filter */ } },+ { name: 'RetWatch' },+ ));+ await app.tick(1 / 60);++ for (let i = 0; i < 200; i++) {+ const e = world.spawn();+ world.insert(e, Bullet, { n: i });+ world.remove(e, Bullet);+ }+ await app.tick(1 / 60);++ expect(world.getStorageSizes().changes.removedRows).toBe(0);+ // Losing a component is still a change to it — that half must survive.+ expect(world.anyChangedSince(Bullet, 0)).toBe(true);+ });+});diff --git a/sdk/tests/system.test.ts b/sdk/tests/system.test.tsindex d89b00e60..c05481de0 100644--- a/sdk/tests/system.test.ts+++ b/sdk/tests/system.test.ts@@ -62,6 +62,12 @@ function createMockWorld() {isChangedSince: vi.fn(() => false),getRemovedEntitiesSince: vi.fn(() => []),enableChangeTracking: vi.fn(),+ // Modelled with the runner's lifecycle in mind: it registers on commit,+ // releases on completion and disposes on evict, and a stub that answers+ // nothing would let a change to that order pass unnoticed here.+ registerRemovedReader: vi.fn(() => 1),+ advanceRemovedReader: vi.fn(),+ disposeRemovedReader: vi.fn(),resolveGetter: vi.fn(() => null),};diff --git a/sdk/tests/world.test.ts b/sdk/tests/world.test.tsindex 20531b7ed..9e54196f9 100644--- a/sdk/tests/world.test.ts+++ b/sdk/tests/world.test.ts@@ -653,7 +653,9 @@ describe('World', () => {it('getRemovedEntitiesSince should detect removed components', () => {const w = new World();- w.enableChangeTracking(Position);+ // History is produced for a component someone READS history of —+ // change tracking alone says the topology moved, not who lost what.+ w.registerRemovedReader(Position);const e = w.spawn();w.insert(e, Position, { x: 0, y: 0 });w.advanceTick();@@ -667,41 +669,10 @@ describe('World', () => {expect(w.getRemovedEntitiesSince(Velocity, -1)).toEqual([]);});- it('cleanRemovedBuffer should remove old entries', () => {- const w = new World();- w.enableChangeTracking(Position);- const e1 = w.spawn();- w.insert(e1, Position, { x: 0, y: 0 });- w.remove(e1, Position);-- w.advanceTick();- w.advanceTick();-- const e2 = w.spawn();- w.insert(e2, Position, { x: 0, y: 0 });- w.remove(e2, Position);-- w.cleanRemovedBuffer(1);- const removed = w.getRemovedEntitiesSince(Position, -1);- expect(removed).toContain(e2);- expect(removed).not.toContain(e1);- });-- it('cleanRemovedBuffer should delete empty buffers', () => {- const w = new World();- w.enableChangeTracking(Position);- const e = w.spawn();- w.insert(e, Position, { x: 0, y: 0 });- w.remove(e, Position);-- w.cleanRemovedBuffer(Infinity);- expect(w.getRemovedEntitiesSince(Position, -1)).toEqual([]);- });-it('despawn should record removed ticks for all components', () => {const w = new World();- w.enableChangeTracking(Position);- w.enableChangeTracking(Velocity);+ w.registerRemovedReader(Position);+ w.registerRemovedReader(Velocity);const e = w.spawn();w.insert(e, Position, { x: 0, y: 0 });w.insert(e, Velocity, { dx: 1, dy: 1 });