Skip to content

A fast 2D and 3D game engine — TypeScript SDK, C++/WebAssembly core, visual editor. Ship one project to web, desktop, WeChat MiniGames, playable ads, and native Android / iOS.

fix(ecs): removal history is kept for the readers that still need it

esengine2bed0668446d
diff --git a/docs/MUTATION_CONTRACT.md b/docs/MUTATION_CONTRACT.md
index c0f9c81dc..4d0637c11 100644
--- a/docs/MUTATION_CONTRACT.md
+++ b/docs/MUTATION_CONTRACT.md
@@ -175,6 +175,42 @@ shadow scan 看得见,ChangeTracker 看不见。** 只要还有一条这样的
判据现在先查 `sdk/src → sdk/dist → template` 这条链,陈旧就报 `did NOT run`(exit 2)。
(`.generated.ts` 不计入,原生构建会在打包之后重写它们。)
+## 3.7 PR6d:removal 历史有主了
+
+`cleanRemovedBuffer` 是一个**没有归属权**的全局 destructive API,而且它有一个调用者:
+`App.tick` 每帧末尾 `cleanRemovedBuffer(worldTick - 2)`。所以这不是内存问题,是**正确性缺陷**——
+任何超过 2 帧没运行的 `Removed(C)` system(`runIf` 关着、FixedUpdate 没到步、条件系统)
+会静默丢失它本该看到的 removal。判据 `sdk/tests/removed-retention.test.ts` 第一条先红在这里。
+
+现在是 **per-component 的 reader watermark**:
+
+```
+ChangeTracker
+├─ 变更观测:Added / Changed / anyChangedSince ← trackedComponents_
+└─ removal 历史:每 component 一张 reader 表 ← removedReaders_
+ readerId → retainFromTick,prune 到最低的那个
+```
+
+- **watermark 按 component 分**,不是全局:一个组件的慢 reader 没有资格钉住另一个组件的历史。
+- **cursor 在 `flushSystem_` 推进**,不在 `resetTick`——system 可以 `await` 到一半,
+ 在 body 读之前就宣告「读完了」,会让另一个 reader 的 prune 删掉它仍然欠着的行。
+- **认领发生在参数提交之后**,不在构造函数里:一组参数中途抛异常时,
+ 已经建好的那个 reader 没有任何人能释放它,那个 component 的历史从此永久被钉住。
+- **`evict()` / `reset()` 真的 dispose**:丢掉 JS 引用不等于放弃认领。
+- **没有 reader 就不产生 removal 行**。`Changed(C)` 需要知道拓扑动了
+ (`componentLastChangedTick_` 照常更新,UI 布局依赖它),但不需要知道谁在 tick 738 掉了 C。
+ 于是「开着 Changed(C)、没有 Removed(C)、despawn 十万次」从十万行变成 **0 行**。
+- **历史从 reader 认领的那一刻开始**(`retainFromTick = worldTick + 1`),
+ 新 reader 不认领别人碰巧留下的古老 removal。
+
+慢的**活着的** reader 钉住历史不是 bug——`Removed` 的语义就是「自这个 system 上次运行以来」。
+真正的 bug 是已经 evict 掉的 reader 还在钉,所以 ownership 才必须显式。
+
+八条判据,六次 sabotage 各红一条:把 release 挪到 body 之前/改成全局 watermark/
+`evict` 不 dispose/`reset` 不 dispose/在构造函数里认领/让任何 tracked 组件都写 removal 行。
+
+`World.cleanRemovedBuffer` 已删——一个谁都能替别人删历史的接缝,在有了归属权之后没有位置。
+
## 4. 待裁定
普查给出的结论是明确的:**`get`/`tryGet`/裸 `Query` 必须收成只读**,而且必须是
@@ -188,7 +224,4 @@ shadow scan 看得见,ChangeTracker 看不见。** 只要还有一条这样的
同时未判定、留给后续的两件:
- ~~**AOT 写回**~~:已实测,见 3.6。
-- **`cleanRemovedBuffer` 的归属权**:它是全局 destructive cleanup,而每个
- `RemovedQueryInstance` 各自持有 `lastRunTick`。复制层若替所有 `Removed()` 消费者删历史,
- 会让 frontier 落后的消费者永久错过那段 removal。这是正确性契约而不是内存优化,
- 属于 PR6d(共享 retention watermark),本文只记录事实。
+- ~~**`cleanRemovedBuffer` 的归属权**~~:已收掉,见 3.7。
diff --git a/sdk/benchmarks/system.bench.ts b/sdk/benchmarks/system.bench.ts
index 9f0cd97d5..632f7bfee 100644
--- a/sdk/benchmarks/system.bench.ts
+++ b/sdk/benchmarks/system.bench.ts
@@ -208,6 +208,5 @@ describe('System - Full frame simulation (10 systems)', () => {
for (const sys of systems) {
runner.run(sys);
}
- world.cleanRemovedBuffer(2);
});
});
diff --git a/sdk/benchmarks/world-internals.bench.ts b/sdk/benchmarks/world-internals.bench.ts
index d318c32c1..ba384d2b8 100644
--- a/sdk/benchmarks/world-internals.bench.ts
+++ b/sdk/benchmarks/world-internals.bench.ts
@@ -44,9 +44,6 @@ describe('World internals - per-frame operations', () => {
world.resetQueryPool();
});
- bench('cleanRemovedBuffer (nothing to clean)', () => {
- world.cleanRemovedBuffer(2);
- });
});
describe('World internals - getEntitiesWithComponents (cache behavior)', () => {
diff --git a/sdk/etc/index.api.md b/sdk/etc/index.api.md
index dd6007fc1..b1b6cfd67 100644
--- a/sdk/etc/index.api.md
+++ b/sdk/etc/index.api.md
@@ -6371,6 +6371,9 @@ send: (data: string | ArrayBuffer) => void
## RemovedQueryInstance — class @public
```
+@internal activateRetention: () => void
+@internal dispose: () => void
+@internal releaseThrough: (lastRunTick: number) => void
@internal resetTick: (tick: number) => void
[Symbol.iterator]: () => Iterator<Entity>
isEmpty: () => boolean
@@ -9489,15 +9492,16 @@ tabIndex: number | undefined
```
@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined
@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined
+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void
@internal advanceTick: () => void
@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean
@internal applyEntityOrder: (entities: readonly Entity[]) => void
@internal beginIteration: () => void
@internal builtin: BuiltinBridge
@internal changes_: ChangeTracker
-@internal cleanRemovedBuffer: (beforeTick: number) => void
@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void
@internal disconnectCpp: () => void
+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void
@internal enableChangeTracking: (component: AnyComponentDef) => void
@internal endIteration: () => void
@internal getCppRegistry: () => CppRegistry | null
@@ -9517,6 +9521,8 @@ tabIndex: number | undefined
@internal queries_: QueryCache
@internal queryCostEnabled: boolean
@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]
+@internal registerRemovedReader: (component: AnyComponentDef) => number
+@internal removedReaderCount: (component: AnyComponentDef) => number
@internal resetIterationDepth: () => void
@internal resetQueryCacheStats: () => void
@internal resetQueryPool: () => void
diff --git a/sdk/etc/index.native.api.md b/sdk/etc/index.native.api.md
index 9f4948c6d..85cd92d02 100644
--- a/sdk/etc/index.native.api.md
+++ b/sdk/etc/index.native.api.md
@@ -6564,6 +6564,9 @@ send: (data: string | ArrayBuffer) => void
## RemovedQueryInstance — class @public
```
+@internal activateRetention: () => void
+@internal dispose: () => void
+@internal releaseThrough: (lastRunTick: number) => void
@internal resetTick: (tick: number) => void
[Symbol.iterator]: () => Iterator<Entity>
isEmpty: () => boolean
@@ -9682,15 +9685,16 @@ tabIndex: number | undefined
```
@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined
@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined
+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void
@internal advanceTick: () => void
@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean
@internal applyEntityOrder: (entities: readonly Entity[]) => void
@internal beginIteration: () => void
@internal builtin: BuiltinBridge
@internal changes_: ChangeTracker
-@internal cleanRemovedBuffer: (beforeTick: number) => void
@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void
@internal disconnectCpp: () => void
+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void
@internal enableChangeTracking: (component: AnyComponentDef) => void
@internal endIteration: () => void
@internal getCppRegistry: () => CppRegistry | null
@@ -9710,6 +9714,8 @@ tabIndex: number | undefined
@internal queries_: QueryCache
@internal queryCostEnabled: boolean
@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]
+@internal registerRemovedReader: (component: AnyComponentDef) => number
+@internal removedReaderCount: (component: AnyComponentDef) => number
@internal resetIterationDepth: () => void
@internal resetQueryCacheStats: () => void
@internal resetQueryPool: () => void
diff --git a/sdk/etc/index.node.api.md b/sdk/etc/index.node.api.md
index cdf9af7cc..65e451031 100644
--- a/sdk/etc/index.node.api.md
+++ b/sdk/etc/index.node.api.md
@@ -6366,6 +6366,9 @@ send: (data: string | ArrayBuffer) => void
## RemovedQueryInstance — class @public
```
+@internal activateRetention: () => void
+@internal dispose: () => void
+@internal releaseThrough: (lastRunTick: number) => void
@internal resetTick: (tick: number) => void
[Symbol.iterator]: () => Iterator<Entity>
isEmpty: () => boolean
@@ -9484,15 +9487,16 @@ tabIndex: number | undefined
```
@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined
@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined
+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void
@internal advanceTick: () => void
@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean
@internal applyEntityOrder: (entities: readonly Entity[]) => void
@internal beginIteration: () => void
@internal builtin: BuiltinBridge
@internal changes_: ChangeTracker
-@internal cleanRemovedBuffer: (beforeTick: number) => void
@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void
@internal disconnectCpp: () => void
+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void
@internal enableChangeTracking: (component: AnyComponentDef) => void
@internal endIteration: () => void
@internal getCppRegistry: () => CppRegistry | null
@@ -9512,6 +9516,8 @@ tabIndex: number | undefined
@internal queries_: QueryCache
@internal queryCostEnabled: boolean
@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]
+@internal registerRemovedReader: (component: AnyComponentDef) => number
+@internal removedReaderCount: (component: AnyComponentDef) => number
@internal resetIterationDepth: () => void
@internal resetQueryCacheStats: () => void
@internal resetQueryPool: () => void
diff --git a/sdk/etc/minigame.api.md b/sdk/etc/minigame.api.md
index c84c22335..64a53fa74 100644
--- a/sdk/etc/minigame.api.md
+++ b/sdk/etc/minigame.api.md
@@ -6612,6 +6612,9 @@ send: (data: string | ArrayBuffer) => void
## RemovedQueryInstance — class @public
```
+@internal activateRetention: () => void
+@internal dispose: () => void
+@internal releaseThrough: (lastRunTick: number) => void
@internal resetTick: (tick: number) => void
[Symbol.iterator]: () => Iterator<Entity>
isEmpty: () => boolean
@@ -9736,15 +9739,16 @@ tabIndex: number | undefined
```
@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined
@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined
+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void
@internal advanceTick: () => void
@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean
@internal applyEntityOrder: (entities: readonly Entity[]) => void
@internal beginIteration: () => void
@internal builtin: BuiltinBridge
@internal changes_: ChangeTracker
-@internal cleanRemovedBuffer: (beforeTick: number) => void
@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void
@internal disconnectCpp: () => void
+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void
@internal enableChangeTracking: (component: AnyComponentDef) => void
@internal endIteration: () => void
@internal getCppRegistry: () => CppRegistry | null
@@ -9764,6 +9768,8 @@ tabIndex: number | undefined
@internal queries_: QueryCache
@internal queryCostEnabled: boolean
@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]
+@internal registerRemovedReader: (component: AnyComponentDef) => number
+@internal removedReaderCount: (component: AnyComponentDef) => number
@internal resetIterationDepth: () => void
@internal resetQueryCacheStats: () => void
@internal resetQueryPool: () => void
diff --git a/sdk/etc/wechat.api.md b/sdk/etc/wechat.api.md
index e6d50228a..d450f2648 100644
--- a/sdk/etc/wechat.api.md
+++ b/sdk/etc/wechat.api.md
@@ -6373,6 +6373,9 @@ send: (data: string | ArrayBuffer) => void
## RemovedQueryInstance — class @public
```
+@internal activateRetention: () => void
+@internal dispose: () => void
+@internal releaseThrough: (lastRunTick: number) => void
@internal resetTick: (tick: number) => void
[Symbol.iterator]: () => Iterator<Entity>
isEmpty: () => boolean
@@ -9512,15 +9515,16 @@ tabIndex: number | undefined
```
@internal addressOfComponent: (component: AnyComponentDef, entity: Entity) => number | undefined
@internal addressResolver: (component: AnyComponentDef) => (entity: Entity) => number | undefined
+@internal advanceRemovedReader: (component: AnyComponentDef, readerId: number, lastRunTick: number) => void
@internal advanceTick: () => void
@internal anyChangedSince: (component: AnyComponentDef, sinceTick: number) => boolean
@internal applyEntityOrder: (entities: readonly Entity[]) => void
@internal beginIteration: () => void
@internal builtin: BuiltinBridge
@internal changes_: ChangeTracker
-@internal cleanRemovedBuffer: (beforeTick: number) => void
@internal connectCpp: (cppRegistry: CppRegistry, module?: ESEngineModule, options?: BridgeConnectOptions) => void
@internal disconnectCpp: () => void
+@internal disposeRemovedReader: (component: AnyComponentDef, readerId: number) => void
@internal enableChangeTracking: (component: AnyComponentDef) => void
@internal endIteration: () => void
@internal getCppRegistry: () => CppRegistry | null
@@ -9540,6 +9544,8 @@ tabIndex: number | undefined
@internal queries_: QueryCache
@internal queryCostEnabled: boolean
@internal queryEntities: (components: AnyComponentDef[], withFilters?: AnyComponentDef[], withoutFilters?: AnyComponentDef[], precomputedKey?: string, filter?: QueryFilter, precomputedDepIds?: symbol[]) => readonly Entity[]
+@internal registerRemovedReader: (component: AnyComponentDef) => number
+@internal removedReaderCount: (component: AnyComponentDef) => number
@internal resetIterationDepth: () => void
@internal resetQueryCacheStats: () => void
@internal resetQueryPool: () => void
diff --git a/sdk/src/app/app.ts b/sdk/src/app/app.ts
index f13b26659..d8df71890 100644
--- a/sdk/src/app/app.ts
+++ b/sdk/src/app/app.ts
@@ -1233,9 +1233,6 @@ export class App {
}
}
- const REMOVED_BUFFER_RETENTION = 2;
- this.world_.cleanRemovedBuffer(this.world_.getWorldTick() - REMOVED_BUFFER_RETENTION);
-
for (const observe of this.frameObservers_) {
try { observe(delta * 1000); } catch (e) {
log.error('app', 'Frame observer error', e);
diff --git a/sdk/src/diagnostics/censusProbes.ts b/sdk/src/diagnostics/censusProbes.ts
index fc0f54e14..8f6b1c3c2 100644
--- a/sdk/src/diagnostics/censusProbes.ts
+++ b/sdk/src/diagnostics/censusProbes.ts
@@ -67,8 +67,9 @@ export function installBuiltinCensusProbes(): void {
counter('ecs.despawnCallbacks', s.despawnCallbacks, 'conserved'),
counter('ecs.changeRows.added', s.changes.addedRows, 'conserved'),
counter('ecs.changeRows.changed', s.changes.changedRows, 'conserved'),
- // Drained by cleanRemovedBuffer, not by despawn — so it is legitimately
- // non-zero mid-frame and must still not climb from cycle to cycle.
+ // Held for registered history readers and pruned to their lowest
+ // claim, so it is legitimately non-zero mid-frame and must still
+ // not climb from cycle to cycle.
counter('ecs.changeRows.removed', s.changes.removedRows, 'bounded'),
counter('ecs.nameKeys', s.names.names, 'bounded'),
counter('ecs.scriptStorages', s.scripts.storages, 'bounded'),
diff --git a/sdk/src/ecs/ChangeTracker.ts b/sdk/src/ecs/ChangeTracker.ts
index 0bdc1fc69..eb7659662 100644
--- a/sdk/src/ecs/ChangeTracker.ts
+++ b/sdk/src/ecs/ChangeTracker.ts
@@ -17,6 +17,11 @@ export class ChangeTracker {
// The most recent worldTick at which ANY entity changed each component — an
// O(1) "did anything change since tick T" gate (vs scanning the per-entity map).
private componentLastChangedTick_ = new Map<symbol, number>();
+ // Who still needs removal history, and from which tick. Removal rows are
+ // produced ONLY for a component in here: `Changed(C)` needs to know the
+ // topology moved, never which entity lost C at tick 738.
+ private removedReaders_ = new Map<symbol, Map<number, number>>();
+ private nextReaderId_ = 1;
advanceTick(): void {
this.worldTick_++;
@@ -62,19 +67,71 @@ export class ChangeTracker {
return result;
}
- cleanRemovedBuffer(beforeTick: number): void {
- for (const [id, buffer] of this.componentRemovedBuffer_) {
- let writeIdx = 0;
- for (let i = 0; i < buffer.length; i++) {
- if (buffer[i].tick >= beforeTick) {
- buffer[writeIdx++] = buffer[i];
- }
- }
- buffer.length = writeIdx;
- if (writeIdx === 0) {
- this.componentRemovedBuffer_.delete(id);
- }
+ /**
+ * Take out a claim on `component`'s removal history. History begins HERE:
+ * a new reader does not inherit rows another reader happened to leave, so
+ * ownership is never ambiguous.
+ */
+ registerRemovedReader(component: AnyComponentDef): number {
+ let readers = this.removedReaders_.get(component._id);
+ if (!readers) {
+ readers = new Map();
+ this.removedReaders_.set(component._id, readers);
}
+ const id = this.nextReaderId_++;
+ readers.set(id, this.worldTick_ + 1);
+ return id;
+ }
+
+ /**
+ * This reader has finished a run whose window ended at `lastRunTick`, so it
+ * will never ask for a row at or before it again. `Removed` reads
+ * `tick > lastRunTick`, which is why the claim is the tick after.
+ */
+ advanceRemovedReader(component: AnyComponentDef, readerId: number, lastRunTick: number): void {
+ const readers = this.removedReaders_.get(component._id);
+ if (!readers?.has(readerId)) return;
+ readers.set(readerId, lastRunTick + 1);
+ this.pruneRemoved_(component._id);
+ }
+
+ /** Give up the claim, and let the watermark move at once — history with no
+ * owner is not history anyone can ask for. */
+ disposeRemovedReader(component: AnyComponentDef, readerId: number): void {
+ const readers = this.removedReaders_.get(component._id);
+ if (!readers) return;
+ readers.delete(readerId);
+ this.pruneRemoved_(component._id);
+ }
+
+ /** @internal How many readers hold `component`'s history — for the fixtures
+ * that assert a disposed reader stopped pinning it. */
+ removedReaderCount(component: AnyComponentDef): number {
+ return this.removedReaders_.get(component._id)?.size ?? 0;
+ }
+
+ /**
+ * Drop what no reader of THIS component can still ask for. Per component,
+ * never global: a slow reader of one component has no business pinning
+ * another's history.
+ */
+ private pruneRemoved_(componentId: symbol): void {
+ const readers = this.removedReaders_.get(componentId);
+ if (!readers || readers.size === 0) {
+ this.removedReaders_.delete(componentId);
+ this.componentRemovedBuffer_.delete(componentId);
+ return;
+ }
+ const buffer = this.componentRemovedBuffer_.get(componentId);
+ if (!buffer) return;
+ let safe = Infinity;
+ for (const from of readers.values()) if (from < safe) safe = from;
+ let writeIdx = 0;
+ for (let i = 0; i < buffer.length; i++) {
+ if (buffer[i].tick >= safe) buffer[writeIdx++] = buffer[i];
+ }
+ buffer.length = writeIdx;
+ if (writeIdx === 0) this.componentRemovedBuffer_.delete(componentId);
}
recordAdded(component: AnyComponentDef, entity: Entity): void {
@@ -118,13 +175,18 @@ export class ChangeTracker {
* recorded for one kind and dropped for the other.
*/
recordRemovedById(componentId: symbol, entity: Entity): void {
- if (!this.trackedComponents_.has(componentId)) return;
- let buffer = this.componentRemovedBuffer_.get(componentId);
- if (!buffer) {
- buffer = [];
- this.componentRemovedBuffer_.set(componentId, buffer);
+ // A row is stored only for a component someone reads history of. With
+ // `Changed(C)` alone this is the whole difference between a buffer that
+ // grows with every despawn for the life of the world, and none at all.
+ if (this.removedReaders_.has(componentId)) {
+ let buffer = this.componentRemovedBuffer_.get(componentId);
+ if (!buffer) {
+ buffer = [];
+ this.componentRemovedBuffer_.set(componentId, buffer);
+ }
+ buffer.push({ entity, tick: this.worldTick_ });
}
- buffer.push({ entity, tick: this.worldTick_ });
+ if (!this.trackedComponents_.has(componentId)) return;
this.componentAddedTicks_.get(componentId)?.delete(entity);
this.componentChangedTicks_.get(componentId)?.delete(entity);
// Losing a component IS a change to it. Without this, `anyChangedSince`
@@ -138,9 +200,9 @@ export class ChangeTracker {
/**
* @internal Live map sizes for the resource census.
*
- * `removedRows` is the one that can run away: it is drained only by
- * {@link cleanRemovedBuffer}, so a tracked component whose consumer stops
- * calling that accumulates a row per despawn for the life of the world.
+ * `removedRows` is bounded by what registered readers still owe: rows exist
+ * only while a `Removed` reader holds the component, and are pruned to the
+ * lowest claim whenever one advances or goes away.
*/
sizes(): { tracked: number; addedRows: number; changedRows: number; removedRows: number } {
const rowsIn = (maps: Map<symbol, Map<Entity, number>>): number => {
diff --git a/sdk/src/ecs/query.ts b/sdk/src/ecs/query.ts
index 0d76c362e..560ede913 100644
--- a/sdk/src/ecs/query.ts
+++ b/sdk/src/ecs/query.ts
@@ -816,12 +816,40 @@ export class RemovedQueryInstance<T extends AnyComponentDef> implements Iterable
private readonly world_: World;
private readonly component_: T;
private lastRunTick_: number;
+ private readerId_: number | null = null;
+ /** Constructing one has NO side effect on the world: a runner that builds
+ * several params and throws on a later one would otherwise leave a claim
+ * on removal history that nothing can ever release. */
constructor(world: World, component: T, lastRunTick: number) {
this.world_ = world;
this.component_ = component;
this.lastRunTick_ = lastRunTick;
- world.enableChangeTracking(component);
+ }
+
+ /** @internal Claim the history, once the whole parameter set is committed. */
+ activateRetention(): void {
+ if (this.readerId_ !== null) return;
+ this.world_.enableChangeTracking(this.component_);
+ this.readerId_ = this.world_.registerRemovedReader(this.component_);
+ }
+
+ /**
+ * @internal This run is over, so rows up to `lastRunTick` will never be
+ * asked for again. NOT `resetTick`: a system may await mid-body, and a claim
+ * released before the body read would let another reader's prune take rows
+ * this one is still owed.
+ */
+ releaseThrough(lastRunTick: number): void {
+ if (this.readerId_ === null) return;
+ this.world_.advanceRemovedReader(this.component_, this.readerId_, lastRunTick);
+ }
+
+ /** @internal Give up the claim — the runner evicted or reset this system. */
+ dispose(): void {
+ if (this.readerId_ === null) return;
+ this.world_.disposeRemovedReader(this.component_, this.readerId_);
+ this.readerId_ = null;
}
/** @internal Update lastRunTick for reuse across system runs */
diff --git a/sdk/src/ecs/system.ts b/sdk/src/ecs/system.ts
index 002d1ea81..82783c826 100644
--- a/sdk/src/ecs/system.ts
+++ b/sdk/src/ecs/system.ts
@@ -462,6 +462,9 @@ export class SystemRunner {
this.argsCache_.delete(systemId);
this.systemTicks_.delete(systemId);
this.queryCache_.delete(systemId);
+ // Dropping the reference is not releasing the claim: a reader nothing
+ // can reach still pins its component's removal history forever.
+ for (const removed of this.removedCache_.get(systemId) ?? []) removed.dispose();
this.removedCache_.delete(systemId);
}
@@ -470,6 +473,9 @@ export class SystemRunner {
this.argsCache_.clear();
this.systemTicks_.clear();
this.queryCache_.clear();
+ for (const removeds of this.removedCache_.values()) {
+ for (const removed of removeds) removed.dispose();
+ }
this.removedCache_.clear();
}
@@ -534,6 +540,9 @@ export class SystemRunner {
if (firstRun) {
this.queryCache_.set(system._id, queries);
this.removedCache_.set(system._id, removeds);
+ // After the commit, never during: a param that threw halfway would
+ // otherwise leave a claim on history with no owner to release it.
+ for (const removed of removeds) removed.activateRetention();
}
const t0 = this.timings_ ? performance.now() : 0;
@@ -597,7 +606,14 @@ export class SystemRunner {
this.timings_.set(name, (this.timings_.get(name) ?? 0) + (performance.now() - t0));
}
this.world_.resetIterationDepth();
- this.systemTicks_.set(system._id, this.world_.getWorldTick());
+ // The window closes here, not where it opened: a system that awaited
+ // mid-body reads its rows after `resetTick`, so releasing there would let
+ // another reader prune what this one is still owed.
+ const ranAt = this.world_.getWorldTick();
+ for (const removed of this.removedCache_.get(system._id) ?? []) {
+ removed.releaseThrough(ranAt);
+ }
+ this.systemTicks_.set(system._id, ranAt);
}
private harvestQueryCost_(system: SystemDef): void {
diff --git a/sdk/src/ecs/world.ts b/sdk/src/ecs/world.ts
index bb5356f25..666f05d3c 100644
--- a/sdk/src/ecs/world.ts
+++ b/sdk/src/ecs/world.ts
@@ -1197,9 +1197,24 @@ export class World {
return this.changes_.getRemovedEntitiesSince(component, sinceTick);
}
- /** @internal */
- cleanRemovedBuffer(beforeTick: number): void {
- this.changes_.cleanRemovedBuffer(beforeTick);
+ /** @internal A `Removed` reader's claim on history, from now on. */
+ registerRemovedReader(component: AnyComponentDef): number {
+ return this.changes_.registerRemovedReader(component);
+ }
+
+ /** @internal That reader is done with everything up to `lastRunTick`. */
+ advanceRemovedReader(component: AnyComponentDef, readerId: number, lastRunTick: number): void {
+ this.changes_.advanceRemovedReader(component, readerId, lastRunTick);
+ }
+
+ /** @internal The reader is gone; its share of the watermark goes with it. */
+ disposeRemovedReader(component: AnyComponentDef, readerId: number): void {
+ this.changes_.disposeRemovedReader(component, readerId);
+ }
+
+ /** @internal How many readers hold `component`'s removal history. */
+ removedReaderCount(component: AnyComponentDef): number {
+ return this.changes_.removedReaderCount(component);
}
/** @internal Mark component as changed without writing data (for in-place Mut query) */
diff --git a/sdk/tests/query-mut-change-tracking.test.ts b/sdk/tests/query-mut-change-tracking.test.ts
index 0ae8aec77..82790af2c 100644
--- a/sdk/tests/query-mut-change-tracking.test.ts
+++ b/sdk/tests/query-mut-change-tracking.test.ts
@@ -129,7 +129,7 @@ describe('removal is a change', () => {
it('still lists the entity as removed since that tick', () => {
const world = new World();
- world.enableChangeTracking(Transform);
+ world.registerRemovedReader(Transform);
const e = world.spawn();
world.insert(e, Transform, defaultTransform());
diff --git a/sdk/tests/query.test.ts b/sdk/tests/query.test.ts
index 1ef3c0906..aa2f40d80 100644
--- a/sdk/tests/query.test.ts
+++ b/sdk/tests/query.test.ts
@@ -537,37 +537,37 @@ describe('Query System', () => {
expect(desc._component).toBe(Position);
});
+ // History begins when the reader claims it: a reader does not inherit
+ // removals from before it existed, so the claim comes first.
it('should iterate over removed entities', () => {
- world.enableChangeTracking(Position);
+ const instance = new RemovedQueryInstance(world, Position, -1);
+ instance.activateRetention();
const e1 = world.spawn();
world.insert(e1, Position, { x: 1, y: 1 });
world.remove(e1, Position);
- const instance = new RemovedQueryInstance(world, Position, -1);
- const removed = instance.toArray();
- expect(removed).toContain(e1);
+ expect(instance.toArray()).toContain(e1);
});
it('should report isEmpty correctly', () => {
- world.enableChangeTracking(Position);
- const instance = new RemovedQueryInstance(world, Position, Infinity);
+ const instance = new RemovedQueryInstance(world, Position, -1);
+ instance.activateRetention();
expect(instance.isEmpty()).toBe(true);
const e1 = world.spawn();
world.insert(e1, Position, { x: 1, y: 1 });
world.remove(e1, Position);
- const instance2 = new RemovedQueryInstance(world, Position, -1);
- expect(instance2.isEmpty()).toBe(false);
+ expect(instance.isEmpty()).toBe(false);
});
it('should be iterable', () => {
- world.enableChangeTracking(Position);
+ const instance = new RemovedQueryInstance(world, Position, -1);
+ instance.activateRetention();
const e1 = world.spawn();
world.insert(e1, Position, { x: 1, y: 1 });
world.remove(e1, Position);
- const instance = new RemovedQueryInstance(world, Position, -1);
const results: number[] = [];
for (const entity of instance) {
results.push(entity);
diff --git a/sdk/tests/removed-query-symmetry.test.ts b/sdk/tests/removed-query-symmetry.test.ts
index 7af63c289..9b6be559d 100644
--- a/sdk/tests/removed-query-symmetry.test.ts
+++ b/sdk/tests/removed-query-symmetry.test.ts
@@ -7,12 +7,10 @@
*
* Removal reaches the tracker two ways: recordRemoved(def) for an
* explicit remove() and for builtins on despawn, recordRemovedById(id)
- * for script components on despawn. Only the first checked whether
- * anything tracks the component, so an untracked script component
- * accumulated a removal record on every despawn — and a Removed() query
- * created later reported despawns from before it existed, while the same
- * query over a builtin, or over the same component removed explicitly,
- * reported nothing.
+ * for script components on despawn. Both must answer to the same gate —
+ * a component with a registered history reader records, one without
+ * records nothing — or the same despawn is history one way and gone the
+ * other.
*/
import { describe, it, expect } from 'vitest';
import { World } from '../src/ecs/world';
@@ -21,8 +19,11 @@ import { defineComponent } from '../src/ecs/component';
const Sc = defineComponent('RQSScript', { v: 0 });
-function removedSince(world: World, tick: number): number[] {
- return new RemovedQueryInstance(world, Sc, tick).toArray();
+/** A reader that holds `Sc`'s history from now on, and what it can see. */
+function readerFor(world: World): () => number[] {
+ const inst = new RemovedQueryInstance(world, Sc, -1);
+ inst.activateRetention();
+ return () => inst.toArray();
}
describe('Removed() records only what something is tracking', () => {
@@ -37,13 +38,14 @@ describe('Removed() records only what something is tracking', () => {
removed.insert(b, Sc, { v: 1 });
removed.remove(b, Sc);
- expect(removedSince(despawned, -1)).toEqual([]);
- expect(removedSince(removed, -1)).toEqual([]);
+ // Nothing registered a reader, so neither road stored a row at all.
+ expect(despawned.getRemovedEntitiesSince(Sc, -1)).toEqual([]);
+ expect(removed.getRemovedEntitiesSince(Sc, -1)).toEqual([]);
});
- it('a tracked component records both ways', () => {
+ it('a component with a reader records both ways', () => {
const world = new World();
- world.enableChangeTracking(Sc);
+ const read = readerFor(world);
const viaDespawn = world.spawn();
world.insert(viaDespawn, Sc, { v: 1 });
@@ -53,22 +55,23 @@ describe('Removed() records only what something is tracking', () => {
world.despawn(viaDespawn);
world.remove(viaRemove, Sc);
- expect(removedSince(world, -1).sort()).toEqual([viaDespawn, viaRemove].sort());
+ expect(read().sort()).toEqual([viaDespawn, viaRemove].sort());
});
- it('tracking starts when something asks, not retroactively', () => {
+ it('history starts when a reader asks, not retroactively', () => {
const world = new World();
const before = world.spawn();
world.insert(before, Sc, { v: 1 });
world.despawn(before);
- // The query's own ctor is what enables tracking.
- expect(removedSince(world, -1)).toEqual([]);
+ // A reader does not inherit what happened before it existed.
+ const read = readerFor(world);
+ expect(read()).toEqual([]);
const after = world.spawn();
world.insert(after, Sc, { v: 2 });
world.despawn(after);
- expect(removedSince(world, -1)).toEqual([after]);
+ expect(read()).toEqual([after]);
});
it('losing a tracked component still moves the anyChangedSince watermark', () => {
diff --git a/sdk/tests/removed-retention.test.ts b/sdk/tests/removed-retention.test.ts
new file mode 100644
index 000000000..8bc1dc670
--- /dev/null
+++ b/sdk/tests/removed-retention.test.ts
@@ -0,0 +1,256 @@
+// SPDX-License-Identifier: Apache-2.0
+// SPDX-FileCopyrightText: Copyright (c) 2024-present ESEngine Team
+/**
+ * @file Who owns removal history, and how long it lives.
+ *
+ * `Removed(C)` promises the entities that lost C since THIS system last ran.
+ * That promise is only as good as the retention: a buffer pruned by anything
+ * other than what its readers still need drops rows nobody consented to lose,
+ * and the system sees an empty list rather than an error.
+ */
+import { describe, it, expect } from 'vitest';
+import { App } from '../src/app/app';
+import { defineComponent } from '../src/ecs/component';
+import { defineSystem, Schedule, SystemRunner } from '../src/ecs/system';
+import { World } from '../src/ecs/world';
+import { ResourceStorage } from '../src/ecs/resource';
+import { defineEvent, EventWriter } from '../src/ecs/event';
+import { Removed, Query, Changed } from '../src/ecs/query';
+import type { Entity } from '../src/types';
+
+const Bullet = defineComponent('RetBullet', { n: 0 });
+const Shell = defineComponent('RetShell', { n: 0 });
+
+describe('removal history retention', () => {
+ it('a system that runs every few frames still sees what it missed', async () => {
+ const app = App.new();
+ const world = app.world;
+
+ let armed = false;
+ const seen: Entity[][] = [];
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Removed(Bullet)],
+ (removed) => { seen.push(removed.toArray()); },
+ { name: 'RetReap' },
+ ), { runIf: () => armed });
+
+ // Run once so the reader exists and has a frame to be "since".
+ armed = true;
+ await app.tick(1 / 60);
+ armed = false;
+
+ const e = world.spawn();
+ world.insert(e, Bullet, { n: 1 });
+ await app.tick(1 / 60);
+ world.remove(e, Bullet);
+
+ // Long enough that a fixed-window prune would have dropped the row.
+ for (let i = 0; i < 6; i++) await app.tick(1 / 60);
+
+ armed = true;
+ await app.tick(1 / 60);
+
+ expect(seen.at(-1)).toEqual([e]);
+ });
+
+
+ /** Everything still in `C`'s buffer, whoever it belongs to. */
+ const held = (world: { getRemovedEntitiesSince(c: typeof Bullet, t: number): Entity[] }) =>
+ world.getRemovedEntitiesSince(Bullet, -1);
+
+ /** An app with a fast reader and a slow one over the same component. */
+ function twoReaders() {
+ const app = App.new();
+ let slowArmed = false;
+ const slowSaw: Entity[][] = [];
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Removed(Bullet)], () => { /* keeps up every frame */ }, { name: 'RetFast' },
+ ));
+ const slow = defineSystem(
+ [Removed(Bullet)],
+ (removed) => { slowSaw.push(removed.toArray()); },
+ { name: 'RetSlow' },
+ );
+ app.addSystemToSchedule(Schedule.Update, slow, { runIf: () => slowArmed });
+ return { app, slow, slowSaw, arm: (on: boolean) => { slowArmed = on; } };
+ }
+
+ it('holds history to the LOWEST claim, and lets go when it rises', async () => {
+ const { app, slowSaw, arm } = twoReaders();
+ const world = app.world;
+ arm(true);
+ await app.tick(1 / 60);
+ arm(false);
+
+ // A tick of its own: a removal recorded at the tick a reader last ran is
+ // outside its next window (`tick > lastRunTick`) and owed to nobody.
+ await app.tick(1 / 60);
+ const e = world.spawn();
+ world.insert(e, Bullet, { n: 1 });
+ world.remove(e, Bullet);
+ // The fast reader runs and releases; the slow one has not, so the row stays.
+ for (let i = 0; i < 4; i++) await app.tick(1 / 60);
+ expect(held(world)).toEqual([e]);
+
+ arm(true);
+ await app.tick(1 / 60);
+ expect(slowSaw.at(-1)).toEqual([e]);
+ // Both have now read past it, so nobody is owed it.
+ await app.tick(1 / 60);
+ expect(held(world)).toEqual([]);
+ });
+
+ it('a slow reader of one component does not pin another', async () => {
+ const app = App.new();
+ const world = app.world;
+ let armed = false;
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Removed(Bullet)], () => { /* every frame */ }, { name: 'RetA' },
+ ));
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Removed(Shell)], () => { /* only when armed */ }, { name: 'RetB' },
+ ), { runIf: () => armed });
+ armed = true;
+ await app.tick(1 / 60);
+ armed = false;
+ // One frame with only the fast reader running, so the two claims differ.
+ await app.tick(1 / 60);
+
+ const a = world.spawn();
+ world.insert(a, Bullet, { n: 1 });
+ world.remove(a, Bullet);
+ const b = world.spawn();
+ world.insert(b, Shell, { n: 1 });
+ world.remove(b, Shell);
+ for (let i = 0; i < 4; i++) await app.tick(1 / 60);
+
+ // A global watermark would hold both, because B's reader is behind.
+ expect(world.getRemovedEntitiesSince(Bullet, -1)).toEqual([]);
+ expect(world.getRemovedEntitiesSince(Shell, -1)).toEqual([b]);
+ });
+
+ it('evicting the slow system lets the watermark rise at once', async () => {
+ const { app, slow, arm } = twoReaders();
+ const world = app.world;
+ arm(true);
+ await app.tick(1 / 60);
+ arm(false);
+
+ await app.tick(1 / 60);
+ const e = world.spawn();
+ world.insert(e, Bullet, { n: 1 });
+ world.remove(e, Bullet);
+ for (let i = 0; i < 3; i++) await app.tick(1 / 60);
+ expect(held(world)).toEqual([e]);
+
+ app.removeSystem(slow._id);
+ expect(held(world)).toEqual([]);
+ expect(world.removedReaderCount(Bullet)).toBe(1);
+ });
+
+ it('an async system keeps its rows while it is parked', async () => {
+ const app = App.new();
+ const world = app.world;
+ let saw: Entity[] | null = null;
+ let armed = false;
+ let other: number | null = null;
+ let victim: Entity | null = null;
+
+ // Removed IN-FRAME: the window is `tick > lastRunTick`, so a removal
+ // between two frames carries the tick the reader just ran at and is
+ // outside the next window by construction.
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [], () => {
+ if (!armed || victim === null) return;
+ world.remove(victim, Bullet);
+ },
+ { name: 'RetKiller' },
+ ));
+
+ // A second claim, advanced WHILE the reader below is parked mid-body.
+ // Its own claim has not moved — it has not read yet — so the row is
+ // still owed and the prune must leave it.
+ const gate = () => new Promise<void>((resolve) => {
+ queueMicrotask(() => {
+ world.advanceRemovedReader(Bullet, other!, world.getWorldTick());
+ resolve();
+ });
+ });
+
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Removed(Bullet)],
+ async (removed) => {
+ if (!armed) return;
+ await gate();
+ saw = removed.toArray();
+ },
+ { name: 'RetAsync' },
+ ));
+ await app.tick(1 / 60);
+ other = world.registerRemovedReader(Bullet);
+
+ victim = world.spawn();
+ world.insert(victim, Bullet, { n: 1 });
+ armed = true;
+ await app.tick(1 / 60);
+
+ expect(saw).toEqual([victim]);
+ });
+
+ it('a reset gives up every claim it was holding', async () => {
+ const world = new World();
+ const runner = new SystemRunner(world, new ResourceStorage());
+ const sys = defineSystem([Removed(Bullet)], () => { /* reads */ }, { name: 'RetReset' });
+
+ runner.run(sys);
+ expect(world.removedReaderCount(Bullet)).toBe(1);
+
+ runner.reset();
+ expect(world.removedReaderCount(Bullet)).toBe(0);
+ // With nobody left to ask, the rows are not history — they are garbage.
+ const e = world.spawn();
+ world.insert(e, Bullet, { n: 1 });
+ world.remove(e, Bullet);
+ expect(world.getRemovedEntitiesSince(Bullet, -1)).toEqual([]);
+ });
+
+ it('a parameter set that fails to build leaves no claim behind', () => {
+ const world = new World();
+ // No event registry, so the SECOND parameter throws while the first has
+ // already been constructed. A claim taken during construction would have
+ // no owner to release it — and would pin this component forever.
+ const runner = new SystemRunner(world, new ResourceStorage());
+ const sys = defineSystem(
+ [Removed(Bullet), EventWriter(defineEvent<{ n: number }>('RetBoom'))],
+ () => { /* never runs */ },
+ { name: 'RetBadParams' },
+ );
+
+ expect(() => runner.run(sys)).toThrow('EventRegistry');
+ expect(world.removedReaderCount(Bullet)).toBe(0);
+ });
+
+ it('change tracking alone does not accumulate removal rows', async () => {
+ const app = App.new();
+ const world = app.world;
+
+ // A Changed() reader, and nothing that asks for removal history.
+ app.addSystemToSchedule(Schedule.Update, defineSystem(
+ [Query(Changed(Bullet))],
+ (q) => { for (const _row of q) { /* counted by the filter */ } },
+ { name: 'RetWatch' },
+ ));
+ await app.tick(1 / 60);
+
+ for (let i = 0; i < 200; i++) {
+ const e = world.spawn();
+ world.insert(e, Bullet, { n: i });
+ world.remove(e, Bullet);
+ }
+ await app.tick(1 / 60);
+
+ expect(world.getStorageSizes().changes.removedRows).toBe(0);
+ // Losing a component is still a change to it — that half must survive.
+ expect(world.anyChangedSince(Bullet, 0)).toBe(true);
+ });
+});
diff --git a/sdk/tests/system.test.ts b/sdk/tests/system.test.ts
index d89b00e60..c05481de0 100644
--- a/sdk/tests/system.test.ts
+++ b/sdk/tests/system.test.ts
@@ -62,6 +62,12 @@ function createMockWorld() {
isChangedSince: vi.fn(() => false),
getRemovedEntitiesSince: vi.fn(() => []),
enableChangeTracking: vi.fn(),
+ // Modelled with the runner's lifecycle in mind: it registers on commit,
+ // releases on completion and disposes on evict, and a stub that answers
+ // nothing would let a change to that order pass unnoticed here.
+ registerRemovedReader: vi.fn(() => 1),
+ advanceRemovedReader: vi.fn(),
+ disposeRemovedReader: vi.fn(),
resolveGetter: vi.fn(() => null),
};
diff --git a/sdk/tests/world.test.ts b/sdk/tests/world.test.ts
index 20531b7ed..9e54196f9 100644
--- a/sdk/tests/world.test.ts
+++ b/sdk/tests/world.test.ts
@@ -653,7 +653,9 @@ describe('World', () => {
it('getRemovedEntitiesSince should detect removed components', () => {
const w = new World();
- w.enableChangeTracking(Position);
+ // History is produced for a component someone READS history of —
+ // change tracking alone says the topology moved, not who lost what.
+ w.registerRemovedReader(Position);
const e = w.spawn();
w.insert(e, Position, { x: 0, y: 0 });
w.advanceTick();
@@ -667,41 +669,10 @@ describe('World', () => {
expect(w.getRemovedEntitiesSince(Velocity, -1)).toEqual([]);
});
- it('cleanRemovedBuffer should remove old entries', () => {
- const w = new World();
- w.enableChangeTracking(Position);
- const e1 = w.spawn();
- w.insert(e1, Position, { x: 0, y: 0 });
- w.remove(e1, Position);
-
- w.advanceTick();
- w.advanceTick();
-
- const e2 = w.spawn();
- w.insert(e2, Position, { x: 0, y: 0 });
- w.remove(e2, Position);
-
- w.cleanRemovedBuffer(1);
- const removed = w.getRemovedEntitiesSince(Position, -1);
- expect(removed).toContain(e2);
- expect(removed).not.toContain(e1);
- });
-
- it('cleanRemovedBuffer should delete empty buffers', () => {
- const w = new World();
- w.enableChangeTracking(Position);
- const e = w.spawn();
- w.insert(e, Position, { x: 0, y: 0 });
- w.remove(e, Position);
-
- w.cleanRemovedBuffer(Infinity);
- expect(w.getRemovedEntitiesSince(Position, -1)).toEqual([]);
- });
-
it('despawn should record removed ticks for all components', () => {
const w = new World();
- w.enableChangeTracking(Position);
- w.enableChangeTracking(Velocity);
+ w.registerRemovedReader(Position);
+ w.registerRemovedReader(Velocity);
const e = w.spawn();
w.insert(e, Position, { x: 0, y: 0 });
w.insert(e, Velocity, { dx: 1, dy: 1 });