Skip to content

Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.

feat(video): a restored cell caps the one-step key excess its resample made, keeping the source's coverage

Soohong Kim814cdfaef623
diff --git a/docs/loop-comparison.md b/docs/loop-comparison.md
index aae3862..b867c5c 100644
--- a/docs/loop-comparison.md
+++ b/docs/loop-comparison.md
@@ -117,46 +117,76 @@ quality bounds, and [video pipeline](video-pipeline.md) for processing stages.
`video-loop-repair` proposes a separate output for one damaged interior cell.
It preserves the cut, metadata, normal cells, boundary cells, and actual GIF/WebP
schedule. No interpolation is called. This operation has its own contract:
-repair report `schema_version: 2`, comparison `schema_version: 3`,
-`metric_version: source-restoration-v2`, `policy_version:
-key-protected-source-copy-v1`, `scope: processing-defect-restoration`,
-`operation: restore_active_cut`. The v1 comparison above is unchanged.
-`source-restoration-v1`, an unreleased draft that copied the whole source cell,
-is not this contract: its evidence and receipts are refused, never read as v2.
+repair report `schema_version: 3`, comparison `schema_version: 4`,
+`metric_version: source-restoration-v3`, `policy_version:
+key-protected-source-copy-one-step-cap-v1`, `scope:
+processing-defect-restoration`, `operation: restore_active_cut`. The v1
+comparison above is unchanged. Two unreleased drafts are not this contract:
+`source-restoration-v1` copied the whole source cell and `source-restoration-v2`
+never capped a pixel. Their evidence and receipts are refused, never read as v3.
### What a restored cell is
In the target cell, a pixel that differs from the source at the same place and
the same source time becomes the source's pixel, **unless the source's pixel
-would raise the key-colour guard there**. At those places the delivered pixel
-stays. Every pixel of the output is the whole RGBA of one of the two at that
-place: nothing is blended, moved, borrowed from another frame or recoloured,
-and a colour hidden under zero coverage never gains coverage.
+would raise the key-colour guard there**. Such a place is one of two kinds.
+
+- **A made excess of one step.** A cell is a resample of its source frame, and
+ mixing two colours that are both under the bar can give a colour over it
+ ([video pipeline](video-pipeline.md), "Cells"). Where the source's cell is
+ exactly one step over the bar and no visible source pixel its colour was mixed
+ from is over it, the source's pixel is written with the keyed channels one
+ step lower (green: `G - 1`; magenta: `R - 1` and `B - 1`). Its coverage and
+ its other channel are the source's.
+- **Anything else** keeps the delivered pixel: an excess a source pixel under
+ that place already carries (the subject's own key-coloured material, or spill
+ the keying left), and a made excess deeper than one step.
+
+Every other pixel of the output is the whole RGBA of the origin's or the
+source's at that place: nothing is blended, moved or borrowed from another
+frame, and a colour hidden under zero coverage never gains coverage.
```text
K(p) = alpha(p) x max(0, key hue excess(p) - 8) whole numbers
+window(p) = the source pixels the resample mixed the cell's colour at p from
differing = places where the origin's cell and the source's differ in RGBA
-protected = differing places where K(source) > K(origin)
-copied = differing places that are not protected
-output(p) = source(p) where copied, origin(p) everywhere else
+raised = differing places where K(source) > K(origin)
+capped = raised places where key hue excess(source) = 9
+ and K(q) = 0 for every q in window(p)
+protected = raised places that are not capped
+copied = differing places that are not raised
+output(p) = source(p) where copied,
+ source(p) with the keyed channels one lower where capped,
+ origin(p) everywhere else
```
`K` is 255 times the alpha-weighted excess of `axes.key_colour`, with the same
full-spill bar and the same hue test (green: `G - max(R, B)`; magenta:
-`min(R, B) - G`), so yellow, cyan, red and blue are not the key's hue. The mask
-is made by the engine from the actual files and has no manual input: no
-dilation, blur, key cleanup or tuning. When nothing is protected the output is
-the whole source cell.
-
-**The output's coverage must be the source's at every pixel.** Where a protected
-place has another alpha in the origin than in the source, keeping the delivered
-pixel would keep a silhouette the source does not have, so that proposal ends
-with `status: unknown`, `common_failure: false`, reason
-`protected-pixel-alpha-conflict` and empty `outputs`. Nothing is written. It
-used that proposal's attempt; the next independent proposal may be tried. A
-target whose every differing place is protected ends the same way with
-`no-unprotected-source-pixel`. No protected pixel is given up to make a
-candidate pass, however few they are.
+`min(R, B) - G`), so yellow, cyan, red and blue are not the key's hue. The
+window is the resampler's own (`sprite_gen/util/resample.py`, `mix_windows`),
+read on the cleaned, ramped and cropped source frame the cell was resampled
+from. A cell at its frame's size is a copy, so nothing in it is a made excess.
+The masks are made by the engine from the actual files and have no manual
+input: no dilation, blur, key cleanup or tuning. When nothing is raised the
+output is the whole source cell.
+
+One step is the least a whole-number channel can change. It bounds what the cap
+does to a pixel; it does not say that a deeper cap would look right, and none is
+applied. A bright seam, such as yellow meeting cyan under a green key or red
+meeting blue under a magenta one, mixes far over the bar and is a real colour
+of the picture: capping it would draw a darker line than either side. The
+general resample is not changed and caps nothing; only this operation, which
+knows the key and both pictures, does.
+
+**The output's coverage must be the source's at every pixel.** A capped pixel
+has the source's alpha. Where a protected place has another alpha in the origin
+than in the source, keeping the delivered pixel would keep a silhouette the
+source does not have, so that proposal ends with `status: unknown`,
+`common_failure: false`, reason `protected-pixel-alpha-conflict` and empty
+`outputs`. Nothing is written. It used that proposal's attempt; the next
+independent proposal may be tried. A target with no copied and no capped place
+ends the same way with `no-unprotected-source-pixel`. No protected pixel is
+given up to make a candidate pass, however few they are.
### The origin
@@ -215,7 +245,7 @@ source-relative damage. `--proposal-index` is one-based, limited to 1–3. A cel
already restored, a strip or playback boundary cell and a cell that is not
played are never proposed. The stable `proposal_id` binds the baseline's five
digests, the origin's five digests, the source inputs and sequence, the
-projection, the policy version, the target and the digests of its two masks.
+projection, the policy version, the target and the digests of its three masks.
Repeating the same index is the same proposal, not a retry strategy. The
application owns its deadline, attempted IDs and adoption transaction.
@@ -229,9 +259,10 @@ application owns its deadline, attempted IDs and adoption transaction.
| `applied_cells` | Cells earlier requests restored, in order, as rebuilt from the origin |
| `origin_artifacts`, `baseline_artifacts`, `candidate_artifacts` | Exact five-file byte receipts (`sha256`, `bytes` per file) |
| `source_artifacts`, `projection` | Source inputs, fixed transform, sample PTS and exact correspondence |
-| `partial.copied`, `partial.protected` | The two masks of the target: `size`, `count`, row-major run lengths `rle` (the first run is clear) and `sha256` of the packed bits |
+| `partial.copied`, `partial.capped`, `partial.protected` | The three masks of the target: `size`, `count`, row-major run lengths `rle` (the first run is clear) and `sha256` of the packed bits |
| `partial.alpha_equals_source` | Whether the output's coverage is the source's; `false` only with `protected-pixel-alpha-conflict` |
-| `partial.protected_pixels` | `count`, `alpha_conflicts`, and for the first 256 in row order (`listed`) `x`, `y`, `origin_rgba`, `source_rgba`, `origin_key_weight`, `source_key_weight`, `alpha_equal` |
+| `partial.capped_pixels` | `count`, and for the first 256 in row order (`listed`) `x`, `y`, `origin_rgba`, `source_rgba`, `output_rgba`, and `source_window` (`[left, top, right, bottom]`, inclusive, in the cropped source frame) |
+| `partial.protected_pixels` | `count`, `alpha_conflicts`, and for the first 256 in row order (`listed`) `x`, `y`, `origin_rgba`, `source_rgba`, `origin_key_weight`, `source_key_weight`, `source_key_excess`, `source_window_key_weight` (the most `K` in its window), `alpha_equal`. A place is protected, not capped, when `source_key_excess` is not 9 or `source_window_key_weight` is not 0 |
| `comparison` | Initial measurement of written files; compare again after finalization |
| `interpolation_calls` | Always zero in this operation |
@@ -268,7 +299,7 @@ comparison reports, not in the loop report, which cannot hold its own digest.
| `projection` | Recipe, crop, cell, scale, wrap, samples, PTS, reference pixel hashes |
| `applied` | One entry per restored cell, in the order they were adopted; a cell appears once |
| `applied[].target`, `source_index` | The cell and its source frame |
-| `applied[].copied`, `protected` | The two masks, as in the repair report |
+| `applied[].copied`, `capped`, `protected` | The three masks, as in the repair report |
| `applied[].origin_rgba_sha256`, `source_rgba_sha256`, `output_rgba_sha256` | The cell in the origin, in the source projection and as restored |
| `applied[].alpha_equals_source` | Always `true` for an applied cell |
| `applied[].proposal_id`, `baseline_artifacts`, `baseline_pixels_sha256` | The proposal, the five files it was proposed on and their decoded cells |
@@ -279,11 +310,12 @@ the baseline, proves the projection on **the origin's** normal cells, then
makes each applied cell again from the origin and the source in the recorded
order. Each must have been a proposal of the loop as it stood before it, must
give the recorded masks and digests, and must pass the protections on that
-loop. The cells read from the baseline must then equal the rebuilt ones: an
-applied cell its partial copy, every other cell the origin's. The rebuilt
-pixels are an expectation to compare with; they never replace the baseline's
-files. A restored cell therefore stays verified although it is no longer the
-source's whole cell.
+loop. Which places are capped is decided again from the source frames and the
+resampler's windows, never read from the receipt. The cells read from the
+baseline must then equal the rebuilt ones: an applied cell its partial copy,
+every other cell the origin's. The rebuilt pixels are an expectation to compare
+with; they never replace the baseline's files. A restored cell therefore stays
+verified although it is no longer the source's whole cell.
These are errors, with nonzero exit: an origin that already carries a receipt;
a baseline that differs from the origin without a receipt; a receipt naming
@@ -337,8 +369,9 @@ The report carries `origin`, `baseline` and `candidate` (each with its five
`improved` requires that the one changed cell is byte-exact the partial copy
made from the origin and the source, its coverage the source's, the candidate's
receipt the recomputed one, at least one cleared actual fault, unchanged normal
-cells and metadata bytes, and no worsening protected axis. Changed pixels
-outside the copied mask return `regressed` with
+cells and metadata bytes, and no worsening protected axis. A changed cell that
+is not that copy (a pixel outside the masks, a capped place left uncapped or
+capped further) returns `regressed` with
`changed-cell-is-not-the-verified-source-copy`; a receipt that differs returns
`unknown` with `candidate-report-differs-from-verified-receipt`; changed
metadata bytes `candidate-metadata-bytes-changed`; equal pixels with any other
@@ -352,20 +385,25 @@ Boundary pixels and actual playback intervals stay identical. GIF quantization
and WebP decoding are checked against the strip export mapping, with their
actual integer delays recorded separately from fractional strip timing.
-Key-colour protection is read again from the final strip, whatever the mask
+Key-colour protection is read again from the final strip, whatever the masks
promised. `axes.key_colour.introduced_excess` sums only the positive per-pixel
increases of the alpha-weighted excess on the fixed canvas, per cell, and
`introduced_pixels` counts the pixels that increased. The sum is not a pixel
count. A reduction elsewhere cannot offset newly introduced spill, even when
-the cell's total tint decreases.
+the cell's total tint decreases. The bar, the per-pixel rule and the coverage
+rule are the same for a capped cell: the cap is what lets the source's pixel
+meet them, not an exception to them. A normal cell that already carries a made
+excess is not a target and is never rewritten; `axes.key_colour.baseline`
+reports it as delivered.
For an outline-specific UI success message, require `verdict == improved` and
an `outline` entry in `cleared_faults[].faults`. `axes` contains raw baseline,
reference and candidate values for outline/smear, source-relative partial
coverage and key colour. `gait.source_order == preserved` proves source motion
-order only; `gait.absolute == unverified` remains explicit. A restored cell is
-the source's pixel or the delivered one at each place; these measurements do
-not claim the source's whole drawing, nor to repair mistakes already in it.
+order only; `gait.absolute == unverified` remains explicit. A restored cell is,
+at each place, the source's pixel, the source's pixel capped one step, or the
+delivered one; these measurements do not claim the source's whole drawing, nor
+to repair mistakes already in it.
A candidate-specific unknown or regression may be followed by the next unused
proposal within budget. Shared unknown stops the request. Upload and activate
diff --git a/docs/video-pipeline.md b/docs/video-pipeline.md
index 9a330a7..20118a4 100644
--- a/docs/video-pipeline.md
+++ b/docs/video-pipeline.md
@@ -1014,7 +1014,14 @@ Outputs:
at its own size is copied as it is). Coverage keeps LANCZOS's edge, held between the least and
the most coverage of the source pixels the colour mixes from, so nothing spills outside the
silhouette and nothing opens inside it. Colour is a Hamming mix of premultiplied colour, a
- filter with no negative lobe, so every pixel's colour is a mix of the colours under it. The
+ filter with no negative lobe, so every pixel's colour is a mix of the colours under it. Each
+ channel therefore stays within the colours under it, but the key hue's excess (green:
+ `G - max(R, B)`) does not: a colour led by red mixed with one led by blue is led by neither as
+ far, so a cell can carry a little more key hue than any source pixel under it. Every filter
+ that mixes colours does this, in a cell and again wherever the cell is scaled later; the
+ resample knows no key and caps nothing (`tests/video/test_strip_resample.py`). The one
+ operation that compares a cell with the key bar, source restoration, handles it itself
+ ([loop-comparison.md](loop-comparison.md#what-a-restored-cell-is)). The
strip used LANCZOS over premultiplied RGBA before, which weighs the colours across an edge
against each other and divides by the edge's low coverage: on a keyed frame, whose edge holds
a light rim and ink with a little of the key's green, it drew a lighter rim and greener ink
diff --git a/sprite_gen/util/resample.py b/sprite_gen/util/resample.py
index 96bab5d..bf99830 100644
--- a/sprite_gen/util/resample.py
+++ b/sprite_gen/util/resample.py
@@ -15,7 +15,11 @@ from PIL import Image
from sprite_gen._deps import np
-def _support_bounds(n_src: int, n_out: int, support: float) -> tuple[np.ndarray, np.ndarray]:
+# How far `resize_cell`'s colour filter (Hamming) reaches, in source pixels at 1:1.
+MIX_SUPPORT = 1.0
+
+
+def support_bounds(n_src: int, n_out: int, support: float) -> tuple[np.ndarray, np.ndarray]:
"""First and last source index Pillow's resample reads for each output index, for a filter
of `support` source pixels (`precompute_coeffs` in Pillow's Resample.c: the support widens
by the scale when shrinking)."""
@@ -27,7 +31,15 @@ def _support_bounds(n_src: int, n_out: int, support: float) -> tuple[np.ndarray,
return lo, np.maximum(lo, hi)
-def _window_extrema(a: np.ndarray, rows: tuple[np.ndarray, np.ndarray], cols: tuple[np.ndarray, np.ndarray]) -> tuple[np.ndarray, np.ndarray]:
+def mix_windows(src_size: tuple[int, int], size: tuple[int, int]) -> tuple[tuple[np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray]]:
+ """The source pixels `resize_cell` mixes each output pixel's colour from: the first and last
+ source row for each output row, then the same for the columns. `resize_cell` holds its
+ coverage to these windows; a caller that has to know what a cell's pixel was made of reads
+ the same ones (a cell at its own size is a copy: its window still holds its own pixel)."""
+ return support_bounds(src_size[1], size[1], MIX_SUPPORT), support_bounds(src_size[0], size[0], MIX_SUPPORT)
+
+
+def window_extrema(a: np.ndarray, rows: tuple[np.ndarray, np.ndarray], cols: tuple[np.ndarray, np.ndarray]) -> tuple[np.ndarray, np.ndarray]:
"""Min and max of `a` over each output pixel's source window (separable, rows then columns)."""
def along(v: np.ndarray, lo: np.ndarray, hi: np.ndarray, axis: int) -> tuple[np.ndarray, np.ndarray]:
low = high = np.take(v, lo, axis=axis)
@@ -50,7 +62,11 @@ def resize_cell(image: Image.Image, size: tuple[int, int]) -> Image.Image:
LANCZOS's crisp edge, held to the range of the source coverage the colour mixes from (no
halo outside the silhouette, no hole inside it). Colour is a Hamming mix of premultiplied
colour, a filter with no negative lobe: every pixel's colour is a weighted mix of the
- colours under it and never one they did not have. docs/video-pipeline.md "Cells"."""
+ colours under it and never one they did not have. Each channel therefore stays within the
+ colours under it. The key hue's excess (G - max(R, B) for green) does not: a colour led by
+ red mixed with one led by blue is led by neither as far, so the mix can carry more excess
+ than both. The cell is left as mixed; nothing here knows the key. docs/video-pipeline.md
+ "Cells"."""
if image.size == tuple(size):
return image.copy()
w, h = size
@@ -62,7 +78,7 @@ def resize_cell(image: Image.Image, size: tuple[int, int]) -> Image.Image:
mix_alpha = scaled(alpha, Image.Resampling.HAMMING)
mix = np.stack([scaled(src[..., c] * alpha, Image.Resampling.HAMMING) for c in range(3)], axis=-1)
- low, high = _window_extrema(alpha, _support_bounds(image.height, h, 1.0), _support_bounds(image.width, w, 1.0))
+ low, high = window_extrema(alpha, *mix_windows(image.size, size))
cover = np.clip(scaled(alpha, Image.Resampling.LANCZOS), low, high)
cover = np.where(mix_alpha > 1e-3, np.round(cover), 0)
colour = np.where((cover > 0)[..., None], np.round(mix / np.maximum(mix_alpha, 1e-3)[..., None]), 0)
diff --git a/sprite_gen/video/restoration.py b/sprite_gen/video/restoration.py
index 640007d..cf9e480 100644
--- a/sprite_gen/video/restoration.py
+++ b/sprite_gen/video/restoration.py
@@ -2,10 +2,12 @@
"""Restore one damaged delivered cell from its verified source time, without RIFE.
A source pixel is copied only where it does not raise the key guard over the
-origin's pixel; the origin's pixel stays everywhere else, and the cell's whole
-coverage must still be the source's. The origin is the five files the first
-restoration started from: every request rebuilds the current baseline from it
-and the source, so a receipt is recomputed, never trusted.
+origin's pixel. Where it would, and the excess is one step that the resample
+made out of source pixels that have none, the source pixel is written with its
+key hue capped to the bar; anywhere else the origin's pixel stays. The cell's
+whole coverage must still be the source's. The origin is the five files the
+first restoration started from: every request rebuilds the current baseline
+from it and the source, so a receipt is recomputed, never trusted.
Source correspondence, proposal generation and final acceptance are separate.
A failed independent proposal never hides the next one; a missing shared source
@@ -25,26 +27,31 @@ from PIL import Image
from sprite_gen._deps import np
from sprite_gen.frames.cutout import KEY_TARGETS
+from sprite_gen.frames.decontam import _cap_key_hue
from sprite_gen.frames.extract import _key_channel_split, _key_excess_field, _SPILL_FULL_MIN_TINT
from sprite_gen.spec.runio import atomic_write_text
+from sprite_gen.util import resample
from sprite_gen.video import evidence, playback, rife, source
from sprite_gen.video.compare import KIND as COMPARISON_KIND, Loop
from sprite_gen.video.interpolation_quality import SMEAR_WARN, OUTLINE_WARN, faults
-METRIC = "source-restoration-v2"
-POLICY = "key-protected-source-copy-v1"
+METRIC = "source-restoration-v3"
+POLICY = "key-protected-source-copy-one-step-cap-v1"
SCOPE = "processing-defect-restoration"
OPERATION = "restore_active_cut"
KIND = "sprite-gen-video-loop-restoration"
-REPAIR_SCHEMA = 2
-COMPARISON_SCHEMA = 3
+REPAIR_SCHEMA = 3
+COMPARISON_SCHEMA = 4
MAX_PROPOSALS = 3
ARTIFACTS = ("strip", "meta", "report", "gif", "webp")
-PROTECTED_LISTED = 256 # protected pixels a report spells out; the mask always holds all of them
+LISTED = 256 # capped or protected pixels a report spells out; the masks always hold all of them
# The guard in whole numbers is the same formula only while the bar is a whole number.
_BAR = int(_SPILL_FULL_MIN_TINT)
assert _BAR == _SPILL_FULL_MIN_TINT
+# The only excess that is capped: one step over the bar, the least a whole-number channel can
+# change. It bounds the change; it is not a promise about how any deeper cap would look.
+CAP_DEPTH = 1
def read_loop(paths: dict[str, Path]) -> tuple[Loop, dict[str, playback.Playback]]:
@@ -58,15 +65,18 @@ def measure(frames: list[Image.Image], neighbours: list[Image.Image]) -> list[di
return [rife.smear(f, neighbours[k - 1], neighbours[(k + 1) % len(frames)]) for k, f in enumerate(frames)]
+def _key_hue(rgba: Any, key: str) -> Any:
+ channels, others = _key_channel_split(KEY_TARGETS[key])
+ return _key_excess_field(rgba[..., :3].astype(np.int64), channels, others)
+
+
def key_weight(rgba: Any, key: str) -> Any:
"""Alpha times the key hue's excess over the full-spill bar, per pixel, in whole numbers.
255 times the alpha-weighted excess the comparison reports. Includes newly
restored coverage: returning to a source pixel does not excuse its key spill.
"""
- channels, others = _key_channel_split(KEY_TARGETS[key])
- hue = _key_excess_field(rgba[..., :3].astype(np.int64), channels, others)
- return rgba[..., 3].astype(np.int64) * np.maximum(0, hue - _BAR)
+ return rgba[..., 3].astype(np.int64) * np.maximum(0, _key_hue(rgba, key) - _BAR)
def _source_key(src: source.Source) -> str | None:
@@ -76,26 +86,59 @@ def _source_key(src: source.Source) -> str | None:
return key if key in ("green", "magenta") else None
+def source_key_weight(cropped: Image.Image, size: tuple[int, int], key: str) -> Any:
+ """The most key weight among the source pixels each cell pixel's colour was mixed from.
+
+ Zero where every visible source pixel under it is at or under the bar: an
+ excess there was made by mixing them, not carried by any of them. The windows
+ are the resample's own (`resample.mix_windows`).
+ """
+ weight = key_weight(np.asarray(cropped, dtype=np.uint8), key)
+ return resample.window_extrema(weight, *resample.mix_windows(cropped.size, size))[1]
+
+
+def made_excess(reference: Image.Image, cropped: Image.Image, key: str) -> Any:
+ """Where the source's cell is exactly `CAP_DEPTH` over the bar and no source pixel under it is over."""
+ s = np.asarray(reference, dtype=np.uint8)
+ return ((s[..., 3] > 0) & (_key_hue(s, key) == _BAR + CAP_DEPTH)
+ & (source_key_weight(cropped, reference.size, key) == 0))
+
+
+def _capped(rgba: Any, key: str) -> Any:
+ """The engine's key-hue cap to the bar on whole pixels: the keyed channels come down, nothing else moves."""
+ channels, others = _key_channel_split(KEY_TARGETS[key])
+ out = rgba.copy()
+ out[..., :3] = _cap_key_hue(rgba[..., :3], channels, others, _BAR).astype(np.uint8)
+ return out
+
+
@dataclass
class Partial:
- frame: Image.Image # the source where it is copied, the origin where it is protected
+ frame: Image.Image # per place: the source's pixel, the source's with its key hue capped, or the origin's
copied: Any
+ capped: Any
protected: Any
alpha_conflict: bool # a protected pixel whose coverage is not the source's
-def partial(origin: Image.Image, reference: Image.Image, key: str) -> Partial:
+def partial(origin: Image.Image, reference: Image.Image, cropped: Image.Image, key: str) -> Partial:
"""Where the two differ, the source's pixel unless it would raise the key guard there.
- Every pixel is the origin's or the source's whole RGBA at the same place: no
- colour is mixed, moved or made, and a hidden colour never gains coverage.
+ A place where it would is capped only when its excess is a made one of one
+ step (`made_excess`): the source's pixel with the keyed channels one lower,
+ its coverage unchanged. Any other such place keeps the origin's whole RGBA.
+ No colour is moved or taken from another place, and a hidden colour never
+ gains coverage.
"""
b, s = (np.asarray(f, dtype=np.uint8) for f in (origin, reference))
differs = (b != s).any(axis=-1)
- protected = differs & (key_weight(s, key) > key_weight(b, key))
- copied = differs & ~protected
- frame = Image.fromarray(np.where(copied[..., None], s, b))
- return Partial(frame, copied, protected, bool((b[..., 3] != s[..., 3])[protected].any()))
+ raised = differs & (key_weight(s, key) > key_weight(b, key))
+ capped = raised & made_excess(reference, cropped, key)
+ protected = raised & ~capped
+ copied = differs & ~raised
+ out = np.where(copied[..., None], s, b)
+ out[capped] = _capped(s[capped], key)
+ return Partial(Image.fromarray(out), copied, capped, protected, bool((b[..., 3] != s[..., 3])[protected].any()))
def _mask(mask: Any) -> dict[str, Any]:
@@ -108,21 +151,28 @@ def _mask(mask: Any) -> dict[str, Any]:
"sha256": evidence.digest(f"{w}x{h}:".encode() + np.packbits(flat).tobytes())}
-def _protected_pixels(part: Partial, origin: Image.Image, reference: Image.Image, key: str) -> dict[str, Any]:
- b, s = (np.asarray(f, dtype=np.uint8) for f in (origin, reference))
- kb, ks = key_weight(b, key), key_weight(s, key)
- places = np.argwhere(part.protected)
- return {"count": len(places), "alpha_conflicts": int((b[..., 3] != s[..., 3])[part.protected].sum()),
- "listed": min(len(places), PROTECTED_LISTED),
- "pixels": [{"x": int(x), "y": int(y), "origin_rgba": b[y, x].tolist(), "source_rgba": s[y, x].tolist(),
- "origin_key_weight": int(kb[y, x]), "source_key_weight": int(ks[y, x]),
- "alpha_equal": bool(b[y, x, 3] == s[y, x, 3])} for y, x in places[:PROTECTED_LISTED]]}
-
-
-def _describe(part: Partial, origin: Image.Image, reference: Image.Image, key: str) -> dict[str, Any]:
- return {"copied": _mask(part.copied), "protected": _mask(part.protected),
+def _describe(part: Partial, origin: Image.Image, reference: Image.Image, cropped: Image.Image,
+ key: str) -> dict[str, Any]:
+ """The three masks, and the capped and the protected pixels spelled out with what decided each."""
+ b, s, f = (np.asarray(im, dtype=np.uint8) for im in (origin, reference, part.frame))
+ kb, ks, hue = key_weight(b, key), key_weight(s, key), _key_hue(s, key)
+ under = source_key_weight(cropped, reference.size, key)
+ (top, bottom), (left, right) = resample.mix_windows(cropped.size, reference.size)
+ capped, protected = np.argwhere(part.capped), np.argwhere(part.protected)
+ return {"copied": _mask(part.copied), "capped": _mask(part.capped), "protected": _mask(part.protected),
"alpha_equals_source": not part.alpha_conflict,
- "protected_pixels": _protected_pixels(part, origin, reference, key)}
+ "capped_pixels": {"count": len(capped), "listed": min(len(capped), LISTED), "pixels": [
+ {"x": int(x), "y": int(y), "origin_rgba": b[y, x].tolist(), "source_rgba": s[y, x].tolist(),
+ "output_rgba": f[y, x].tolist(),
+ "source_window": [int(left[x]), int(top[y]), int(right[x]), int(bottom[y])]}
+ for y, x in capped[:LISTED]]},
+ "protected_pixels": {"count": len(protected),
+ "alpha_conflicts": int((b[..., 3] != s[..., 3])[part.protected].sum()),
+ "listed": min(len(protected), LISTED), "pixels": [
+ {"x": int(x), "y": int(y), "origin_rgba": b[y, x].tolist(), "source_rgba": s[y, x].tolist(),
+ "origin_key_weight": int(kb[y, x]), "source_key_weight": int(ks[y, x]),
+ "source_key_excess": int(hue[y, x]), "source_window_key_weight": int(under[y, x]),
+ "alpha_equal": bool(b[y, x, 3] == s[y, x, 3])} for y, x in protected[:LISTED]]}}
def proposals(frames: list[Image.Image], projection: source.Projection, shown: list[int],
@@ -149,7 +199,8 @@ def proposal_id(current: dict[str, Any], origin: dict[str, Any], src: source.Sou
"source": {"artifacts": src.artifacts, "sequence": src.record["source"]},
"projection": _portable(projection.record), "operation": OPERATION, "metric": METRIC,
"policy": POLICY, "target": target,
- "mask": {k: _mask(m)["sha256"] for k, m in (("copied", part.copied), ("protected", part.protected))}}
+ "mask": {k: _mask(m)["sha256"] for k, m in (("copied", part.copied), ("capped", part.capped),
+ ("protected", part.protected))}}
return evidence.digest(json.dumps(identity, sort_keys=True, separators=(",", ":")).encode())
@@ -160,7 +211,7 @@ def _entry(target: int, part: Partial, origin: Loop, current: dict[str, Any], be
"proposal_id": proposal_id(current, origin.artifacts, src, projection, target, part),
"baseline_artifacts": current,
"baseline_pixels_sha256": evidence.digest(b"".join(f.tobytes() for f in before)),
- "copied": _mask(part.copied), "protected": _mask(part.protected),
+ "copied": _mask(part.copied), "capped": _mask(part.capped), "protected": _mask(part.protected),
"origin_rgba_sha256": evidence.digest(origin.frames[target].tobytes()),
"source_rgba_sha256": evidence.digest(projection.frames[target].tobytes()),
"output_rgba_sha256": evidence.digest(part.frame.tobytes()),
@@ -272,7 +323,7 @@ def chain(origin: Loop, current: Loop, src: source.Source) -> tuple[Chain | None
previous = entry.get("baseline_artifacts")
if not isinstance(previous, dict) or (step == 0 and previous != origin.artifacts):
raise ValueError("an applied restoration names another baseline than its prefix")
- part = partial(origin.frames[target], projection.frames[target], key)
+ part = partial(origin.frames[target], projection.frames[target], projection.cropped[target], key)
after = [part.frame if k == target else f for k, f in enumerate(frames)]
if (part.alpha_conflict or _portable(entry) != _portable(_entry(target, part, origin, previous, frames, src, projection))
or judge(frames, after, projection.frames, key)["verdict"] != "improved"):
@@ -294,7 +345,9 @@ def _base(origin: Loop, baseline: Loop, candidate: Loop, src: source.Source) ->
"source": {"artifacts": src.artifacts, "sequence": src.record["source"]},
"axes": {}, "gait": {"absolute": "unverified", "source_order": "unverified"},
"limits": ["same-time processing damage only; no absolute gait or drawing-quality certification",
- "a restored cell is the source's pixel or the origin's at each place, never the whole source drawing",
+ "a restored cell is, at each place, the source's pixel, the source's pixel with its key hue "
+ "capped one step to the bar, or the origin's; never the whole source drawing",
+ "one step is the least a channel can change: a bound on the cap, not a verdict on how it looks",
"source and origin receipts bind supplied bytes; they are not signatures of the original producer"]}
@@ -345,12 +398,12 @@ def compare(origin: Loop, baseline: Loop, candidate: Loop, src: source.Source, *
if len(changed) != 1 or changed[0] not in state.targets(baseline.frames):
return finish("unknown", ["changed-cells-outside-verified-proposal"])
target = changed[0]
- part = partial(origin.frames[target], ref[target], state.key)
+ part = partial(origin.frames[target], ref[target], projection.cropped[target], state.key)
entry = _entry(target, part, origin, baseline.artifacts, baseline.frames, src, projection)
if repair_evidence.get("proposal_id") != entry["proposal_id"] or repair_evidence.get("target") != target:
raise ValueError("repair proposal identity differs from actual changed cells")
- result["restoration"] = {"target": target, "proposal_id": entry["proposal_id"],
- **_describe(part, origin.frames[target], ref[target], state.key)}
+ result["restoration"] = {"target": target, "proposal_id": entry["proposal_id"], **_describe(
+ part, origin.frames[target], ref[target], projection.cropped[target], state.key)}
if part.alpha_conflict:
return finish("unknown", ["protected-pixel-alpha-conflict"])
if candidate.frames[target].tobytes() != part.frame.tobytes():
@@ -433,11 +486,11 @@ def restore(paths: dict[str, Path], origin_paths: dict[str, Path], src: source.S
result.update(target=target, proposal_id=None, common_failure=False, reasons=[], status="no_change")
frame = receipt = None
if target is not None:
- part = partial(origin.frames[target], projection.frames[target], state.key)
+ part = partial(origin.frames[target], projection.frames[target], projection.cropped[target], state.key)
entry = _entry(target, part, origin, baseline.artifacts, baseline.frames, src, projection)
- result.update(proposal_id=entry["proposal_id"], status="candidate",
- partial=_describe(part, origin.frames[target], projection.frames[target], state.key))
- if part.alpha_conflict or not part.copied.any():
+ result.update(proposal_id=entry["proposal_id"], status="candidate", partial=_describe(
+ part, origin.frames[target], projection.frames[target], projection.cropped[target], state.key))
+ if part.alpha_conflict or not (part.copied | part.capped).any():
return {**result, "status": "unknown", "reasons": [
"protected-pixel-alpha-conflict" if part.alpha_conflict else "no-unprotected-source-pixel"]}
frame, receipt = part.frame, _receipt(origin, src, projection, [*state.applied, entry])
diff --git a/sprite_gen/video/source.py b/sprite_gen/video/source.py
index 3d7953e..7e2b964 100644
--- a/sprite_gen/video/source.py
+++ b/sprite_gen/video/source.py
@@ -138,6 +138,7 @@ class Source:
class Projection:
frames: list[Image.Image]
record: dict[str, Any]
+ cropped: list[Image.Image] # what each cell was resampled from: the cleaned, ramped source frame cut to the crop
def project(baseline: Any, source: Source) -> tuple[Projection | None, str | None]:
@@ -244,7 +245,8 @@ def project(baseline: Any, source: Source) -> tuple[Projection | None, str | Non
if not hits:
return None, "legacy-source-correspondence-unverified"
crop = hits[0]
- projected = [resize_cell(frame.crop(crop), (w, h)) for frame in frames]
+ cropped = [frame.crop(crop) for frame in frames]
+ projected = [resize_cell(frame, (w, h)) for frame in cropped]
receipt = {"recipe": RECIPE, "crop": crop, "cell": [w, h], "wrap_dx_px": dx,
"source": source.record["source"], "inputs": source.artifacts,
"measurement_engine": evidence.engine_identity(),
@@ -252,7 +254,7 @@ def project(baseline: Any, source: Source) -> tuple[Projection | None, str | Non
"samples": list(range(start, start + n)), "pts_seconds": pts[start:start + n],
"unmodified_cells": keep, "repair_hints": hints, "exact_projections": 1,
"reference_pixels_sha256": [evidence.digest(f.tobytes()) for f in projected]}
- return Projection(projected, receipt), None
+ return Projection(projected, receipt, cropped), None
def add_arguments(parser: Any) -> None:
diff --git a/tests/video/test_source_restoration.py b/tests/video/test_source_restoration.py
index 21c993a..0f93e1f 100644
--- a/tests/video/test_source_restoration.py
+++ b/tests/video/test_source_restoration.py
@@ -8,16 +8,23 @@ import shutil
import subprocess
from pathlib import Path
+import numpy as np
import pytest
from PIL import Image, ImageDraw
from sprite_gen.video import evidence, source, restoration, playback, loop
from sprite_gen.cli import main
from sprite_gen.util.gif_utils import save_clean_gif
+from sprite_gen.util.resample import resize_cell
from sprite_gen.video.compare import Loop
W, H, N = 64, 80, 12
FILL = (210, 160, 110, 255)
+METRIC, POLICY = 'source-restoration-v3', 'key-protected-source-copy-one-step-cap-v1'
+# Two colours under the green key's bar (hue excess -6 each), one led by red and one by blue:
+# their even mix (105, 114, 105) is one step over it. ROSE and IRIS are the same for magenta.
+WARM, COOL = (120, 114, 90, 255), (90, 114, 120, 255)
+ROSE, IRIS = (124, 100, 94, 255), (94, 100, 124, 255)
def frames():
@@ -36,9 +43,9 @@ def frames():
return result
-def damage(im):
+def damage(im, zoom=1):
made = im.copy()
- for y in range(8, 58):
+ for y in range(8*zoom, 58*zoom):
for x in range(made.width):
if made.getpixel((x,y))[:3] in ((20,20,20),(30,30,30)):
made.putpixel((x,y),FILL)
@@ -67,27 +74,33 @@ def cells(paths):
return restoration.read_loop(paths)[0].frames
-def case(tmp_path):
+def build(tmp_path, zoom=1, key='green', paint=None):
+ """A loop with two damaged cells and its keyed source, `zoom` times the cell's size.
+
+ At zoom 1 a cell is a copy of its frame; at 2 every cell is a resample, as a delivered loop's are.
+ `paint` draws on chosen source frames before the loop is cut from them.
+ """
if not shutil.which('ffmpeg') or not loop.img2webp_supports_exact():
pytest.skip('ffmpeg and img2webp >=1.5 required')
- fs = frames()
+ fs = [f.resize((W*zoom,H*zoom),Image.Resampling.NEAREST) for f in frames()]
+ for k,draw in (paint or {}).items(): draw(fs[k])
src_dir=tmp_path/'keyed'
src_dir.mkdir()
for k,f in enumerate(fs): f.save(src_dir/f'frame-{k:04}.png')
canvas=src_dir/'frame-0000.png'
freport=tmp_path/'frames.json'
- freport.write_text(json.dumps({'kind':'sprite-gen-video-frames-report','fps':24.,'frames':12,'width':64,'height':80,'stream_index':0,'key':'green'}))
+ freport.write_text(json.dumps({'kind':'sprite-gen-video-frames-report','fps':24.,'frames':12,'width':W*zoom,'height':H*zoom,'stream_index':0,'key':key}))
manifest=tmp_path/'source.json'
inputs=dict(source_manifest=manifest,source_frames_dir=src_dir,source_canvas=canvas,source_frames_report=freport)
mp4=tmp_path/'clip.mp4'
subprocess.run(['ffmpeg','-v','error','-framerate','24','-i',str(src_dir/'frame-%04d.png'),'-c:v','libx264','-pix_fmt','yuv420p',str(mp4)],check=True)
inputs['source_clip']=mp4
manifest.write_text(json.dumps(source.manifest(clip=mp4,canvas=canvas,frames_report=freport,files=sorted(src_dir.glob('*.png')),timestamps=source.timestamps(mp4,0))))
- clean=[cleaned(f) for f in fs]
- bad=list(clean); bad[2]=damage(clean[2]); bad[8]=damage(clean[8])
+ clean=[resize_cell(cleaned(f),(W,H)) for f in fs]
+ bad=list(clean)
+ for k in (2,8): bad[k]=resize_cell(damage(cleaned(fs[k]),zoom),(W,H))
paths={k:tmp_path/('b'+ext) for k,ext in [('strip','.png'),('meta','.json'),('report','.loop.json'),('gif','.gif'),('webp','.webp')]}
- m={'frames':12,'w':64,'h':80,'delay_ms':41.67,'cycle_seconds':.5,'cycle_frames':12,'subsampled':False,'loop':True,'scale':1.,'source_rect':[0,0,64,80],'sample_indices':list(range(12)),'foot_anchor':'none'}
+ m={'frames':12,'w':64,'h':80,'delay_ms':41.67,'cycle_seconds':.5,'cycle_frames':12,'subsampled':False,'loop':True,'scale':1/zoom,'source_rect':[0,0,W*zoom,H*zoom],'sample_indices':list(range(12)),'foot_anchor':'none'}
src=source.Source.read(**inputs)
m['source_cut']={'source':src.record['source'],'start':0,'length':12,'samples':list(range(12))}
report={'kind':'sprite-gen-video-loop-report','status':'passed','fps':24.,'frames_total':12,'source':src.record['source'],'cycle':{'start':0,'length':12},'anchor':'none','jump_repair':{'replaced':[2,8]},'strip':m,'n_out':12,'delay_ms':42}
@@ -96,6 +109,23 @@ def case(tmp_path):
return paths, inputs, clean
+def case(tmp_path):
+ return build(tmp_path)
+
+
+def scaled(tmp_path,request):
+ return build(tmp_path,zoom=2,key=getattr(request,'param','green'))
+
+
+def seam_patch(left,right):
+ """On a twice-size keyed frame: two colours inside the body that meet within one cell column (27, rows 25-30)."""
+ def paint(f):
+ f.paste(left,(48,48,55,64)); f.paste(right,(55,48,64,64))
+ return paint
+
+
def repair(case,tmp_path,index=1,*,baseline=None,origin=None,out=None):
paths,inputs,_=case
return restoration.restore(baseline or paths,origin or paths,source.Source.read(**inputs),
@@ -130,16 +160,16 @@ def repaint_source(case, index, paint):
def test_source_restoration_preserves_normal_motion_and_timing(case,tmp_path):
result=repair(case,tmp_path)
- assert result['status']=='candidate' and result['schema_version']==2
+ assert result['status']=='candidate' and result['schema_version']==3
assert result['origin_artifacts']==result['baseline_artifacts']
comparison=compare_result(case,result)
assert comparison['verdict']=='improved', comparison['reasons']
- assert (comparison['metric_version'],comparison['schema_version'])==('source-restoration-v2',3)
+ assert (comparison['metric_version'],comparison['policy_version'],comparison['schema_version'])==(METRIC,POLICY,4)
target=result['target']
a,b=cells(case[0]),cells(outputs(result))
assert all(x.tobytes()==y.tobytes() for k,(x,y) in enumerate(zip(a,b)) if k!=target)
- # Nothing is protected here, so the copy is the whole source cell.
- assert result['partial']['protected']['count']==0
+ # Nothing is protected or capped here, so the copy is the whole source cell.
+ assert result['partial']['protected']['count']==result['partial']['capped']['count']==0
assert b[target].tobytes()==case[2][target].tobytes()
axis=comparison['axes']['introduced_partial']
assert axis['raw_candidate'][target]>axis['raw_baseline'][target]
@@ -261,7 +291,7 @@ def test_cli_real_json_needs_the_origin(case,tmp_path,capsys):
assert not (tmp_path/'guessed').exists() and not report.exists()
assert main(['video-loop-repair',*baseline,*origin,*src,'--out-dir',str(tmp_path/'cli'),'--report',str(report)])==0
r=json.loads(report.read_bytes())
- assert (r['schema_version'],r['metric_version'])==(2,'source-restoration-v2')
+ assert (r['schema_version'],r['metric_version'],r['policy_version'])==(3,METRIC,POLICY)
candidate=[a for k,p in r['outputs'].items() for a in ('--candidate-'+k,p)]
comparison=tmp_path/'comparison.json'
with pytest.raises(SystemExit,match='origin'):
@@ -269,7 +299,7 @@ def test_cli_real_json_needs_the_origin(case,tmp_path,capsys):
assert not comparison.exists()
assert main(['video-loop-compare',*baseline,*origin,*candidate,*src,'--repair-evidence',str(report),'--report',str(comparison)])==0
c=json.loads(comparison.read_bytes())
- assert c['verdict']=='improved' and c['schema_version']==3 and c['metric_version']=='source-restoration-v2'
+ assert c['verdict']=='improved' and c['schema_version']==4 and c['metric_version']==METRIC
assert set(c['candidate']['artifacts'])==set(c['origin']['artifacts'])=={'strip','meta','report','gif','webp'}
assert c['origin']['artifacts']==r['origin_artifacts']
@@ -282,10 +312,11 @@ def test_key_spill_stays_out_and_the_rest_of_the_cell_is_restored(case,tmp_path)
assert {r['comparison']['verdict'] for r in outcomes.values()}=={'improved'}
r=outcomes[2]
assert r['partial']['protected']['count']==16 and r['partial']['alpha_equals_source']
- assert r['partial']['protected_pixels']['alpha_conflicts']==0
+ assert r['partial']['protected_pixels']['alpha_conflicts']==0 and r['partial']['capped']['count']==0
assert r['partial']['protected_pixels']['pixels'][0]=={
'x':25,'y':25,'origin_rgba':list(FILL),'source_rgba':[180,205,120,255],
- 'origin_key_weight':0,'source_key_weight':255*(205-180-8),'alpha_equal':True}
+ 'origin_key_weight':0,'source_key_weight':255*(205-180-8),
+ 'source_key_excess':205-180,'source_window_key_weight':255*(205-180-8),'alpha_equal':True}
made,before,tinted=cells(outputs(r))[2],cells(case[0])[2],cleaned(tinted)
for y in range(H):
for x in range(W):
@@ -307,7 +338,8 @@ def test_protected_pixel_with_other_coverage_ends_that_proposal_only(case,tmp_pa
assert r['proposal_id'] and 'comparison' not in r and 'candidate_artifacts' not in r
assert r['partial']['protected_pixels']=={'count':1,'alpha_conflicts':1,'listed':1,'pixels':[{
'x':15,'y':y,'origin_rgba':list(FILL),'source_rgba':[30,60,30,170],
- 'origin_key_weight':0,'source_key_weight':170*(60-30-8),'alpha_equal':False}]}
+ 'origin_key_weight':0,'source_key_weight':170*(60-30-8),
+ 'source_key_excess':60-30,'source_window_key_weight':170*(60-30-8),'alpha_equal':False}]}
assert not (tmp_path/f"candidate-{r['proposal_index']}").exists()
# The other proposal is independent of it.
assert outcomes[8]['comparison']['verdict']=='improved'
@@ -365,7 +397,9 @@ def test_partial_copy_formula():
for x,pixel in enumerate(pixels): im.putpixel((x,0),pixel)
return im
def made(origin,reference,key='green'):
- part=restoration.partial(image(origin),image(reference),key)
+ # A cell at its frame's size is a copy: every excess in it is the source's own, so none is capped.
+ part=restoration.partial(image(origin),image(reference),image(reference),key)
+ assert not part.capped.any()
return part,[part.frame.getpixel((x,0)) for x in range(len(origin))]
# A larger drop at one place does not pay for a rise at another.
part,pixels=made([(100,110,100,255),(100,108,100,255)],[(100,108,100,255),(100,109,100,255)])
@@ -387,6 +421,85 @@ def test_partial_copy_formula():
assert made([FILL],[accent],'green')[1]==[accent]
+def seam(left,right):
+ """A 16 px picture whose columns 0-6 are one colour and the rest another: halved, column 3 mixes them evenly."""
+ im=Image.new('RGBA',(16,16),right)
+ im.paste(left,(0,0,7,16))
+ return im
+
+
+def halved(im):
+ return resize_cell(im,(im.width//2,im.height//2))
+
+
+COLUMN=[[x==3 for x in range(8)]]*8
+
+
[email protected]('key,left,right,mixed,capped',[
+ ('green',WARM,COOL,(105,114,105,255),(105,113,105,255)),
+ # Magenta's hue is min(R, B) - G: both keyed channels come down one step.
+ ('magenta',ROSE,IRIS,(109,100,109,255),(108,100,108,255)),
+ # A part-covered side: the mix is weighted by coverage, and the capped pixel keeps the source's.
+ ('green',(126,120,103,255),(81,120,126,127),(111,120,111,191),(111,119,111,191))])
+def test_one_step_the_resample_made_is_capped(key,left,right,mixed,capped):
+ drawn=seam(left,right); ref=halved(drawn)
+ origin=Image.new('RGBA',ref.size,FILL)
+ # No source pixel is over the bar; the cell's mixed column is exactly one step over it.
+ assert {ref.getpixel((3,y)) for y in range(8)}=={mixed}
+ assert int(restoration.source_key_weight(drawn,ref.size,key).max())==0
+ assert int(restoration.key_weight(np.asarray(ref),key)[:,3].min())==mixed[3]
+ part=restoration.partial(origin,ref,drawn,key)
+ assert part.capped.tolist()==COLUMN and not part.protected.any() and not part.alpha_conflict
+ assert part.copied.tolist()==[[x!=3 for x in range(8)]]*8
+ for y in range(8):
+ for x in range(8):
+ assert part.frame.getpixel((x,y))==(capped if x==3 else ref.getpixel((x,y)))
+ assert not restoration.key_weight(np.asarray(part.frame),key).any()
+ # The other key does not read that colour as its hue: the whole source cell is copied.
+ other=restoration.partial(origin,ref,drawn,'magenta' if key=='green' else 'green')
+ assert not other.capped.any() and not other.protected.any() and other.frame.tobytes()==ref.tobytes()
+ # The cap answers a rise only. Where the delivered pixel already carries more, the source's is copied as it is.
+ stained=Image.new('RGBA',ref.size,(100,140,100,255) if key=='green' else (140,100,140,255))
+ kept=restoration.partial(stained,ref,drawn,key)
+ assert kept.copied.all() and not kept.capped.any() and kept.frame.tobytes()==ref.tobytes()
+
+
[email protected]('key,left,right,dark',[
+ ('green',(120,115,90,255),(90,115,120,255),None), # two steps
+ ('green',(240,230,40,255),(40,230,240,255),None), # yellow meets cyan: the mix is a bright green
+ ('magenta',(230,60,60,255),(60,60,230,255),None), # red meets blue
+ ('green',(240,230,40,255),(40,230,240,255),(5,6))]) # a dark pixel under the mix does not make a deep cap a small one
+def test_a_deeper_made_excess_keeps_the_delivered_pixel(key,left,right,dark):
+ drawn=seam(left,right)
+ if dark: drawn.putpixel(dark,(20,20,20,255))
+ ref=halved(drawn)
+ part=restoration.partial(Image.new('RGBA',ref.size,FILL),ref,drawn,key)
+ # Nothing under those places is over the bar either: it is the depth alone that refuses the cap.
+ assert int(restoration.source_key_weight(drawn,ref.size,key).max())==0
+ assert not part.capped.any() and part.protected.tolist()==COLUMN and not part.alpha_conflict
+ assert all(part.frame.getpixel((3,y))==FILL for y in range(8))
+
+
+def test_an_excess_a_source_pixel_carries_is_not_capped():
+ origin=Image.new('RGBA',(8,8),FILL)
+ # A key-coloured source pixel, barely covered, under two of the mixed places. Those two still read
+ # one step over the bar, but the excess is the source's own there: the delivered pixel stays, and
+ # its coverage is not the source's.
+ drawn=seam(WARM,COOL); drawn.putpixel((5,6),(60,200,60,1))
+ ref=halved(drawn)
+ part=restoration.partial(origin,ref,drawn,'green')
+ assert [ref.getpixel((3,y))[:3] for y in (2,3)]==[(105,114,105)]*2
+ assert part.protected.tolist()==[[x==3 and y in (2,3) for x in range(8)] for y in range(8)]
+ assert part.capped.tolist()==[[x==3 and y not in (2,3) for x in range(8)] for y in range(8)]
+ assert part.alpha_conflict
+ # Under zero coverage the same colour carries nothing: every mixed place is capped, with the source's coverage.
+ drawn.putpixel((5,6),(60,200,60,0))
+ ref=halved(drawn)
+ part=restoration.partial(origin,ref,drawn,'green')
+ assert part.capped.tolist()==COLUMN and not part.protected.any() and not part.alpha_conflict
+ assert ref.getpixel((3,3))[3]<255 and part.frame.getchannel('A').tobytes()==ref.getchannel('A').tobytes()
+
+
def recolour_outline(colour):
def paint(f):
for y in range(8,58):
@@ -578,6 +691,8 @@ def test_replaced_origin_or_source_is_error(case,tmp_path,adopted):
def test_other_policy_receipt_is_not_read_as_this_one(case,tmp_path,adopted):
origin,first,current=adopted
for change in (lambda r: r['restoration'].update(policy_version='another-policy'),
+ # The same cell under the contract before this one, which never capped: its receipt is not this one's.
+ lambda r: r['restoration'].update(metric_version='source-restoration-v2',policy_version='key-protected-source-copy-v1'),
lambda r: r.update(restoration={'operation':'restore_active_cut','metric_version':'source-restoration-v1',
'target':2,'restored_cells':[2]})):
rewrite_receipt(current,change)
@@ -596,7 +711,8 @@ def test_old_proposal_cannot_be_compared_on_the_new_baseline(case,tmp_path,adopt
with pytest.raises(ValueError,match='does not bind'):
compare_result(case,second,baseline=case[0],origin=origin)
# Evidence of another schema or metric is not this contract's.
- for field,value in (('schema_version',1),('metric_version','source-restoration-v1'),('policy_version','another-policy')):
+ for field,value in (('schema_version',2),('metric_version','source-restoration-v2'),
+ ('policy_version','key-protected-source-copy-v1'),('policy_version','another-policy')):
with pytest.raises(ValueError,match='does not bind'):
compare_result(case,{**second,field:value},baseline=current,origin=origin)
# A candidate whose receipt drops the earlier cell is not the verified one.
@@ -607,6 +723,139 @@ def test_old_proposal_cannot_be_compared_on_the_new_baseline(case,tmp_path,adopt
assert (outcome['verdict'],outcome['reasons'])==('unknown',['candidate-report-differs-from-verified-receipt'])
[email protected]('scaled,left,right,mixed,capped',[
+ ('green',WARM,COOL,(105,114,105,255),(105,113,105,255)),
+ ('magenta',ROSE,IRIS,(109,100,109,255),(108,100,108,255))],indirect=['scaled'])
+def test_cli_restores_a_resampled_cell_with_its_made_excess_capped(scaled,tmp_path,left,right,mixed,capped):
+ paths,_,_=scaled
+ painted={k:repaint_source(scaled,k,seam_patch(left,right)) for k in (2,8)}
+ baseline,origin,src=cli_args(scaled)
+ report=tmp_path/'repair.json'
+ assert main(['video-loop-repair',*baseline,*origin,*src,'--out-dir',str(tmp_path/'cli'),'--report',str(report)])==0
+ r=json.loads(report.read_bytes())
+ assert (r['status'],r['schema_version'],r['metric_version'],r['policy_version'])==('candidate',3,METRIC,POLICY)
+ target,part=r['target'],r['partial']
+ assert (part['capped']['count'],part['protected']['count'],part['alpha_equals_source'])==(6,0,True)
+ assert part['capped_pixels']=={'count':6,'listed':6,'pixels':[
+ {'x':27,'y':y,'origin_rgba':list(FILL),'source_rgba':list(mixed),'output_rgba':list(capped),
+ 'source_window':[53,2*y-1,56,2*y+2]} for y in range(25,31)]}
+ candidate=[a for k,p in r['outputs'].items() for a in ('--candidate-'+k,p)]
+ comparison=tmp_path/'comparison.json'
+ assert main(['video-loop-compare',*baseline,*origin,*candidate,*src,'--repair-evidence',str(report),'--report',str(comparison)])==0
+ c=json.loads(comparison.read_bytes())
+ assert (c['verdict'],c['schema_version'],c['metric_version'],c['policy_version'])==('improved',4,METRIC,POLICY), c['reasons']
+ key=c['axes']['key_colour']
+ assert (key['introduced_pixels'][target],key['introduced_excess'][target],key['candidate'][target])==(0,0,0)
+ assert c['cleared_faults']==[{'cell':target,'faults':['outline']}]
+ assert c['restoration']['capped']==part['capped'] and c['restoration']['capped_pixels']==part['capped_pixels']
+ # The final strip: the source's cell but for the six capped pixels, their coverage the source's; no other cell touched.
+ made,before=cells(outputs(r)),cells(paths)
+ reference=resize_cell(cleaned(painted[target]),(W,H))
+ assert reference.getpixel((27,25))==mixed
+ for y in range(H):
+ for x in range(W):
+ assert made[target].getpixel((x,y))==(capped if x==27 and 25<=y<=30 else reference.getpixel((x,y)))
+ assert all(a.tobytes()==b.tobytes() for k,(a,b) in enumerate(zip(before,made)) if k!=target)
+ receipt=json.loads(outputs(r)['report'].read_bytes())['restoration']
+ assert (receipt['metric_version'],receipt['policy_version'])==(METRIC,POLICY)
+ assert receipt['applied'][0]['capped']==part['capped']
+
+
+def test_whole_source_cell_with_a_made_excess_is_still_refused(scaled,tmp_path):
+ paths,inputs,_=scaled
+ for k in (2,8): repaint_source(scaled,k,seam_patch(WARM,COOL))
+ r=repair(scaled,tmp_path)
+ target=r['target']
+ # The guard itself is unchanged: the source's own cell there raises it at each of the six places.
+ o,_=restoration.read_loop(paths)
+ state,_=restoration.chain(o,o,source.Source.read(**inputs))
+ whole=[state.projection.frames[k] if k==target else f for k,f in enumerate(o.frames)]
+ judged=restoration.judge(o.frames,whole,state.projection.frames,state.key)
+ assert judged['axes']['key_colour']['introduced_pixels'][target]==6
+ assert judged['verdict']=='regressed' and 'key_colour:regressed' in judged['reasons']
+ # Written out as the candidate, one capped place given the source's pixel back is not the verified cell.
+ forged=outputs(r)
+ fs=cells(forged); fs[target].putpixel((27,25),(105,114,105,255)); write_strip(forged,fs)
+ c,_=restoration.read_loop(forged); r['candidate_artifacts']=c.artifacts
+ outcome=compare_result(scaled,r)
+ assert (outcome['verdict'],outcome['reasons'])==('regressed',['changed-cell-is-not-the-verified-source-copy'])
+
+
+def test_deeper_made_excess_in_a_resampled_cell_keeps_the_delivered_pixels(scaled,tmp_path):
+ for k in (2,8): repaint_source(scaled,k,seam_patch((120,115,90,255),(90,115,120,255)))
+ r=repair(scaled,tmp_path)
+ part=r['partial']
+ assert (part['capped']['count'],part['protected']['count'],part['alpha_equals_source'])==(0,6,True)
+ assert part['protected_pixels']['pixels'][0]=={
+ 'x':27,'y':25,'origin_rgba':list(FILL),'source_rgba':[105,115,105,255],'origin_key_weight':0,
+ 'source_key_weight':255*2,'source_key_excess':10,'source_window_key_weight':0,'alpha_equal':True}
+ assert r['comparison']['verdict']=='improved', r['comparison']['reasons']
+ made=cells(outputs(r))[r['target']]
+ assert all(made.getpixel((27,y))==FILL for y in range(25,31))
+
+
+def test_normal_cell_with_a_made_excess_is_left_as_delivered(tmp_path):
+ # A delivered normal cell can carry such an excess itself. It is not a target and is never rewritten.
+ case=build(tmp_path,zoom=2,paint={5:seam_patch(WARM,COOL)})
+ paths,_,clean=case
+ assert [clean[5].getpixel((27,y)) for y in range(25,31)]==[(105,114,105,255)]*6
+ r=repair(case,tmp_path)
+ assert r['comparison']['verdict']=='improved', r['comparison']['reasons']
+ assert r['comparison']['axes']['key_colour']['baseline'][5]==r['comparison']['axes']['key_colour']['candidate'][5]==6
+ assert cells(outputs(r))[5].tobytes()==clean[5].tobytes()
+ # With nothing damaged the request copies the five files, that cell included.
+ write_strip(paths,clean)
+ noop=repair(case,tmp_path,out='noop')
+ assert noop['status']=='no_change' and noop['comparison']['verdict']=='non_regressing'
+ assert all(outputs(noop)[k].read_bytes()==paths[k].read_bytes() for k in paths)
+
+
+def capped_baseline(scaled,tmp_path):
+ """A resampled loop whose first restored cell holds capped pixels, adopted as the next baseline."""
+ for k in (2,8): repaint_source(scaled,k,seam_patch(WARM,COOL))
+ first=repair(scaled,tmp_path)
+ assert first['comparison']['verdict']=='improved' and first['partial']['capped']['count']==6
+ return scaled[0],first,outputs(first)
+
+
+def test_capped_cell_is_rebuilt_from_the_origin_by_later_requests(scaled,tmp_path,capped_baseline):
+ origin,first,current=capped_baseline
+ second=repair(scaled,tmp_path,baseline=current,origin=origin,out='next-request')
+ assert second['comparison']['verdict']=='improved', second['comparison']['reasons']
+ assert second['applied_cells']==[first['target']] and {first['target'],second['target']}=={2,8}
+ assert second['baseline_artifacts']==first['candidate_artifacts'] and second['origin_artifacts']==first['origin_artifacts']
+ receipt=json.loads(outputs(second)['report'].read_bytes())['restoration']
+ assert [e['target'] for e in receipt['applied']]==[first['target'],second['target']]
+ assert [e['capped']['count'] for e in receipt['applied']]==[6,6]
+ # A separate comparison of the final files agrees with the preview.
+ assert compare_result(scaled,second,baseline=current,origin=origin)['verdict']=='improved'
+ final=outputs(second)
+ third=repair(scaled,tmp_path,baseline=final,origin=origin,out='all-restored')
+ assert third['status']=='no_change' and third['comparison']['verdict']=='non_regressing'
+ assert third['applied_cells']==[first['target'],second['target']] and third['target'] is None
+ assert all(outputs(third)[k].read_bytes()==final[k].read_bytes() for k in final)
+
+
[email protected]('tamper,refusal',[
+ ('capped_mask',REBUILT),('capped_digest',REBUILT),('capped_field_removed',REBUILT),
+ ('uncapped_pixel',PIXELS),('deeper_cap',PIXELS),('cap_elsewhere',PIXELS)])
+def test_changed_capped_cell_or_its_receipt_is_error(scaled,tmp_path,capped_baseline,tamper,refusal):
+ origin,first,current=capped_baseline
+ target=first['target']
+ entry=lambda report: report['restoration']['applied'][0]
+ if tamper=='capped_mask': rewrite_receipt(current,lambda r: entry(r)['capped']['rle'].__setitem__(0,entry(r)['capped']['rle'][0]+1))
+ elif tamper=='capped_digest': rewrite_receipt(current,lambda r: entry(r)['capped'].update(sha256='0'*64))
+ elif tamper=='capped_field_removed': rewrite_receipt(current,lambda r: entry(r).pop('capped'))
+ # A capped place given the source's own pixel back, capped a step further, or a cap where none was due.
+ elif tamper=='uncapped_pixel': repaint_cell(current,target,(27,25),(105,114,105,255))
+ elif tamper=='deeper_cap': repaint_cell(current,target,(27,25),(105,112,105,255))
+ else: repaint_cell(current,target,(26,25),(120,113,90,255))
+ with pytest.raises(ValueError,match=refusal):
+ repair(scaled,tmp_path,baseline=current,origin=origin,out='next-request')
+ assert not (tmp_path/'next-request').exists()
+
+
def test_low_scale_legacy_search_is_bounded():
from sprite_gen.util.resample import resize_cell
im=Image.new('RGBA',(64,80))
diff --git a/tests/video/test_strip_resample.py b/tests/video/test_strip_resample.py
index 71b48d1..3402a20 100644
--- a/tests/video/test_strip_resample.py
+++ b/tests/video/test_strip_resample.py
@@ -80,17 +80,41 @@ def test_a_cell_at_its_own_size_is_untouched() -> None:
assert out is not src and out.tobytes() == src.tobytes()
[email protected]("scale", (0.37, 0.5, 0.9662, 1.0256, 1.5, 2.3))
-def test_coverage_clamp_reads_every_pixel_the_colour_filter_mixes(scale: float) -> None:
- n_src = 41
- n_out = round(n_src * scale)
- lo, hi = resample._support_bounds(n_src, n_out, 1.0)
- for x in range(n_src):
- impulse = np.zeros((1, n_src), np.float32)
- impulse[0, x] = 1.0
- response = np.asarray(Image.fromarray(impulse).resize((n_out, 1), Image.Resampling.HAMMING))[0]
- for u in np.nonzero(response > 1e-6)[0]:
- assert lo[u] <= x <= hi[u], (scale, x, u, lo[u], hi[u])
+def test_mix_windows_hold_every_pixel_the_colour_filter_mixes() -> None:
+ # Every pair of lengths up to 48, shrinking and enlarging, and a few long ones: one impulse per
+ # source column, as the rows of one picture, scaled along the columns only.
+ pairs = [(n_src, n_out) for n_src in range(2, 49) for n_out in range(1, 2 * n_src + 1)]
+ for n_src, n_out in [*pairs, (720, 256), (256, 720), (997, 431), (431, 997)]:
+ (_, _), (lo, hi) = resample.mix_windows((n_src, 1), (n_out, 1))
+ impulses = Image.fromarray(np.eye(n_src, dtype=np.float32))
+ response = np.asarray(impulses.resize((n_out, n_src), Image.Resampling.HAMMING))
+ x, u = np.nonzero(response > 1e-6)
+ assert ((lo[u] <= x) & (x <= hi[u])).all(), (n_src, n_out)
+
+
+def test_a_mix_can_carry_more_key_hue_than_the_colours_under_it() -> None:
+ # Each channel of a cell stays inside the colours under it. The key hue's excess (green:
+ # G - max(R, B)) does not: a colour led by red mixed with one led by blue is led by neither
+ # as far. `resize_cell` leaves the mix as it is; it knows no key and caps nothing.
+ def over_the_bar(image: Image.Image) -> int:
+ a = np.asarray(image).astype(int)
+ return int(np.count_nonzero((a[..., 3] > 0) & (a[..., 1] - np.maximum(a[..., 0], a[..., 2]) > 8)))
+
+ def stripes(one: tuple[int, ...], other: tuple[int, ...]) -> Image.Image:
+ image = Image.new("RGBA", (60, 40), one)
+ for x in range(5, 60, 10):
+ image.paste(other, (x, 0, x + 5, 40))
+ return image
+
+ crossed = stripes((120, 114, 90, 255), (90, 114, 120, 255)) # led by red, led by blue
+ same = stripes((120, 114, 90, 255), (150, 130, 60, 255)) # both led by red
+ assert over_the_bar(crossed) == over_the_bar(same) == 0
+ for size in ((30, 20), (58, 39), (61, 41), (90, 60)):
+ cell = resample.resize_cell(crossed, size)
+ assert over_the_bar(cell) > 0, size
+ assert colour_outside(crossed, cell) == 0, size
+ assert over_the_bar(resample.resize_cell(same, size)) == 0, size
+ assert over_the_bar(resample.resize_cell(crossed, crossed.size)) == 0
@pytest.mark.parametrize("anchor", ("none", "feet"))