Generate clean 2D game sprites & animation atlases — component-row pipeline: state rows, alpha cleanup, frame extraction, runtime atlases. Codex/Claude skill.
fix(video): a clip that is done and whose download fails is downloaded again, then unfinished on its request_id; bytes that are not an mp4 are unfinished, not refused
Soohong Kim7b8e3bd0da50
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex eb4e7b3..c3e6332 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -13,6 +13,10 @@ All notable public changes to `sprite-gen` are recorded here. Versions track the- **`tests/gen/test_video.py`** asks again by id and loses polls: `xai.http_json` raises `xai.TransportError` with the old line for a DNS failure, a timeout and a reset; lost, pending, refused, lost, done is asked on the one request with waits 8, 4, 8, 16 and published with `polls_unanswered` 2; lost until a 30 s timeout waits 8, 16, 6 and ends unfinished with `last_poll.error`; a lost POST is sent once and is not unfinished; a 401 or 403 poll, on the login or on the key, ends unfinished at once naming its fix; `video-resume` polls with no POST and announces no charge, ends unfinished on the same id when refused, lost or pending past the timeout or when its credential is rejected, ends with `(HTTP 404)` for an unknown id, refuses eight ids that cannot be one before any call, and its CLI writes the resume report, the unfinished report and an empty stdout; the unified CLI routes it under `gen`.- **Fix: a poll whose answer is cut short, garbled or not JSON is asked again on the same request — the run no longer ends with no `request_id`.** A poll answer that broke after it began — a body that closed before its `Content-Length` (`http.client.IncompleteRead`), a status line that is not one (`BadStatusLine`), a refusal's body cut the same way — escaped `video`, `video-extend`, `video-edit` and `video-resume` as a raw exception, with no `request_id`, no unfinished report and no resume command, so a caller could only submit the clip again and buy it twice; a `200` poll whose body is not JSON ended the run on `xai: response was not valid JSON`, also with no id. Both are now a poll that got no answer it could read, as a lost one is: asked again on the same `request_id` with the same backoff and stderr note (`got no answer (IncompleteRead(10 bytes read, 190 more expected))`, `got no answer (response was not valid JSON)`), counted in `polls_unanswered`, and unfinished at the poll timeout with that reason in `last_poll.error`. A POST so answered still ends at once and is never sent again — no `request_id` could be read: on the transport's line for a broken answer (a raw exception before), on the not-JSON line it always had. `xai.http_json` raises `xai.TransportError` for every answer it cannot read whole, each with the line it always had; `reason` is the `http.client` exception's repr for a broken answer, `response was not valid JSON` for the other.- **`tests/gen/test_video.py`** answers over a real socket from a local server: `xai.http_json` raises `xai.TransportError` for a body closed 10 bytes into its 200, a 503 body cut the same way, a garbage status line (the transport's line) and a whole non-JSON `200` (the not-JSON line); cut, garbled, not JSON, cut refusal, pending, done is asked on the one request with waits 8, 16, 32, 60, 4 and published with `polls_unanswered` 4; a resumed poll cut short is asked again and published; cut until a 30 s timeout, the CLI leaves the unfinished report with `last_poll.error` the `IncompleteRead` and an empty stdout; a cut POST is sent once and is not unfinished.+- **Fix: a clip that is done and whose download fails leaves the run unfinished on its `request_id` — never a second purchase.** Once the poll said done the clip is made and may be billed, yet a failed download ended `video`, `video-extend`, `video-edit` and `video-resume` with no `request_id`, no unfinished report and no resume command: a refused connection on `video: cannot download the clip: …`, a body cut short of its `Content-Length` (`http.client.IncompleteRead`) or a read that timed out as a raw exception, a refused download on `video: download failed with HTTP <n>`, and bytes that are not an mp4 on `… refusing to publish` — so a caller could only submit the clip again and buy it twice. A download that gets no answer it can read, or that the clip's host refuses with `429`, `500`, `502`, `503` or `504`, is now downloaded again after 8 s and then 16 s, three downloads in all, with a stderr note before each (`download N of generation <id> got no answer (<reason>); downloading again in 8s`, `… was refused (503); …`); one still failing, one refused with another status (a `401` or `403` still asked once more with the bearer token first), or bytes that are not an mp4 end the run unfinished on that id: ``… is done, but its download got no answer (last: <reason>) after 3 attempts; the clip is made and may be billed — ask for request_id=<id> again with `sprite-gen video-resume --request-id <id> --out <out>` …``, no `(HTTP <n>)` on it. `video-resume` polls afresh, so it downloads from the URL the new poll names. `video.http_download` raises `xai.TransportError` (its line still `video: cannot download the clip: <reason>`) for a download that got no answer and `video.DownloadRefused`, a `SystemExit` carrying `status`, for a refused one; `xai.NO_ANSWER` and `xai.no_answer` are the transport's classifier, now shared by both.+- **Output change: bytes that came whole and are not an mp4 are unfinished, not a refusal.** xAI's one content-policy signal is `respect_moderation: false` on the done poll, read before any download; anything else at the clip's URL — an expired link's page, a proxy's error page served `200` — is a fault in delivering a clip that is made, and the `request_id` is the only way back to it. `video-resume` costs a poll and asks for a fresh URL; a refusal left only a new submission. The same URL brings the same bytes, so the run does not download it again itself (docs/video.md, An unfinished generation). `video.verify_mp4` is gone.+- **Output change: reports count the clip's downloads.** The success report gains `downloads`, how many downloads the clip took, the last one whole (1 unless one failed). An unfinished report of a clip that could not be downloaded gains `downloads` and `last_download` — `http` the status the clip's host refused it with, else `null`; `error` what the transport said, or `not an mp4 (no ftyp box)` with the `bytes` that came — and its `last_poll` is the done poll; an unfinished report of a poll is unchanged.+- **`tests/gen/test_video.py`** downloads the clip over a real socket from the local server: a clip body closed 10 bytes into its 76, a read held open past a 1 s timeout and a connection to a port nothing holds are each downloaded three times with waits 8 and 16, and the CLI leaves the unfinished report naming the request with `downloads` 3, `last_download.error` the transport's words and an empty stdout; a cut download, then a `503`, then the whole clip publishes with `downloads` 3; a `503` three times and a `404` once end unfinished with `last_download.http`, no `(HTTP n)`; 34 whole bytes of HTML end unfinished at once with `last_download.bytes`; and a run left unfinished by its downloads is published by `video-resume` on the id its report names, from the fresh URL of a new poll and with no POST. The fake transport's non-mp4 refusal test is that HTML test now. A port held bound and not listening is no refused port on macOS (its SYN is dropped and the connect times out), so the test takes a free port and lets it go.- **Output change: an `openai` image report carries the response's rate-limit headers — `extra.rate_limit`.** The provider dropped the response headers, so what the account may spend per minute and has left could not be read off real traffic. `extra.rate_limit` now holds every `x-ratelimit-*` header the response carried, names lower-cased and values as sent, and `{}` when none came (docs/gen.md). `openai_provider.http_image` returns the response headers as a third value.- **`tests/gen/test_openai_provider.py`** answers with headers: the report holds exactly the `x-ratelimit-*` ones among others of any case, and `{}` for a response without any.diff --git a/docs/video.md b/docs/video.mdindex 81488c8..076fe06 100644--- a/docs/video.md+++ b/docs/video.md@@ -165,7 +165,17 @@ What happens, in order:typically a login that expired — is unfinished at once. No clip is written on anyfailure path.4. Download `video.url`, verify the bytes start with an mp4 `ftyp` box, then move- the file into `--out` atomically (`.part` staging).+ the file into `--out` atomically (`.part` staging). The clip is made by now and may+ be billed, so a download that fails never ends the run without its `request_id`. One+ that gets no answer it can read (the connection refused or timed out, the body cut+ short of its `Content-Length`) or that the clip's host refuses with `429`, `500`,+ `502`, `503` or `504` is downloaded again, after 8 s and then 16 s — three downloads+ in all, each with its own 120 s read timeout and none bounded by the poll timeout —+ and a `[video] download N of generation <id> got no answer (<reason>); downloading+ again in 8s` (or `… was refused (503); …`) line goes to stderr before each. One still+ failing, one refused with another status (a `401` or `403` is asked once more with+ the bearer token first), or bytes that are not an mp4 end the run *unfinished*+ (below).Reference-to-video tends to lock the opening composition to a referenced sceneimage (probe m3: the clip opened on the third reference's framing), so put@@ -224,8 +234,10 @@ the request — the unfinished report's `auth_source`; refresh an expired login- The report is a `sprite-gen-video-report` with `mode: "resume"`, the same`request_id`, `inputs: {}` and `duration_requested: null`; `model` and`duration_reported` are what the finished poll says.-- Cut short again — the timeout, or a rejected credential — it is unfinished on the- same id, `verb: "video-resume"` (below), and can be resumed again.+- Cut short again — the timeout, a rejected credential, or a download that failed —+ it is unfinished on the same id, `verb: "video-resume"` (below), and can be resumed+ again. Each resume polls afresh, so a clip whose download failed is fetched from the+ URL the new poll names.- A request xAI does not know (`404`), or one that failed or expired, ends with itsstatus like any failed poll.- The clip is checked as an mp4 (`ftyp` box) only: `video-extend`'s length rule needs@@ -240,22 +252,29 @@ the request — the unfinished report's `auth_source`; refresh an expired login`video`), `bytes`, `host`, requested/reported duration (`input_duration` forextend / edit), resolution, aspect ratio, audio flag, `elapsed_seconds`, and`polls`, `polls_refused` (how many of `polls` xAI answered 429 or 5xx, each asked-again) and `polls_unanswered` (how many got no answer they could read, each asked-again). The pre-modes `image` field stays. **Tokens and download URLs are never+again), `polls_unanswered` (how many got no answer they could read, each asked+again) and `downloads` (how many downloads the clip took, the last one whole: 1 unless+one failed). The pre-modes `image` field stays. **Tokens and download URLs are neverprinted or written** — only the download host (`vidgen.x.ai`).### An unfinished generationxAI accepted the request and the run ended before it did — the poll timeout came with-its polls still pending, refused or unanswered, or xAI rejected the credential on a-poll. It may finish and be billed, so the run ends naming the request and the-`video-resume` command that asks for it again, never as a refusal to send again:+its polls still pending, refused or unanswered, xAI rejected the credential on a+poll, or the poll said done and the clip could not be downloaded (step 4). It may+finish and be billed — a clip that is done is made — so the run ends naming the+request and the `video-resume` command that asks for it again, never as a refusal to+send again:```video: generation <id> was accepted, but its polls were refused (last answer 429) until the poll timeout (600s, 14 polls); it may still finish and be billed — ask for request_id=<id> again with `sprite-gen video-resume --request-id <id> --out /abs/clip.mp4` instead of submitting a new request; nothing was writtenvideo: generation <id> was accepted, but its polls got no answer (last: timed out) until the poll timeout (600s, 14 polls); it may still finish and be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was writtenvideo: generation <id> still 'pending' after 151 polls — poll timeout; it may still finish and be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was writtenvideo: generation <id> was accepted, but xAI rejected the credential (grok-login) on poll 30 (answer 401); it may still finish and be billed — refresh the login with `grok models` run from an empty directory (e.g. `cd "$(mktemp -d)"`) or `grok login`, then ask for request_id=<id> again with `…` instead of submitting a new request; nothing was written+video: generation <id> is done, but its download got no answer (last: IncompleteRead(1048576 bytes read, 3195904 more expected)) after 3 attempts; the clip is made and may be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was written+video: generation <id> is done, but its download was refused (last answer 503) after 3 attempts; the clip is made and may be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was written+video: generation <id> is done, but its download was refused (answer 404); the clip is made and may be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was written+video: generation <id> is done, but its download brought 34 bytes that are not an mp4 (no ftyp box); the clip is made and may be billed — ask for request_id=<id> again with `…` instead of submitting a new request; nothing was written```The line carries no `(HTTP <n>)`, the status of a refusal line (docs/gen.md): a caller@@ -273,9 +292,39 @@ same reason. With `--report` the run leaves a`last_poll` gains `"error"` (what the transport said: `timed out`,`IncompleteRead(10 bytes read, 190 more expected)`, `response was not valid JSON`, …)-when the last poll got no answer it could read, its `http` then `null`. `auth_source`-is the credential that submitted it — the one that can ask for it again. A report-already at that path is replaced, as on success.+when the last poll got no answer it could read, its `http` then `null`. A run whose+clip could not be downloaded adds `downloads` (how many were tried) and+`last_download`, how the last one ended: `http` the status the clip's host refused it+with, else `null`; `error` what the transport said when it got no answer it could read,+or `not an mp4 (no ftyp box)` with the `bytes` that came; its `last_poll` is the done+poll (`{"http": 200, "status": "done"}`):++```json+{"kind": "sprite-gen-video-unfinished", "provider": "grok-imagine", "verb": "video",+ "auth_source": "grok-login", "model": "grok-imagine-video-1.5", "request_id": "<id>",+ "out": "/abs/clip.mp4", "polls": 31, "polls_refused": 0, "polls_unanswered": 0,+ "last_poll": {"http": 200, "status": "done"}, "downloads": 3,+ "last_download": {"http": null, "error": "IncompleteRead(1048576 bytes read, 3195904 more expected)"},+ "elapsed_seconds": 144.0}+```++Bytes that came whole and are not an mp4 make the run unfinished, not refused, and+are not downloaded again:++- They are no verdict of xAI's on the request. Its one content-policy signal is+ `respect_moderation: false` on the done poll, read before any download and refused+ as a content-policy block. Anything else at the clip's URL — an expired link's page,+ a proxy's or captive portal's error page served `200` — is a fault in delivering a+ clip that is made.+- The `request_id` is the only way back to a clip that may be billed. A refusal would+ leave one path, a new submission, which buys it again; `video-resume` costs a poll+ and polls afresh, so it downloads from the URL the new poll names. If those bytes+ are not an mp4 either, it is unfinished again — never a second purchase.+- The same URL would bring the same bytes, so the run does not download it again+ itself.++`auth_source` is the credential that submitted it — the one that can ask for it again.+A report already at that path is replaced, as on success.Every verb that polls (`video`, `video-extend`, `video-edit`, `video-resume`) doesthe same, `verb` naming it; a `video-resume` report's `model` is `null`.diff --git a/sprite_gen/gen/video.py b/sprite_gen/gen/video.pyindex ac5fd43..19e1435 100644--- a/sprite_gen/gen/video.py+++ b/sprite_gen/gen/video.py@@ -66,7 +66,7 @@ from sprite_gen.video import facing as facing_modfrom .xai import (API_BASE, AUTH_ENV, AUTH_SOURCE_API_KEY, AUTH_SOURCE_GROK_LOGIN,HTTP_TIMEOUT_SECONDS, GROK_REFRESH_COMMAND, GROK_REFRESH_WHERE, GROK_LOGIN_COMMAND,- Credential, HttpCall, TransportError, grok_home, resolve_credential, http_json,+ NO_ANSWER, Credential, HttpCall, TransportError, grok_home, resolve_credential, http_json, no_answer,)DEFAULT_MODEL = "grok-imagine-video-1.5"@@ -109,6 +109,13 @@ POLL_TIMEOUT_SECONDS = int(os.environ.get("SPRITE_GEN_VIDEO_TIMEOUT_SECONDS", "6# which bounds the whole poll however its polls are answered.POLL_RETRY_STATUSES = frozenset({429, 500, 502, 503, 504})POLL_BACKOFF_MAX_SECONDS = 60.0+# A clip the poll says is done is made, and may be billed: its request_id is the one way back to it. A+# download that gets no answer it can read, or that the clip's host refuses with one of+# POLL_RETRY_STATUSES, is downloaded again after the polls' doubling wait (8 s, then 16 s), up to+# DOWNLOAD_ATTEMPTS downloads in all. One still failing, one refused with another status, or bytes+# that are not an mp4, end the run unfinished on that request_id: `video-resume` polls it again for+# a fresh URL, where a new submission would buy the clip twice.+DOWNLOAD_ATTEMPTS = 3# Every mp4/mov starts with a size-prefixed `ftyp` box: bytes 4..8 spell it.MP4_FTYP_OFFSET = 4MP4_FTYP = b"ftyp"@@ -118,7 +125,7 @@ _FAILED_STATUSES = ("failed", "error", "expired")class GenerationUnfinished(SystemExit):"""xAI accepted the request (it may bill it) and the run ended before it did: the poll timeout- came, or xAI rejected the credential on a poll.+ came, xAI rejected the credential on a poll, or the clip it made could not be downloaded.The line names `request_id=<id>` and the `video-resume` command that asks for it again, andcarries no `(HTTP n)`, a refusal line's status: a caller that reads `(HTTP 429)` there sends the@@ -127,7 +134,8 @@ class GenerationUnfinished(SystemExit):def __init__(self, message: str, *, verb: str, request_id: str, auth_source: str, model: Any, out: Path,polls: int, polls_refused: int, polls_unanswered: int, last_http: int | None, last_status: Any,- last_error: str | None, elapsed_seconds: float) -> None:+ last_error: str | None, elapsed_seconds: float, downloads: int = 0,+ last_download: dict[str, Any] | None = None) -> None:super().__init__(message)self.verb = verbself.request_id = request_id@@ -141,6 +149,8 @@ class GenerationUnfinished(SystemExit):self.last_status = last_statusself.last_error = last_error # what the transport said when the last poll got no answer it could readself.elapsed_seconds = elapsed_seconds+ self.downloads = downloads # downloads of a clip the poll said is done; 0 when none was+ self.last_download = last_download # how the last of them faileddef to_dict(self) -> dict[str, Any]:return {@@ -156,26 +166,42 @@ class GenerationUnfinished(SystemExit):"polls_unanswered": self.polls_unanswered,"last_poll": {"http": self.last_http, "status": self.last_status,**({"error": self.last_error} if self.last_error is not None else {})},+ **({"downloads": self.downloads, "last_download": self.last_download} if self.downloads else {}),"elapsed_seconds": round(self.elapsed_seconds, 3),}+class DownloadRefused(SystemExit):+ """The clip's host answered its download with an error `status` (a 401 or 403 once more with the bearer token)."""++ def __init__(self, status: int, line: str) -> None:+ super().__init__(line)+ self.status = status++def http_download(url: str, token: str) -> bytes:+ """The clip's bytes, read whole. An error status raises DownloadRefused; a download that gets no answer+ it can read whole — the connection failed or timed out, the body broke on the way — raises+ xai.TransportError, its words in `reason`."""request = urllib.request.Request(url)try:- with urllib.request.urlopen(request, timeout=HTTP_TIMEOUT_SECONDS) as response:- return response.read()- except urllib.error.HTTPError as first:- if first.code not in (401, 403):- raise SystemExit(f"video: download failed with HTTP {first.code}") from first- request.add_header("Authorization", f"Bearer {token}")try:with urllib.request.urlopen(request, timeout=HTTP_TIMEOUT_SECONDS) as response:return response.read()- except urllib.error.HTTPError as second:- raise SystemExit(f"video: download failed with HTTP {second.code} even with the bearer token") from second- except urllib.error.URLError as exc:- raise SystemExit(f"video: cannot download the clip: {exc.reason}") from exc+ except urllib.error.HTTPError as first:+ if first.code not in (401, 403):+ raise DownloadRefused(first.code, f"video: download failed with HTTP {first.code}") from first+ request.add_header("Authorization", f"Bearer {token}")+ try:+ with urllib.request.urlopen(request, timeout=HTTP_TIMEOUT_SECONDS) as response:+ return response.read()+ except urllib.error.HTTPError as second:+ raise DownloadRefused(+ second.code, f"video: download failed with HTTP {second.code} even with the bearer token"+ ) from second+ except NO_ANSWER as exc:+ reason = no_answer(exc).reason+ raise TransportError(reason, f"video: cannot download the clip: {reason}") from excdef _data_url(path: Path) -> str:@@ -202,11 +228,8 @@ def _redact_host(url: str) -> str | None:return parts[2] if len(parts) > 2 else None-def verify_mp4(data: bytes) -> None:- if len(data) < MP4_FTYP_OFFSET + len(MP4_FTYP) or data[MP4_FTYP_OFFSET : MP4_FTYP_OFFSET + len(MP4_FTYP)] != MP4_FTYP:- raise SystemExit(- f"video: downloaded {len(data)} bytes but they are not an mp4 (no ftyp box) — refusing to publish"- )+def _is_mp4(data: bytes) -> bool:+ return data[MP4_FTYP_OFFSET : MP4_FTYP_OFFSET + len(MP4_FTYP)] == MP4_FTYPdef _resolved(path: Path | None) -> Path | None:@@ -243,7 +266,7 @@ def _require_mp4_clip(clip: Path, *, verb: str) -> None:raise SystemExit(f"{verb}: input clip not found: {clip}")with clip.open("rb") as handle:head = handle.read(MP4_FTYP_OFFSET + len(MP4_FTYP))- if len(head) < MP4_FTYP_OFFSET + len(MP4_FTYP) or head[MP4_FTYP_OFFSET:] != MP4_FTYP:+ if not _is_mp4(head):raise SystemExit(f"{verb}: {clip} is not an mp4 (no ftyp box); refusing to upload it")@@ -318,6 +341,7 @@ class VideoResult:duration_reported: float | None = Nonepolls_refused: int = 0 # polls xAI answered 429 or 5xx, each asked againpolls_unanswered: int = 0 # polls that got no answer they could read (xai.TransportError), each asked again+ downloads: int = 1 # downloads of the clip, the last one whole; more than one when one failedhost: str | None = Noneimage: Path | None = Noneprompt: str = ""@@ -356,6 +380,7 @@ class VideoResult:"polls": self.polls,"polls_refused": self.polls_refused,"polls_unanswered": self.polls_unanswered,+ "downloads": self.downloads,**({"extra": self.extra} if self.extra else {}),**({"facing": self.facing} if self.facing is not None else {}),}@@ -442,6 +467,7 @@ class _Published:polls: intpolls_refused: intpolls_unanswered: int+ downloads: intelapsed_seconds: float@@ -524,21 +550,23 @@ def _poll_publish(staged file is removed and `out` is never written.The poll timeout counts from here. A run that ends before the request does — the timeout- came, or xAI rejected the credential — raises GenerationUnfinished naming the request."""+ came, xAI rejected the credential, or the clip it made could not be downloaded (DOWNLOAD_ATTEMPTS)+ — raises GenerationUnfinished naming the request."""timeout = POLL_TIMEOUT_SECONDS if poll_timeout is None else poll_timeoutdeadline = time.monotonic() + timeout- polls = polls_refused = polls_unanswered = failed_in_a_row = 0+ polls = polls_refused = polls_unanswered = failed_in_a_row = downloads = 0video_url: str | None = Noneduration_reported: float | None = Nonemodel_reported: str | None = Nonefinal: dict[str, Any] = {}- def unfinished(line: str, http: int | None, poll_status: Any, lost: str | None) -> GenerationUnfinished:+ def unfinished(line: str, http: int | None, poll_status: Any, lost: str | None,+ last_download: dict[str, Any] | None = None) -> GenerationUnfinished:return GenerationUnfinished(f"{verb}: generation {request_id} {line}", verb=verb, request_id=request_id,auth_source=credential.source, model=model, out=out, polls=polls, polls_refused=polls_refused,polls_unanswered=polls_unanswered, last_http=http, last_status=poll_status, last_error=lost,- elapsed_seconds=time.monotonic() - started,+ elapsed_seconds=time.monotonic() - started, downloads=downloads, last_download=last_download,)while True:@@ -612,8 +640,42 @@ def _poll_publish(if not video_url:raise SystemExit(f"{verb}: generation {request_id} is done but reported no video url")- data = download(video_url, credential.token)- verify_mp4(data)+ # The clip is made: whatever stops its download, the run ends naming the request, never a second purchase.+ while True:+ downloads += 1+ said: str | None = None # what the transport said when the download got no answer it could read+ refused: int | None = None # the status the clip's host refused it with+ try:+ data = download(video_url, credential.token)+ except TransportError as failed:+ said = failed.reason+ except DownloadRefused as failed:+ refused = failed.status+ else:+ if _is_mp4(data):+ break+ # Bytes read whole that are not an mp4 are no verdict of xAI's on the request (that is+ # respect_moderation, above) and the same URL brings the same bytes: video-resume asks for a fresh one.+ raise unfinished(f"is done, but its download brought {len(data)} bytes that are not an mp4 (no ftyp box); "+ f"the clip is made and may be billed — {_ask_again(request_id, out)}", http, poll_status, None,+ {"http": None, "error": "not an mp4 (no ftyp box)", "bytes": len(data)})+ again = said is not None or refused in POLL_RETRY_STATUSES+ if not again or downloads == DOWNLOAD_ATTEMPTS:+ if said is not None:+ failure = f"got no answer (last: {said}) after {downloads} attempts"+ elif again:+ failure = f"was refused (last answer {refused}) after {downloads} attempts"+ else:+ failure = f"was refused (answer {refused})"+ raise unfinished(f"is done, but its download {failure}; the clip is made and may be billed — "+ f"{_ask_again(request_id, out)}", http, poll_status, None,+ {"http": refused, **({"error": said} if said is not None else {})})+ wait = POLL_INTERVAL_SECONDS * 2 ** downloads+ # The status in plain digits, as the poll notes say it.+ failure = f"got no answer ({said})" if said is not None else f"was refused ({refused})"+ print(f"[{verb}] download {downloads} of generation {request_id} {failure}; downloading again in {wait:g}s",+ file=sys.stderr)+ sleep(wait)elapsed = time.monotonic() - startedout.parent.mkdir(parents=True, exist_ok=True)@@ -629,7 +691,7 @@ def _poll_publish(return _Published(data=data, request_id=request_id, model=model_reported, duration_reported=duration_reported,host=_redact_host(video_url), polls=polls, polls_refused=polls_refused, polls_unanswered=polls_unanswered,- elapsed_seconds=elapsed,+ downloads=downloads, elapsed_seconds=elapsed,)@@ -695,6 +757,7 @@ def generate_video(polls=published.polls,polls_refused=published.polls_refused,polls_unanswered=published.polls_unanswered,+ downloads=published.downloads,duration_requested=request.duration,duration_reported=published.duration_reported,host=published.host,@@ -776,7 +839,7 @@ def extend_video(out=out, bytes=len(published.data), model=published.model or request.model, request_id=published.request_id,auth_source=credential.source, elapsed_seconds=published.elapsed_seconds, polls=published.polls,polls_refused=published.polls_refused, polls_unanswered=published.polls_unanswered,- duration_requested=request.duration, duration_reported=published.duration_reported, host=published.host,+ downloads=published.downloads, duration_requested=request.duration, duration_reported=published.duration_reported, host=published.host,prompt=request.prompt, mode=MODE_EXTEND, inputs={"video": str(clip)}, input_duration=input_seconds,output_duration=measured.get("output"),)@@ -837,7 +900,7 @@ def edit_video(out=out, bytes=len(published.data), model=published.model or request.model, request_id=published.request_id,auth_source=credential.source, elapsed_seconds=published.elapsed_seconds, polls=published.polls,polls_refused=published.polls_refused, polls_unanswered=published.polls_unanswered,- duration_requested=0, duration_reported=published.duration_reported, host=published.host,+ downloads=published.downloads, duration_requested=0, duration_reported=published.duration_reported, host=published.host,prompt=request.prompt, mode=MODE_EDIT, inputs={"video": str(clip)}, input_duration=input_seconds,output_duration=measured.get("output"),)@@ -876,7 +939,7 @@ def resume_video(out=out, bytes=len(published.data), model=published.model, request_id=published.request_id,auth_source=credential.source, elapsed_seconds=published.elapsed_seconds, polls=published.polls,polls_refused=published.polls_refused, polls_unanswered=published.polls_unanswered,- duration_requested=None, duration_reported=published.duration_reported, host=published.host, mode=MODE_RESUME,+ downloads=published.downloads, duration_requested=None, duration_reported=published.duration_reported, host=published.host, mode=MODE_RESUME,)diff --git a/sprite_gen/gen/xai.py b/sprite_gen/gen/xai.pyindex f8b87c4..010625d 100644--- a/sprite_gen/gen/xai.py+++ b/sprite_gen/gen/xai.py@@ -140,7 +140,7 @@ class TransportError(SystemExit):after it began (a body shorter than its Content-Length, a status line that is not one), or it camewhole and was not JSON — and the server may have taken the request. Still the SystemExit and lineit always was (`NOT_JSON` for the last); a caller that may safely ask again (a poll of an accepted- request) catches this type. `reason` is what the transport said."""+ request, the download of a clip that is done) catches this type. `reason` is what the transport said."""def __init__(self, reason: str, line: str = TRANSPORT_FAILED) -> None:super().__init__(line)@@ -150,10 +150,10 @@ class TransportError(SystemExit):# Everything that ends a request before its answer is read whole. http.client raises its own# HTTPException family (IncompleteRead, BadStatusLine, LineTooLong, …) past urllib once the answer# has begun; the socket's errors are OSError, inside a URLError while the request is still sent.-_NO_ANSWER = (OSError, http.client.HTTPException)+NO_ANSWER = (OSError, http.client.HTTPException)-def _no_answer(exc: BaseException) -> TransportError:+def no_answer(exc: BaseException) -> TransportError:said = exc.reason if isinstance(exc, urllib.error.URLError) else exc# An http.client exception carries what the server sent (BadStatusLine its line, CRLF included);# its repr names the kind and keeps that on one line. RemoteDisconnected is a reset too: its words.@@ -179,11 +179,11 @@ def http_json(method: str, url: str, token: str, body: dict | None = None, *, ti# A refusal's status alone is not an answer read whole: its body broken on the way is no answer too.try:raw = exc.read().decode("utf-8", "replace")- except _NO_ANSWER as cut:- raise _no_answer(cut) from cut+ except NO_ANSWER as cut:+ raise no_answer(cut) from cuttry:return exc.code, json.loads(raw)except json.JSONDecodeError:return exc.code, refusal.RawBody(raw)- except _NO_ANSWER as exc:- raise _no_answer(exc) from exc+ except NO_ANSWER as exc:+ raise no_answer(exc) from excdiff --git a/tests/gen/test_video.py b/tests/gen/test_video.pyindex 991d945..bb4226c 100644--- a/tests/gen/test_video.py+++ b/tests/gen/test_video.py@@ -1110,8 +1110,7 @@ def test_a_submission_cut_short_is_not_unfinished_and_is_never_sent_again(tmp_pa# that timed out) or that the clip's host refuses with 429 or 5xx is downloaded again, up to three# downloads; one still failing — or refused with another status, or bytes that are not an mp4 —# leaves the run unfinished on that request_id, never an exception that names no request, and-# `video-resume` polls it again for a fresh URL. Before, a refused connection ended on-# `video: cannot download the clip: …` and a cut or timed-out read escaped raw.+# `video-resume` polls it again for a fresh URL.CLIP = _answer("200 OK", MP4, content_type="video/mp4")CLIP_CUT = _answer("200 OK", MP4[:10], length=len(MP4), content_type="video/mp4") # closes 10 bytes in