Lightweight coding agent that runs in your terminal
Rust
Use native transparency in the imagegen skill (#37788)
alicec-oai8cabf5a6cf10
diff --git a/codex-rs/skills/src/assets/samples/imagegen/SKILL.md b/codex-rs/skills/src/assets/samples/imagegen/SKILL.mdindex 4285e5e641..c39b1f921c 100644--- a/codex-rs/skills/src/assets/samples/imagegen/SKILL.md+++ b/codex-rs/skills/src/assets/samples/imagegen/SKILL.md@@ -11,8 +11,8 @@ Generates or edits images for the current project (for example website assets, gThis skill has exactly two top-level modes:-- **Default built-in tool mode (preferred):** built-in `image_gen` tool for normal image generation, editing, and simple transparent-image requests. Does not require `OPENAI_API_KEY`.-- **Fallback CLI mode:** `scripts/image_gen.py` CLI. Use when the user explicitly asks for the CLI/API/model path, or after the user explicitly confirms a true model-native transparency fallback with `gpt-image-1.5`. Requires `OPENAI_API_KEY`.+- **Default built-in tool mode (preferred):** built-in `image_gen` tool for image generation, editing, and transparent-image requests. Does not require `OPENAI_API_KEY`.+- **Fallback CLI mode:** `scripts/image_gen.py` CLI. Use when the user explicitly asks for or confirms the CLI/API/model path. Requires `OPENAI_API_KEY`.Within CLI fallback, the CLI exposes three subcommands:@@ -23,9 +23,8 @@ Within CLI fallback, the CLI exposes three subcommands:Rules:- Use the built-in `image_gen` tool by default for normal image generation and editing requests.- Do not switch to CLI fallback for ordinary quality, size, or file-path control.-- If the user explicitly asks for a transparent image/background, stay on built-in `image_gen` first: prompt for a flat removable chroma-key background, then remove it locally with the installed helper at `$CODEX_HOME/skills/.system/imagegen/scripts/remove_chroma_key.py`.-- Never silently switch from built-in `image_gen` or CLI `gpt-image-2` to CLI `gpt-image-1.5`. Treat this as a model/path downgrade and ask the user before doing it, unless the user has already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback.-- If a transparent request appears too complex for clean chroma-key removal, asks for true/native transparency, or local removal fails validation, explain that true transparency requires CLI `gpt-image-1.5 --background transparent --output-format png` because `gpt-image-2` does not support `background=transparent`, then ask whether to proceed. Run the CLI fallback only after the user confirms.+- For transparent images, ask built-in `image_gen` for a transparent background and preserve the generated alpha.+- Never silently switch from built-in `image_gen` or CLI `gpt-image-2` to CLI `gpt-image-1.5`; ask the user first unless they explicitly requested `gpt-image-1.5`.- The word `batch` by itself does not mean CLI fallback. If the user asks for many assets or says to batch-generate assets without explicitly asking for CLI/API/model controls, stay on the built-in path and issue one built-in call per requested asset or variant.- If the built-in tool fails or is unavailable, tell the user the CLI fallback exists and that it requires `OPENAI_API_KEY`. Proceed only if the user explicitly asks for that fallback.- If the user explicitly asks for CLI mode, use the bundled `scripts/image_gen.py` workflow. Do not create one-off SDK runners.@@ -50,9 +49,6 @@ Fallback-only docs/resources for CLI mode:- `references/codex-network.md`- `scripts/image_gen.py`-Local post-processing helper:-- `$CODEX_HOME/skills/.system/imagegen/scripts/remove_chroma_key.py`: removes a flat chroma-key background from a generated image and writes a PNG/WebP with alpha. Prefer auto-key sampling, soft matte, and despill for antialiased edges.-## When to use- Generate a new image (concept art, product shot, cover, website hero)- Generate a new image using one or more reference images for style, composition, or mood@@ -92,7 +88,7 @@ Execution strategy:Assume the user wants a new image unless they clearly ask to change an existing one.## Workflow-1. Decide the top-level mode: built-in by default, including simple transparent-output requests; fallback CLI only if explicitly requested or after the user explicitly confirms a transparent-output fallback.+1. Decide the top-level mode: built-in by default, including transparent-output requests; fallback CLI only if explicitly requested or confirmed.2. Decide the intent: `generate` or `edit`.3. Decide whether the output is preview-only or meant to be consumed by the current project.4. Decide the execution strategy: single asset vs repeated built-in calls vs CLI `generate-batch`.@@ -107,7 +103,7 @@ Assume the user wants a new image unless they clearly ask to change an existing- If the user's prompt is already specific and detailed, normalize it into a clear spec without adding creative requirements.- If the user's prompt is generic, add tasteful augmentation only when it materially improves output quality.10. Use the built-in `image_gen` tool by default.-11. For transparent-output requests, follow the transparent image guidance below: generate with built-in `image_gen` on a flat chroma-key background, copy the selected output into the workspace or `tmp/imagegen/`, run the installed `$CODEX_HOME/skills/.system/imagegen/scripts/remove_chroma_key.py` helper, and validate the alpha result before using it. If this path looks unsuitable or fails, ask before switching to CLI `gpt-image-1.5`.+11. For transparent-output requests, ask built-in `image_gen` for a transparent background and preserve the generated alpha channel.12. Inspect outputs and validate: subject, style, composition, text accuracy, and invariants/avoid items.13. Iterate with a single targeted change, then re-check.14. For preview-only work, render the image inline; the underlying file may remain at the default `$CODEX_HOME/generated_images/...` path.@@ -118,43 +114,7 @@ Assume the user wants a new image unless they clearly ask to change an existing## Transparent image requests-Transparent-image requests still use built-in `image_gen` first. Because the built-in tool does not expose a true transparent-background control, create a removable chroma-key source image and then convert the key color to alpha locally.--Default sequence:-1. Use built-in `image_gen` to generate the requested subject on a perfectly flat solid chroma-key background.-2. Choose a key color that is unlikely to appear in the subject: default `#00ff00`, use `#ff00ff` for green subjects, and avoid `#0000ff` for blue subjects.-3. After generation, move or copy the selected source image from `$CODEX_HOME/generated_images/...` into the workspace or `tmp/imagegen/`.-4. Run the installed helper path, not a project-relative script path:- ```bash- python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py" \- --input <source> \- --out <final.png> \- --auto-key border \- --soft-matte \- --transparent-threshold 12 \- --opaque-threshold 220 \- --despill- ```-5. Validate that the output has an alpha channel, transparent corners, plausible subject coverage, and no obvious key-color fringe. If a thin fringe remains, retry once with `--edge-contract 1`; use `--edge-feather 0.25` only when the edge is visibly stair-stepped and the subject is not shiny or reflective.-6. Save the final alpha PNG/WebP in the project if the asset is project-bound. Never leave a project-referenced transparent asset only under `$CODEX_HOME/*`.--Prompt transparent requests like this:--```text-Create the requested subject on a perfectly flat solid #00ff00 chroma-key background for background removal.-The background must be one uniform color with no shadows, gradients, texture, reflections, floor plane, or lighting variation.-Keep the subject fully separated from the background with crisp edges and generous padding.-Do not use #00ff00 anywhere in the subject.-No cast shadow, no contact shadow, no reflection, no watermark, and no text unless explicitly requested.-```--Do not automatically use CLI `gpt-image-1.5 --background transparent --output-format png` instead of chroma keying. Ask the user first when the user asks for true/native transparency, when local removal fails validation, or when the requested image is complex: hair, fur, feathers, smoke, glass, liquids, translucent materials, reflective objects, soft shadows, realistic product grounding, or subject colors that conflict with all practical key colors.--Use a concise confirmation like:--```text-This likely needs true native transparency. The default built-in path uses a chroma-key background plus local removal, but true transparency requires the CLI fallback with gpt-image-1.5 because gpt-image-2 does not support background=transparent. It also requires OPENAI_API_KEY. Should I proceed with that CLI fallback?-```+Ask built-in `image_gen` for a genuinely transparent background and preserve its alpha.## Prompt augmentation@@ -202,7 +162,7 @@ Edit:- identity-preserve — try-on, person-in-scene; lock face/body/pose.- precise-object-edit — remove/replace a specific element (including interior swaps).- lighting-weather — time-of-day/season/atmosphere changes only.-- background-extraction — transparent background / clean cutout. Use built-in `image_gen` with chroma-key removal first for simple opaque subjects; ask before using CLI true transparency for complex subjects.+- background-extraction — transparent background / clean cutout. Ask built-in `image_gen` for actual transparency.- style-transfer — apply reference style while changing subject/scene.- compositing — multi-image insert/merge with matched lighting/perspective.- sketch-to-render — drawing/line art to photoreal render.@@ -273,7 +233,7 @@ Constraints: change only the background; keep the product and its edges unchange- If the prompt is generic, add only the extra detail that will materially help.- If the prompt is already detailed, normalize it instead of expanding it.- For CLI fallback only, see `references/cli.md` and `references/image-api.md` for model, `quality`, `input_fidelity`, masks, output format, and output-path guidance.-- For transparent images, use the built-in-first chroma-key workflow unless the request is complex enough to need true CLI transparency; ask before switching to CLI `gpt-image-1.5`.+- For transparent images, ask built-in `image_gen` for actual transparency and preserve its alpha.More principles shared by both modes: `references/prompting.md`.Copy/paste specs shared by both modes: `references/sample-prompts.md`.@@ -285,8 +245,8 @@ Asset-type templates (website assets, game assets, wireframes, logo) are consoliThe fallback CLI defaults to `gpt-image-2`.-- Use `gpt-image-2` for new CLI/API workflows unless the request needs true model-native transparent output.-- If a transparent request may need CLI fallback, ask before using `gpt-image-1.5` unless the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback. Explain that the built-in chroma-key path is the default, but true transparency requires `gpt-image-1.5` because `gpt-image-2` does not support `background=transparent`.+- Use `gpt-image-2` for new CLI/API workflows unless the user confirms a different model.+- CLI `gpt-image-2` does not support `background=transparent`; ask before using `gpt-image-1.5` unless the user explicitly requested that model.- `gpt-image-2` always uses high fidelity for image inputs; do not set `input_fidelity` with this model.- `gpt-image-2` supports `quality` values `low`, `medium`, `high`, and `auto`.- Use `quality low` for fast drafts, thumbnails, and quick iterations. Use `medium`, `high`, or `auto` for final assets, dense text, diagrams, identity-sensitive edits, or high-resolution outputs.@@ -320,7 +280,7 @@ Required Python package:uv pip install openai```-Required for local chroma-key removal and optional downscaling:+Optional for image inspection and downscaling:```bashuv pip install pillow```@@ -352,5 +312,4 @@ If installation is not possible in this environment, tell the user which depende- `references/cli.md`: fallback-only CLI usage via `scripts/image_gen.py`.- `references/image-api.md`: fallback-only API/CLI parameter reference.- `references/codex-network.md`: fallback-only network/sandbox troubleshooting for CLI mode.-- `scripts/image_gen.py`: fallback-only CLI implementation. Do not load or use it unless the user explicitly chooses CLI mode or explicitly confirms a transparent request's true CLI transparency fallback.-- `$CODEX_HOME/skills/.system/imagegen/scripts/remove_chroma_key.py`: local post-processing helper for built-in transparent-image requests.+- `scripts/image_gen.py`: fallback-only CLI implementation. Use only when the user explicitly chooses or confirms CLI mode.diff --git a/codex-rs/skills/src/assets/samples/imagegen/references/cli.md b/codex-rs/skills/src/assets/samples/imagegen/references/cli.mdindex f4a5a63d3d..748c83c745 100644--- a/codex-rs/skills/src/assets/samples/imagegen/references/cli.md+++ b/codex-rs/skills/src/assets/samples/imagegen/references/cli.md@@ -59,7 +59,7 @@ python "$IMAGE_GEN" edit \- Use the bundled CLI directly (`python "$IMAGE_GEN" ...`) after activating the correct environment.- Do **not** create one-off runners (for example `gen_images.py`) unless the user explicitly asks for a custom wrapper.- **Never modify** `scripts/image_gen.py`. If something is missing, ask the user before doing anything else.-- Do not silently downgrade from CLI `gpt-image-2` or built-in `image_gen` to CLI `gpt-image-1.5`; ask first unless the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback.+- Do not silently downgrade from CLI `gpt-image-2` or built-in `image_gen` to CLI `gpt-image-1.5`; ask first unless the user explicitly requested `gpt-image-1.5`.## Defaults- Model: `gpt-image-2`@@ -79,7 +79,7 @@ python "$IMAGE_GEN" edit \- Square images are typically fastest. Use `--size 1024x1024` for quick square drafts.- If the user asks for 4K-style output, use `--size 3840x2160` for landscape or `--size 2160x3840` for portrait.- Do not pass `--input-fidelity` with `gpt-image-2`; this model always uses high fidelity for image inputs.-- Do not use `--background transparent` with `gpt-image-2`; the default transparent-image workflow uses built-in `image_gen` on a flat chroma-key background plus local removal. Use `gpt-image-1.5` only after the user explicitly confirms the true-transparent CLI fallback, unless they already requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback.+- Do not use `--background transparent` with CLI `gpt-image-2`; ask before using `gpt-image-1.5` unless the user explicitly requested that model.Popular `gpt-image-2` sizes:- `1024x1024`@@ -130,7 +130,7 @@ python "$IMAGE_GEN" generate \True transparent fallback request:-Ask for confirmation before using this command unless the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback.+Ask for confirmation before using this command unless the user explicitly requested `gpt-image-1.5`.```bashpython "$IMAGE_GEN" generate \@@ -141,7 +141,7 @@ python "$IMAGE_GEN" generate \--out output/imagegen/product-cutout.png```-When using this path, explain briefly that built-in `image_gen` plus chroma-key removal is the default transparent-image path, but this request needs true model-native transparency. `gpt-image-2` does not support `background=transparent`, so `gpt-image-1.5` is required for this confirmed fallback.+Explain that CLI `gpt-image-2` does not support `background=transparent`, so transparent CLI output requires the confirmed `gpt-image-1.5` fallback.## Quality, input fidelity, and masks (CLI fallback only)These are explicit CLI controls. They are not built-in `image_gen` tool arguments.@@ -239,4 +239,4 @@ Notes:- API parameter quick reference for fallback CLI mode: `references/image-api.md`- Prompt examples shared across both top-level modes: `references/sample-prompts.md`- Network/sandbox notes for fallback CLI mode: `references/codex-network.md`-- Built-in-first transparent image workflow: `SKILL.md` and `$CODEX_HOME/skills/.system/imagegen/scripts/remove_chroma_key.py`+- Built-in-first transparent image workflow: `SKILL.md`diff --git a/codex-rs/skills/src/assets/samples/imagegen/references/image-api.md b/codex-rs/skills/src/assets/samples/imagegen/references/image-api.mdindex db8567de2f..64eff30ee6 100644--- a/codex-rs/skills/src/assets/samples/imagegen/references/image-api.md+++ b/codex-rs/skills/src/assets/samples/imagegen/references/image-api.md@@ -68,7 +68,7 @@ Model-specific note for `input_fidelity`:## Transparent backgrounds-`gpt-image-2` does not currently support the Image API `background=transparent` parameter. The skill's default transparent-image path is built-in `image_gen` with a flat chroma-key background, followed by local alpha extraction with `python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py"`.+`gpt-image-2` does not currently support the Image API `background=transparent` parameter. In explicit CLI/API fallback mode, keep `gpt-image-2` when a flat chroma-key background plus local alpha extraction with `python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py"` is acceptable.Use CLI `gpt-image-1.5` with `background=transparent` and a transparent-capable output format such as `png` or `webp` only after the user explicitly confirms that fallback, unless they already requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback. If the user asks for true/native transparency, the subject is too complex for clean chroma-key removal, or local background removal fails validation, explain the tradeoff and ask before switching.diff --git a/codex-rs/skills/src/assets/samples/imagegen/references/prompting.md b/codex-rs/skills/src/assets/samples/imagegen/references/prompting.mdindex 9d2da42f19..5a82656358 100644--- a/codex-rs/skills/src/assets/samples/imagegen/references/prompting.md+++ b/codex-rs/skills/src/assets/samples/imagegen/references/prompting.md@@ -73,13 +73,7 @@ Do not add:- Prefer one targeted follow-up at a time over rewriting the whole prompt.## Transparent images-- Use built-in `image_gen` first for transparent-image requests. If the subject is clearly too complex for chroma-key removal, explain the fallback and ask before switching to CLI.-- Prompt for a perfectly flat solid chroma-key background, usually `#00ff00`; use `#ff00ff` when the subject is green, and avoid key colors that appear in the subject.-- Explicitly prohibit shadows, gradients, floor planes, reflections, texture, and lighting variation in the background.-- Ask for crisp edges, generous padding, and no use of the key color inside the subject.-- After generation, remove the background locally with `python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py" --input <source> --out <final.png> --auto-key border --soft-matte --transparent-threshold 12 --opaque-threshold 220 --despill` and validate the alpha result before shipping it.-- Use soft matte and despill for antialiased edges; hard tolerance-only removal is mainly for flat pixel-art or exact-color fixtures.-- Use CLI `gpt-image-1.5 --background transparent --output-format png` only after the user explicitly confirms the fallback, or when the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback. Ask first for true/native transparency requests, failed chroma-key validation, or complex transparent subjects such as hair, fur, glass, smoke, liquids, translucent materials, reflective objects, or soft shadows.+- Ask built-in `image_gen` for a genuinely transparent background and preserve its alpha.## Fallback-only execution controls- `quality`, `input_fidelity`, explicit masks, output format, and output paths are fallback-only execution controls.@@ -87,7 +81,7 @@ Do not add:- If the user explicitly chooses CLI fallback, see `references/cli.md` and `references/image-api.md` for those controls.- In CLI fallback mode, `gpt-image-2` is the default. It supports `quality=low|medium|high|auto`; use `low` for fast drafts and thumbnails, and move to `medium`, `high`, or `auto` for final assets.- `gpt-image-2` always uses high fidelity for image inputs, so do not set `input_fidelity` with that model.-- If a transparent request needs true CLI transparency, ask before using `gpt-image-1.5` unless the user already explicitly chose it. Explain that built-in chroma-key removal is the default path, but `gpt-image-2` does not support `background=transparent`.+- CLI `gpt-image-2` does not support `background=transparent`; ask before using `gpt-image-1.5` unless the user explicitly requested that model.- If the user asks for 4K-style output with `gpt-image-2`, use `3840x2160` for landscape or `2160x3840` for portrait.## Use-case tips@@ -109,7 +103,7 @@ Edit:- identity-preserve: Lock identity (face, body, pose, hair, expression); change only the specified elements; match lighting and shadows.- precise-object-edit: Specify exactly what to remove/replace; preserve surrounding texture and lighting; keep everything else unchanged.- lighting-weather: Change only environmental conditions (light, shadows, atmosphere, precipitation); keep geometry, framing, and subject identity.-- background-extraction: For simple opaque subjects, request a clean cutout on a perfectly flat chroma-key background; crisp silhouette; generous padding; no shadows; no halos; preserve label text exactly; no restyling. Ask before using true CLI transparency for complex subjects.+- background-extraction: Request a clean cutout on a genuinely transparent background; preserve fine edges and label text; no halos or restyling.- style-transfer: Specify style cues to preserve (palette, texture, brushwork) and what must change; add `no extra elements` to prevent drift.- compositing: Reference inputs by index; specify what moves where; match lighting, perspective, and scale; keep the base framing unchanged.- sketch-to-render: Preserve layout, proportions, and perspective; choose materials and lighting that support the supplied sketch without adding new elements.diff --git a/codex-rs/skills/src/assets/samples/imagegen/references/sample-prompts.md b/codex-rs/skills/src/assets/samples/imagegen/references/sample-prompts.mdindex d94929555e..3e6b46fd02 100644--- a/codex-rs/skills/src/assets/samples/imagegen/references/sample-prompts.md+++ b/codex-rs/skills/src/assets/samples/imagegen/references/sample-prompts.md@@ -2,7 +2,7 @@These prompt recipes are shared across both top-level modes of the skill:- built-in `image_gen` tool (default)-- `scripts/image_gen.py` CLI fallback for explicit CLI/API/model requests or user-confirmed true-transparent-output fallback requests+- `scripts/image_gen.py` CLI fallback for explicit or user-confirmed CLI/API/model requestsUse these as starting points. They are intentionally complete prompt recipes, not the default amount of augmentation to add to every user request.@@ -13,13 +13,13 @@ When adapting a user's prompt:The labeled lines are prompt scaffolding, not a closed schema. `Asset type` and `Input images` are prompt-only scaffolding; the CLI does not expose them as dedicated flags.-Execution details such as explicit CLI flags, `quality`, `input_fidelity`, masks, output formats, and local output paths depend on mode. Use the built-in tool by default, including simple transparent-image requests. For transparent images, prompt for a flat chroma-key background and remove it locally with `python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py"`; only apply CLI-specific controls when the user explicitly opts into fallback mode or explicitly confirms that the transparent request should use true CLI transparency.+Execution details such as explicit CLI flags, `quality`, `input_fidelity`, masks, output formats, and local output paths depend on mode. Use built-in `image_gen` by default, request transparent backgrounds directly, and preserve the generated alpha; apply CLI-specific controls only when the user chooses or confirms that fallback.CLI model notes:- `gpt-image-2` is the fallback CLI default for new workflows.- `gpt-image-2` supports `quality` values `low`, `medium`, `high`, and `auto`.- For 4K-style `gpt-image-2` output, use `3840x2160` or `2160x3840`.-- If transparent output needs true CLI fallback, ask before using `gpt-image-1.5` unless the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback. Explain that built-in chroma-key removal is the default path, but `gpt-image-2` does not support `background=transparent`.+- CLI `gpt-image-2` does not support `background=transparent`; ask before using `gpt-image-1.5` unless the user explicitly requested that model.- Do not set `input_fidelity` with `gpt-image-2`; image inputs already use high fidelity.For prompting principles (structure, specificity, invariants, iteration), see `references/prompting.md`.@@ -386,17 +386,6 @@ Primary request: make it look like a winter evening with gentle snowfallConstraints: preserve subject identity, geometry, camera angle, and composition; change only lighting, atmosphere, and weather```-### background-extraction-```-Use case: background-extraction-Input images: Image 1: product photo-Primary request: isolate the product on a clean transparent background-Scene/backdrop: perfectly flat solid #00ff00 chroma-key background for local background removal-Constraints: background must be one uniform color with no shadows, gradients, texture, reflections, floor plane, or lighting variation; crisp silhouette; generous padding; no halos or fringing; preserve label text exactly; no restyling; do not use #00ff00 anywhere in the subject-```--Post-process note: after built-in generation, run `python "${CODEX_HOME:-$HOME/.codex}/skills/.system/imagegen/scripts/remove_chroma_key.py" --input <source> --out <final.png> --auto-key border --soft-matte --transparent-threshold 12 --opaque-threshold 220 --despill`. Ask before using CLI `gpt-image-1.5 --background transparent --output-format png` for true/native transparency, failed chroma-key validation, or complex subjects such as hair, fur, glass, smoke, liquids, translucent materials, reflections, or soft shadows, unless the user already explicitly requested `gpt-image-1.5`, `scripts/image_gen.py`, or CLI fallback.-### style-transfer```Use case: style-transfer