The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.
Rust ffmpegfree-video-editormodel-context-protocolnon-linear-editoroffline-firstopen-source-video-editorself-hostedvideo-editingvideo-editing-automationvideo-editing-softwarevideo-editing-toolsvideo-editorvoice-cloningyoutube-automationyoutube-shorts
fix(release): every bundle is launched before it ships, Windows links an ONNX Runtime any x86-64 can run, and the Windows tests can finish
Jareerd11848401b3b
diff --git a/.github/dependabot.yml b/.github/dependabot.ymlnew file mode 100644index 00000000..d070ec43--- /dev/null+++ b/.github/dependabot.yml@@ -0,0 +1,36 @@+# Dependabot keeps two kinds of pin current, weekly, as pull requests that+# CI tests like any other:+#+# - The actions every workflow uses, each pinned to a commit with its tag+# in a comment. Dependabot moves the commit and the comment together.+# - The Rust dependencies in src/Cargo.lock, grouped into one pull request+# a week (security updates still come on their own, as they are found),+# so a week's bumps are reviewed and tested once rather than twenty+# times.+#+# Not covered: the prebuilt libraries in scripts/ci/deps.lock (FFmpeg,+# ONNX Runtime, sherpa-onnx, the packagers, LLVM, Intel's emulator), which+# are bumped by hand as that file says, and the toolchain in+# src/rust-toolchain.toml.+version: 2+updates:+ - package-ecosystem: github-actions+ directory: /+ schedule:+ interval: weekly+ groups:+ actions:+ patterns: ["*"]+ commit-message:+ prefix: ci++ - package-ecosystem: cargo+ directory: /src+ schedule:+ interval: weekly+ open-pull-requests-limit: 5+ groups:+ cargo:+ patterns: ["*"]+ commit-message:+ prefix: builddiff --git a/.github/workflows/build-app.yml b/.github/workflows/build-app.ymlindex 996914bd..5e5299f5 100644--- a/.github/workflows/build-app.yml+++ b/.github/workflows/build-app.yml@@ -30,6 +30,17 @@ name: Build app# set on the repository, and ad-hoc signed when they are not, so a build# from a fork still opens (with Gatekeeper's warning).#+# - Every bundle is launched before it is uploaded: `concat --version` and+# `concat --smoke` from the staged folder, on a machine with nothing else+# on the path (scripts/ci/smoke-*). On x86_64 the bundle is also started+# under Intel's emulator posing as an Ivy Bridge, and its binaries are+# scanned for instructions newer than AVX (scripts/ci/isa-scan.sh): the+# x86_64 baseline is AVX, and twice now a release died at launch on+# older CPUs (#193, #313). A target whose bundle does not start fails,+# and release.yml publishes nothing.+# - Every prebuilt dependency downloaded here is pinned to one file and its+# digest in scripts/ci/deps.lock, and every action to a commit.+## The six targets are a matrix so one fails without taking the rest down;# `fail-fast` is off for the same reason.on:@@ -55,22 +66,37 @@ on:APPLE_APP_PASSWORD:description: An app-specific password for notarytoolrequired: false+ WINDOWS_CERTIFICATE_PFX:+ description: Base64 of an Authenticode code-signing .pfx+ required: false+ WINDOWS_CERTIFICATE_PASSWORD:+ description: The .pfx's password+ required: falseenv:CARGO_TERM_COLOR: always- # BtbN's autobuilds: GPL (x264 in), shared, FFmpeg 8.1, one per arch.- FFMPEG_RELEASE: https://github.com/BtbN/FFmpeg-Builds/releases/download/latest- FFMPEG_VERSION: 8.1- # ONNX Runtime, for the Linux bundles: the version the ort crate is built- # against (see ort-sys's dist table), as Microsoft's own shared build.+ # The prebuilt dependencies - BtbN's FFmpeg 8.1 (GPL, shared, one per+ # arch), ONNX Runtime, sherpa-onnx, nfpm, appimagetool, LLVM for Windows+ # and Intel's emulator - are each pinned to one file and its digest in+ # scripts/ci/deps.lock, and fetched through scripts/ci/fetch.sh, which+ # refuses a file whose digest has changed. The versions below are the+ # ones those files carry, for the names inside them; a bump changes both.+ #+ # ONNX Runtime, for the Linux and Windows bundles: the version the ort+ # crate is built against (see ort-sys's dist table), as Microsoft's own+ # shared build.ORT_VERSION: 1.28.0# And for Intel Macs, which neither ort nor Microsoft build for any more:- # the last release Microsoft made for them. The ort crate asks for the- # runtime's API 17 and up, which this carries.+ # the last release Microsoft made for them.+ #+ # NOTE: ort 2.0.0-rc.13 with its default features asks the runtime for+ # API 27 (its `api-27` feature), and 1.23.2 serves API 23 and below, so+ # on an Intel Mac the first ONNX session (a cutout, Chatterbox) should+ # fail to get the API. That predates the pinning here, and wants a look+ # on an Intel Mac: either an older `api-*` feature for ort, or no+ # cutouts there.ORT_INTEL_MAC_VERSION: 1.23.2- # The Linux packagers: nfpm for the .deb and .rpm, appimagetool for the- # AppImage; and WiX for the Windows .msi.- NFPM_VERSION: 2.47.0+ # WiX for the Windows .msi, as a dotnet tool, pinned by version.WIX_VERSION: 6.0.2# Oldest macOS the bundles run on. Apple silicon Macs all run newer than# this; it is the Intel floor.@@ -139,6 +165,9 @@ jobs:target: x86_64-pc-windows-msvcffmpeg: win64sherpa: win-x64+ ort: win-x64+ llvm: win64+ vc: x64inno: x64compatiblewix: x64- platform: windows@@ -147,16 +176,22 @@ jobs:target: aarch64-pc-windows-msvcffmpeg: winarm64sherpa: win-arm64+ ort: win-arm64+ llvm: woa64+ vc: arm64inno: arm64wix: arm64runs-on: ${{ matrix.os }}- # The slowest target (Intel macOS, fat LTO) finishes in under half an- # hour. A job still running past this is stuck, not slow.- timeout-minutes: 90+ # The slowest target (Intel macOS, fat LTO) builds in under half an+ # hour, and the checks after the build add a few minutes. The steps+ # that can hang have timeouts of their own, at most 115 minutes+ # between them on any one target, so a stuck step is named by its own+ # timeout; this is for a job stuck anywhere else.+ timeout-minutes: 150steps:# The ref release.yml names, which is its tag: without one a run# started by hand builds main and publishes it under the tag's name.- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}# The toolchain src/rust-toolchain.toml pins, installed by name, so the@@ -165,7 +200,7 @@ jobs:id: toolchainshell: bashrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}@@ -194,8 +229,7 @@ jobs:if: matrix.platform == 'linux'run: |set -euo pipefail- url="$FFMPEG_RELEASE/ffmpeg-n$FFMPEG_VERSION-latest-${{ matrix.ffmpeg }}-gpl-shared-$FFMPEG_VERSION.tar.xz"- curl -fsSL --retry 5 --retry-all-errors -o ffmpeg.tar.xz "$url"+ scripts/ci/fetch.sh "ffmpeg-${{ matrix.ffmpeg }}" ffmpeg.tar.xzmkdir -p ffmpeg-devtar -xf ffmpeg.tar.xz -C ffmpeg-dev --strip-components=1{@@ -223,8 +257,7 @@ jobs:run: |set -euo pipefailstem="onnxruntime-linux-${{ matrix.ort }}-$ORT_VERSION"- curl -fsSL --retry 5 --retry-all-errors -o onnxruntime.tgz \- "https://github.com/microsoft/onnxruntime/releases/download/v$ORT_VERSION/$stem.tgz"+ scripts/ci/fetch.sh "onnxruntime-linux-${{ matrix.ort }}" onnxruntime.tgztar -xzf onnxruntime.tgztest -f "$stem/lib/libonnxruntime.so.$ORT_VERSION"# The tarball carries the soname and the full version; the linker@@ -267,8 +300,7 @@ jobs:run: |set -euo pipefailstem="onnxruntime-osx-x86_64-$ORT_INTEL_MAC_VERSION"- curl -fsSL --retry 5 --retry-all-errors -o onnxruntime.tgz \- "https://github.com/microsoft/onnxruntime/releases/download/v$ORT_INTEL_MAC_VERSION/$stem.tgz"+ scripts/ci/fetch.sh onnxruntime-osx-x86_64 onnxruntime.tgztar -xzf onnxruntime.tgztest -f "$stem/lib/libonnxruntime.$ORT_INTEL_MAC_VERSION.dylib"ln -sf "libonnxruntime.$ORT_INTEL_MAC_VERSION.dylib" "$stem/lib/libonnxruntime.dylib"@@ -279,10 +311,12 @@ jobs:} >> "$GITHUB_ENV"# ── Windows ────────────────────────────────────────────────────────- # LLVM: libclang for bindgen on both architectures, and the C/C++- # compiler on arm64. Chocolatey's LLVM is x64; the arm64 runner takes- # LLVM's own Windows-on-Arm installer. The BtbN zip unpacks to one- # top-level folder.+ # LLVM: libclang for bindgen on both architectures, the C/C+++ # compiler on arm64, and llvm-objdump for the instruction-set scan.+ # LLVM's own installer for the architecture, at the version deps.lock+ # pins, into a folder of its own rather than over the LLVM the runner+ # image may already carry. The BtbN zip unpacks to one top-level+ # folder.## On arm64 the whole C side is compiled by clang-cl rather than cl:# ggml refuses MSVC for ARM outright (its NEON kernels need clang),@@ -299,31 +333,83 @@ jobs:# crate into those flags - spelt with a dash, because the build# step runs under Git's bash, whose MSYS layer rewrites a value# that starts with a slash into a Windows path.- - name: FFmpeg development libraries (Windows)+ #+ # fetch.sh runs under Git's bash, named by its full path: a bare+ # `bash` from PowerShell can be WSL's.+ - name: FFmpeg development libraries and LLVM (Windows)if: matrix.platform == 'windows'+ timeout-minutes: 15shell: pwshrun: |$ErrorActionPreference = "Stop"+ $bash = "$env:ProgramFiles\Git\bin\bash.exe"+ & $bash scripts/ci/fetch.sh "llvm-${{ matrix.llvm }}" llvm.exe+ if ($LASTEXITCODE -ne 0) { throw "the LLVM download failed" }+ Start-Process -Wait -FilePath .\llvm.exe -ArgumentList "/S", "/D=C:\llvm"+ if (-not (Test-Path C:\llvm\bin\libclang.dll)) { throw "LLVM did not install to C:\llvm" }+ Remove-Item llvm.exeif ("${{ matrix.arch }}" -eq "aarch64") {- $llvm = "21.1.0"- curl.exe -fsSL --retry 5 --retry-all-errors -o llvm.exe "https://github.com/llvm/llvm-project/releases/download/llvmorg-$llvm/LLVM-$llvm-woa64.exe"- Start-Process -Wait -FilePath .\llvm.exe -ArgumentList "/S""CC=clang-cl" | Out-File -FilePath $env:GITHUB_ENV -Append"CXX=clang-cl" | Out-File -FilePath $env:GITHUB_ENV -Append"CXXFLAGS=-EHsc" | Out-File -FilePath $env:GITHUB_ENV -Append"CMAKE_GENERATOR=Ninja" | Out-File -FilePath $env:GITHUB_ENV -Append- } else {- choco install llvm -y --no-progress}- "C:\Program Files\LLVM\bin" | Out-File -FilePath $env:GITHUB_PATH -Append- "LIBCLANG_PATH=C:\Program Files\LLVM\bin" | Out-File -FilePath $env:GITHUB_ENV -Append- $url = "$env:FFMPEG_RELEASE/ffmpeg-n$env:FFMPEG_VERSION-latest-${{ matrix.ffmpeg }}-gpl-shared-$env:FFMPEG_VERSION.zip"- curl.exe -fsSL --retry 5 --retry-all-errors -o ffmpeg-win.zip "$url"+ "C:\llvm\bin" | Out-File -FilePath $env:GITHUB_PATH -Append+ "LIBCLANG_PATH=C:\llvm\bin" | Out-File -FilePath $env:GITHUB_ENV -Append+ & $bash scripts/ci/fetch.sh "ffmpeg-${{ matrix.ffmpeg }}" ffmpeg-win.zip+ if ($LASTEXITCODE -ne 0) { throw "the FFmpeg download failed" }Expand-Archive ffmpeg-win.zip -DestinationPath ffdl$dir = (Get-ChildItem ffdl -Directory | Select-Object -First 1).FullName"FFMPEG_DIR=$dir" | Out-File -FilePath $env:GITHUB_ENV -Append"$dir\bin" | Out-File -FilePath $env:GITHUB_PATH -Append+ # ONNX Runtime on Windows: Microsoft's shared build, linked+ # dynamically, and onnxruntime.dll shipped beside concat.exe.+ #+ # Left to itself ort-sys links pyke's static build into concat.exe,+ # and that build is compiled for Haswell from end to end: BMI2+ # shifts in ordinary code, run at start-up, so 0.2.5 and 0.2.6 died+ # with STATUS_ILLEGAL_INSTRUCTION on every CPU before 2013 (#313).+ # Microsoft's build keeps its general code to the baseline and picks+ # AVX2 kernels at run time. pyke's is also the DirectML build, which+ # imports DirectML.dll when the program loads - and the one Windows+ # keeps in System32 is too old on many machines, so the program would+ # not start at all ("DMLCreateDevice1 not found", #303).+ #+ # Why the CPU build and not Microsoft's DirectML one: two runtimes+ # cannot share a process under one name, and sherpa-onnx's DLL loads+ # onnxruntime.dll too. sherpa-onnx 1.13.7 is built against ONNX+ # Runtime 1.27.1 and asks for its API 27 as it loads; ort 2.0.0-rc.13+ # asks for API 27 as well (its default `api-27` feature). Microsoft+ # stopped publishing the DirectML build at 1.24.4, which serves API+ # 24 and below, so it can serve neither. 1.28.0 serves both. The cost+ # is that the cutouts and Chatterbox run on the CPU on Windows: the+ # DirectML provider ort asks for is not in this runtime, and ort+ # carries on without it (it logs the failed registration and falls+ # back; error_on_failure is not set). Getting the GPU back means a+ # DirectML (or WebGPU) runtime at API 27 or later that is built for+ # the baseline - pyke's is neither the one nor shareable with+ # sherpa-onnx - and no DirectML.dll is shipped until then, since+ # nothing in the bundle imports it.+ #+ # The DLL's folder goes on the path too, so anything the build runs+ # that links it (a build script's helper, say) finds it.+ - name: ONNX Runtime (Windows)+ if: matrix.platform == 'windows'+ shell: pwsh+ run: |+ $ErrorActionPreference = "Stop"+ & "$env:ProgramFiles\Git\bin\bash.exe" scripts/ci/fetch.sh "onnxruntime-${{ matrix.ort }}" onnxruntime.zip+ if ($LASTEXITCODE -ne 0) { throw "the ONNX Runtime download failed" }+ Expand-Archive onnxruntime.zip -DestinationPath .+ $dir = "$PWD\onnxruntime-${{ matrix.ort }}-$env:ORT_VERSION"+ if (-not (Test-Path "$dir\lib\onnxruntime.lib")) { throw "$dir\lib\onnxruntime.lib is not there" }+ if (-not (Test-Path "$dir\lib\onnxruntime.dll")) { throw "$dir\lib\onnxruntime.dll is not there" }+ "ORT_LIB_LOCATION=$dir\lib" | Out-File -FilePath $env:GITHUB_ENV -Append+ "ORT_PREFER_DYNAMIC_LINK=1" | Out-File -FilePath $env:GITHUB_ENV -Append+ "ORT_DIR=$dir" | Out-File -FilePath $env:GITHUB_ENV -Append+ "$dir\lib" | Out-File -FilePath $env:GITHUB_PATH -Append+# ── sherpa-onnx ────────────────────────────────────────────────────# The prebuilt library the sys crate would otherwise fetch for# itself, fetched here instead and handed over through@@ -332,17 +418,22 @@ jobs:# cache then replays the build script's answer without the files# it points at, and the crate fails to link the moment anything# makes it recompile. Outside target/, the files are there on every- # run. The version is the one the lockfile resolved, so a bump- # cannot desynchronise.+ # run. The version is the one the lockfile resolved, and deps.lock+ # must pin that version's file: a bump to one without the other+ # stops here, saying so.- name: sherpa-onnx static libraries (macOS and Linux)if: matrix.platform != 'windows'shell: bashrun: |set -euo pipefailversion=$(grep -A1 'name = "sherpa-onnx-sys"' src/Cargo.lock | sed -n 's/^version = "\(.*\)"/\1/p')+ name="sherpa-onnx-${{ matrix.sherpa }}-static-lib"+ case "$(scripts/ci/fetch.sh --url "$name")" in+ */v"$version"/*) ;;+ *) echo "scripts/ci/deps.lock pins $name for another version than Cargo.lock's $version" >&2; exit 1 ;;+ esacstem="sherpa-onnx-v$version-${{ matrix.sherpa }}-static-lib"- echo "Downloading $stem"- curl -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"+ scripts/ci/fetch.sh "$name" sherpa.tar.bz2mkdir -p sherpatar -xjf sherpa.tar.bz2 -C sherpatest -d "sherpa/$stem/lib"@@ -358,6 +449,10 @@ jobs:# DLLs are kept apart and the packaging step ships them beside the# binary.#+ # sherpa-onnx's own onnxruntime.dll and its import library are left+ # out: the bundle carries one ONNX Runtime, the one above, which+ # serves sherpa-onnx and ort alike (see that step).+ ## The archive is .tar.bz2, and Windows' own tar has no bzip2 of its# own: it pipes through an external decoder, and on the x64 image# that pipe can hang without a word. Python's tarfile unpacks it@@ -369,29 +464,34 @@ jobs:run: |$ErrorActionPreference = "Stop"$version = (Select-String -Path src\Cargo.lock -Pattern 'name = "sherpa-onnx-sys"' -Context 0,1).Context.PostContext[0] -replace '.*"(.*)".*', '$1'+ $name = "sherpa-onnx-${{ matrix.sherpa }}-shared-MT-Release-lib"$stem = "sherpa-onnx-v$version-${{ matrix.sherpa }}-shared-MT-Release-lib"+ $bash = "$env:ProgramFiles\Git\bin\bash.exe"+ $url = & $bash scripts/ci/fetch.sh --url $name+ if ($url -notmatch "/v$([regex]::Escape($version))/") { throw "scripts/ci/deps.lock pins $name for another version than Cargo.lock's $version" }Write-Host "Downloading $stem"- curl.exe -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"+ & $bash scripts/ci/fetch.sh $name sherpa.tar.bz2+ if ($LASTEXITCODE -ne 0) { throw "the sherpa-onnx download failed" }Write-Host "Unpacking"python -c "import tarfile; t = tarfile.open('sherpa.tar.bz2'); t.extractall(**({'filter': 'data'} if hasattr(tarfile, 'data_filter') else {}))"if (-not (Test-Path "$stem\lib")) { throw "$stem\lib is not there after unpacking" }New-Item -ItemType Directory -Force sherpa-libs, sherpa-dlls | Out-Null- Copy-Item "$stem\lib\*.lib" sherpa-libs\- Copy-Item "$stem\lib\*.dll" sherpa-dlls\+ Copy-Item "$stem\lib\sherpa-onnx-*.lib" sherpa-libs\+ Copy-Item "$stem\lib\sherpa-onnx-*.dll" sherpa-dlls\"SHERPA_ONNX_LIB_DIR=$PWD\sherpa-libs" | Out-File -FilePath $env:GITHUB_ENV -Append"SHERPA_DLL_DIR=$PWD\sherpa-dlls" | Out-File -FilePath $env:GITHUB_ENV -Append# ── Build ──────────────────────────────────────────────────────────# The cache comes after the environment is set, not before: its key# is made from the compiler and linker variables in force (RUSTFLAGS,- # CC, CXX, CMAKE_*, and GGML_* added to the default set), so a change- # to any of them is a new key rather than a stale target/ served- # under the old one.- - uses: Swatinem/rust-cache@v2+ # CC, CXX, CMAKE_*, and GGML_* and ORT_* added to the default set),+ # so a change to any of them is a new key rather than a stale+ # target/ served under the old one.+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: srckey: ${{ matrix.target }}- env-vars: CARGO CC CFLAGS CXX CMAKE RUST GGML+ env-vars: CARGO CC CFLAGS CXX CMAKE RUST GGML ORT# The shipping profile: fat LTO, panic=abort, stripped. See the# workspace Cargo.toml for what each knob costs.@@ -400,6 +500,7 @@ jobs:# read, but the step summary is public - so on failure the tail of the# build goes there, and a red run explains itself to anyone.- name: Build+ timeout-minutes: 60shell: bashworking-directory: srcrun: |@@ -544,6 +645,14 @@ jobs:fimkdir -p stage && mv "$dmg" stage/+ # The bundle that went into the .dmg, launched: its signature holds,+ # nothing in it loads from Homebrew, and `--version` and `--smoke`+ # exit 0. See scripts/ci/smoke-macos.sh.+ - name: Smoke test (macOS)+ if: matrix.platform == 'macos'+ timeout-minutes: 10+ run: scripts/ci/smoke-macos.sh Concat.app+# ── Package: Linux ─────────────────────────────────────────────────# The binary, the FFmpeg runtime libraries it loads from lib/ beside# it (the rpath above), a desktop entry and an icon, in one folder.@@ -587,6 +696,42 @@ jobs:exit 1fi+ # The staged folder, launched on this machine with nothing but the+ # system and the folder to load from: `--version`, then `--smoke`+ # under Xvfb with every file it opens traced, so a start-up that so+ # much as reaches for CUDA or NVENC fails. On x86_64, `--version`+ # again under Intel's emulator posing as an Ivy Bridge. See+ # scripts/ci/smoke-linux.sh.+ - name: Smoke test (Linux)+ if: matrix.platform == 'linux'+ timeout-minutes: 15+ env:+ VERSION: ${{ steps.version.outputs.version }}+ run: |+ set -euo pipefail+ sudo apt-get install -y xvfb xauth strace libgl1-mesa-dri mesa-vulkan-drivers+ args=()+ if [ "${{ matrix.arch }}" = x86_64 ]; then+ scripts/ci/fetch.sh sde-lin sde.tar.xz+ mkdir -p sde+ tar -xJf sde.tar.xz -C sde --strip-components=1+ args+=(--sde "$PWD/sde/sde64")+ fi+ scripts/ci/smoke-linux.sh "${args[@]}" "stage/Concat-${VERSION}-linux-${{ matrix.arch }}"++ # Every x86_64 binary in the folder, disassembled and counted for+ # instructions an Ivy Bridge does not have, against what+ # scripts/ci/isa-allow.txt allows each one. See scripts/ci/isa-scan.sh.+ - name: Instruction-set scan (Linux x86_64)+ if: matrix.platform == 'linux' && matrix.arch == 'x86_64'+ timeout-minutes: 15+ env:+ VERSION: ${{ steps.version.outputs.version }}+ run: |+ set -euo pipefail+ sudo apt-get install -y llvm+ scripts/ci/isa-scan.sh "stage/Concat-${VERSION}-linux-${{ matrix.arch }}"+# Arch Linux package, from the same folder as the .deb and .rpm.# Arch ships x86_64 only, so aarch64 comes from Arch Linux ARM.- name: Package (Arch Linux)@@ -653,8 +798,7 @@ jobs:chmod +x "$STAGE/pkg/concat"sed 's|^Exec=concat|Exec=/opt/concat/concat %U|' "$STAGE/concat.desktop" > "$STAGE/pkg/concat.desktop"- curl -fsSL --retry 5 --retry-all-errors -o nfpm.tar.gz \- "https://github.com/goreleaser/nfpm/releases/download/v$NFPM_VERSION/nfpm_${NFPM_VERSION}_Linux_$( [ "${{ matrix.arch }}" = x86_64 ] && echo x86_64 || echo arm64 ).tar.gz"+ scripts/ci/fetch.sh "nfpm-$( [ "${{ matrix.arch }}" = x86_64 ] && echo x86_64 || echo arm64 )" nfpm.tar.gzmkdir -p nfpm && tar -xzf nfpm.tar.gz -C nfpm# From inside the folder: nfpm names files relative to where it# runs, and expands no variables in their paths.@@ -684,31 +828,179 @@ jobs:cp "$STAGE/concat.png" "$appdir/usr/share/icons/hicolor/256x256/apps/concat.png"ln -s concat.png "$appdir/.DirIcon"ln -s usr/bin/concat "$appdir/AppRun"- curl -fsSL --retry 5 --retry-all-errors -o appimagetool \- "https://github.com/AppImage/appimagetool/releases/download/continuous/appimagetool-$ARCH.AppImage"+ # The tool and the runtime it puts at the head of the AppImage,+ # both a named release: left to itself the tool downloads the+ # runtime's `continuous` build as it runs.+ scripts/ci/fetch.sh "appimagetool-$ARCH" appimagetool+ scripts/ci/fetch.sh "appimage-runtime-$ARCH" appimage-runtimechmod +x appimagetool# Extract-and-run: the runner has no FUSE to mount the tool with.# No "linux" in its name: every AppImage is for Linux, and the# AppImage catalog's check warns about names that say so.appimage="stage/Concat-${VERSION}-${{ matrix.arch }}.AppImage"- ./appimagetool --appimage-extract-and-run "$appdir" "$appimage"+ ./appimagetool --appimage-extract-and-run --runtime-file appimage-runtime "$appdir" "$appimage"chmod +x "$appimage"rm -r "$STAGE" "$appdir"ls -l stage+ # The .deb and the AppImage in two clean distributions, the oldest+ # and a Debian: apt installs the .deb with exactly the dependencies it+ # declares, and both start (`--version`, and `--smoke` under Xvfb).+ # A library the package forgot to depend on fails here rather than+ # on somebody's machine. See scripts/ci/smoke-linux.sh.+ - name: Smoke test in clean distributions (Linux)+ if: matrix.platform == 'linux'+ timeout-minutes: 25+ env:+ VERSION: ${{ steps.version.outputs.version }}+ run: |+ set -euo pipefail+ for image in ubuntu:22.04 debian:12; do+ scripts/ci/smoke-linux.sh --container "$image" \+ "stage/Concat-${VERSION}-linux-${{ matrix.arch }}.deb" \+ "stage/Concat-${VERSION}-${{ matrix.arch }}.AppImage"+ done+# ── Package: Windows ───────────────────────────────────────────────- # Authenticode-signed with the repository's certificate when it has- # one - the program, then each installer - so SmartScreen and the- # UAC prompt name the publisher instead of warning about an unknown- # one; unsigned otherwise, which is what a fork's build gets.+ # The folder the installers are made from: the program, FFmpeg's+ # DLLs, sherpa-onnx's, ONNX Runtime's (one runtime for both; see its+ # step above) and the Visual C++ runtime the program and its DLLs+ # import. The C++ runtime is copied in from Visual Studio's+ # redistributable folder, as Microsoft allows: a clean Windows does+ # not have it, and a machine that does may have an older one.- name: Stage (Windows)+ if: matrix.platform == 'windows'+ shell: pwsh+ env:+ VERSION: ${{ steps.version.outputs.version }}+ run: |+ $ErrorActionPreference = "Stop"+ $name = "Concat-$env:VERSION-windows-${{ matrix.arch }}"+ $dir = "stage\$name"+ New-Item -ItemType Directory -Force $dir | Out-Null+ Copy-Item "src\target\${{ matrix.target }}\app\concat.exe" $dir+ Copy-Item "$env:FFMPEG_DIR\bin\*.dll" $dir+ # The sherpa-onnx runtime, set aside by the download step above.+ Copy-Item "$env:SHERPA_DLL_DIR\*.dll" $dir+ Copy-Item "$env:ORT_DIR\lib\*.dll" $dir+ Copy-Item LICENSE, THIRD_PARTY_NOTICES.md $dir+ # The Visual C++ runtime DLLs anything in the folder imports, from+ # the newest redistributable folder Visual Studio carries.+ $vs = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -property installationPath+ $crt = Get-ChildItem "$vs\VC\Redist\MSVC\*\${{ matrix.vc }}\Microsoft.VC14*.CRT" -Directory |+ Sort-Object FullName -Descending | Select-Object -First 1+ if (-not $crt) { throw "no Visual C++ redistributable folder for ${{ matrix.vc }} under $vs" }+ $dumpbin = & "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" -latest -products * -find "VC\Tools\MSVC\**\bin\Host*\*\dumpbin.exe" | Select-Object -First 1+ # Twice over: msvcp140.dll, once copied, imports vcruntime140.dll.+ foreach ($pass in 1, 2) {+ foreach ($pe in Get-ChildItem $dir -File -Include *.exe, *.dll -Recurse) {+ foreach ($line in & $dumpbin /nologo /dependents $pe.FullName) {+ $dll = $line.Trim()+ if ($dll -match "\.dll$" -and (Test-Path "$($crt.FullName)\$dll") -and -not (Test-Path "$dir\$dll")) {+ Write-Host "$($pe.Name) imports $dll; copying it from $($crt.FullName)"+ Copy-Item "$($crt.FullName)\$dll" $dir+ }+ }+ }+ }+ Get-ChildItem $dir | Format-Table Name, Length++ # Authenticode, with the repository's certificate when it has one:+ # every program and DLL in the folder that does not already carry a+ # valid signature (Microsoft's ONNX Runtime does, and keeps it), here,+ # and the installers in the step after the checks. Unsigned files+ # are what Smart App Control refuses to load ("Bad Image", 0xc0e90002,+ # #306, #311), and an unsigned DLL is refused even beside a signed+ # program. SmartScreen and the UAC prompt name the publisher instead+ # of warning about an unknown one. A fork's build is unsigned.+ #+ # The certificate is imported into the user's store and signtool is+ # told its thumbprint, so the password never reaches a command line:+ # it goes from the environment to the import as a SecureString. The+ # store is emptied of it again in the last step, whatever happens.+ - name: Sign (Windows)if: matrix.platform == 'windows'shell: pwshenv:VERSION: ${{ steps.version.outputs.version }}CERT_PFX: ${{ secrets.WINDOWS_CERTIFICATE_PFX }}CERT_PASSWORD: ${{ secrets.WINDOWS_CERTIFICATE_PASSWORD }}+ run: |+ $ErrorActionPreference = "Stop"+ if (-not $env:CERT_PFX) {+ Write-Host "No signing certificate: the build is unsigned."+ exit 0+ }+ $pfx = "$env:RUNNER_TEMP\cert.pfx"+ [IO.File]::WriteAllBytes($pfx, [Convert]::FromBase64String($env:CERT_PFX))+ $password = ConvertTo-SecureString $env:CERT_PASSWORD -AsPlainText -Force+ $flags = [Security.Cryptography.X509Certificates.X509KeyStorageFlags]"UserKeySet, PersistKeySet"+ $cert = [Security.Cryptography.X509Certificates.X509Certificate2]::new($pfx, $password, $flags)+ Remove-Item -Force $pfx+ if (-not $cert.HasPrivateKey) { throw "the certificate came in without its private key" }+ $store = [Security.Cryptography.X509Certificates.X509Store]::new("My", "CurrentUser")+ $store.Open("ReadWrite")+ $store.Add($cert)+ $store.Close()+ $signtool = Get-ChildItem "${env:ProgramFiles(x86)}\Windows Kits\10\bin\*\x64\signtool.exe" |+ Sort-Object FullName -Descending | Select-Object -First 1+ if (-not $signtool) { throw "signtool.exe is not in the Windows SDK" }+ "SIGNTOOL=$($signtool.FullName)" | Out-File -FilePath $env:GITHUB_ENV -Append+ "SIGNING_THUMBPRINT=$($cert.Thumbprint)" | Out-File -FilePath $env:GITHUB_ENV -Append+ $dir = "stage\Concat-$env:VERSION-windows-${{ matrix.arch }}"+ $unsigned = Get-ChildItem $dir -Recurse -File -Include *.exe, *.dll |+ Where-Object { (Get-AuthenticodeSignature $_.FullName).Status -ne "Valid" }+ foreach ($file in $unsigned) {+ & $signtool.FullName sign /sha1 $cert.Thumbprint /s My /fd SHA256 `+ /tr http://timestamp.digicert.com /td SHA256 /d Concat $file.FullName+ if ($LASTEXITCODE -ne 0) { throw "signtool failed on $($file.Name)" }+ }+ Write-Host "Signed $(@($unsigned).Count) files."++ # The staged folder, launched before any installer is made of it:+ # every DLL it imports is in it or is Windows' own; `--version` and+ # `--smoke` exit 0 with only Windows on the path; and on x86_64,+ # `--version` again under Intel's emulator posing as an Ivy Bridge+ # (#313). See scripts/ci/smoke-windows.ps1.+ - name: Smoke test (Windows)+ if: matrix.platform == 'windows'+ timeout-minutes: 15+ shell: pwsh+ env:+ VERSION: ${{ steps.version.outputs.version }}+ run: |+ $ErrorActionPreference = "Stop"+ $sde = ""+ if ("${{ matrix.arch }}" -eq "x86_64") {+ & "$env:ProgramFiles\Git\bin\bash.exe" scripts/ci/fetch.sh sde-win sde.tar.xz+ if ($LASTEXITCODE -ne 0) { throw "the SDE download failed" }+ New-Item -ItemType Directory -Force sde | Out-Null+ tar -xf sde.tar.xz -C sde --strip-components=1+ if ($LASTEXITCODE -ne 0) { throw "unpacking SDE failed" }+ $sde = "sde\sde.exe"+ }+ scripts/ci/smoke-windows.ps1 -Bundle "stage\Concat-$env:VERSION-windows-${{ matrix.arch }}" -Sde $sde++ # Every x86_64 binary in the folder, disassembled and counted for+ # instructions an Ivy Bridge does not have, against what+ # scripts/ci/isa-allow.txt allows each one. See scripts/ci/isa-scan.sh.+ - name: Instruction-set scan (Windows x86_64)+ if: matrix.platform == 'windows' && matrix.arch == 'x86_64'+ timeout-minutes: 15+ shell: bash+ env:+ VERSION: ${{ steps.version.outputs.version }}+ # llvm-objdump from the LLVM installed above, on the path.+ run: scripts/ci/isa-scan.sh "stage/Concat-${VERSION}-windows-${{ matrix.arch }}"++ # The installers, from the checked folder: Inno Setup's .exe and+ # WiX's .msi, each signed when the folder was.+ - name: Installers (Windows)+ if: matrix.platform == 'windows'+ shell: pwsh+ env:+ VERSION: ${{ steps.version.outputs.version }}# The identity Partner Center reserved for the app. Unset, no# .msix is made: a fork has no Store listing to make one for.MSSTORE_IDENTITY_NAME: ${{ vars.MSSTORE_IDENTITY_NAME }}@@ -718,29 +1010,12 @@ jobs:run: |$ErrorActionPreference = "Stop"$name = "Concat-$env:VERSION-windows-${{ matrix.arch }}"- $pfx = $null- if ($env:CERT_PFX) {- $pfx = "$env:RUNNER_TEMP\cert.pfx"- [IO.File]::WriteAllBytes($pfx, [Convert]::FromBase64String($env:CERT_PFX))- $signtool = Get-ChildItem "${env:ProgramFiles(x86)}\Windows Kits\10\bin\*\x64\signtool.exe" |- Sort-Object FullName -Descending | Select-Object -First 1- if (-not $signtool) { throw "signtool.exe is not in the Windows SDK" }- } else {- Write-Host "No signing certificate: the build is unsigned."- }function Sign($file) {- if (-not $pfx) { return }- & $signtool.FullName sign /f $pfx /p $env:CERT_PASSWORD /fd SHA256 `+ if (-not $env:SIGNING_THUMBPRINT) { return }+ & $env:SIGNTOOL sign /sha1 $env:SIGNING_THUMBPRINT /s My /fd SHA256 `/tr http://timestamp.digicert.com /td SHA256 /d Concat $fileif ($LASTEXITCODE -ne 0) { throw "signtool failed on $file" }}- New-Item -ItemType Directory -Force "stage\$name" | Out-Null- Copy-Item "src\target\${{ matrix.target }}\app\concat.exe" "stage\$name\"- Sign "stage\$name\concat.exe"- Copy-Item "$env:FFMPEG_DIR\bin\*.dll" "stage\$name\"- # The sherpa-onnx runtime, set aside by the download step above.- Copy-Item "$env:SHERPA_DLL_DIR\*.dll" "stage\$name\"- Copy-Item LICENSE, THIRD_PARTY_NOTICES.md "stage\$name\"# The installer, from that folder: Inno Setup, which the x64# image carries and the ARM image is given. See# assets/windows/concat.iss.@@ -765,7 +1040,6 @@ jobs:if ($LASTEXITCODE -ne 0) { throw "WiX failed with $LASTEXITCODE" }if (-not (Test-Path "stage\$name.msi")) { throw "stage\$name.msi is not there" }Sign "stage\$name.msi"- if ($pfx) { Remove-Item -Force $pfx }# And the .msix, for the Microsoft Store, from the same folder with# the manifest and logos beside it. See assets/windows/AppxManifest.xml.# Into store/, not stage/: the Store signs it and takes it, and an@@ -803,7 +1077,14 @@ jobs:# The folder itself is not published: the two installers are.Remove-Item -Recurse -Force "stage\$name"- - uses: actions/upload-artifact@v7+ # The signing certificate out of the user's store again, whether the+ # steps above passed or not.+ - name: Forget the certificate (Windows)+ if: always() && matrix.platform == 'windows' && env.SIGNING_THUMBPRINT != ''+ shell: pwsh+ run: Remove-Item -Force -DeleteKey "Cert:\CurrentUser\My\$env:SIGNING_THUMBPRINT"++ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2with:name: Concat-${{ matrix.platform }}-${{ matrix.arch }}path: stage/*@@ -811,7 +1092,7 @@ jobs:# The Store's package, apart from the release's bundles: release.yml# bundles the two architectures and submits them.- - uses: actions/upload-artifact@v7+ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2if: matrix.platform == 'windows' && hashFiles('store/*.msix') != ''with:name: Store-windows-${{ matrix.arch }}diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 7284a1af..7ac73be7 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -2,6 +2,12 @@ name: CI# The engine's tests and lints on every push and pull request. The bundles# are built by release.yml, on a tag; see that file.+#+# Every action is pinned to a commit, with the tag it was in a comment+# beside it: a tag can be moved to other code after a run has trusted it,+# a commit cannot. Dependabot proposes the bumps (.github/dependabot.yml).+# The prebuilt libraries the jobs download are pinned the same way, by+# digest, in scripts/ci/deps.lock.on:push:branches: [main]@@ -18,21 +24,27 @@ on:default: ""# Keyed on what is tested, so a release testing its tag and a push to-# main never cancel each other.+# main never cancel each other. Only a pull request's run is cancelled by+# the next push to it: a run on main is the record of whether main is+# green, and cancelling it leaves a gap where the answer should be.concurrency:group: ci-${{ inputs.ref || github.ref }}- cancel-in-progress: true+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}jobs:engine:name: Engine tests and lintsruns-on: ubuntu-latest+ # The steps that can run long have timeouts of their own, 110 minutes+ # between them, so a stuck step is named by its own timeout; this is+ # for a job stuck anywhere else. A warm run takes about twenty minutes.+ timeout-minutes: 130env:- FFMPEG_LINUX: https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-n8.1-latest-linux64-gpl-shared-8.1.tar.xz# See build-app.yml: the version the ort crate is built against.+ # scripts/ci/deps.lock pins the file.ORT_VERSION: 1.28.0steps:- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}# The toolchain src/rust-toolchain.toml pins, installed by name: every@@ -41,13 +53,22 @@ jobs:- name: The pinned toolchainid: toolchainrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}components: clippy, rustfmt- - uses: Swatinem/rust-cache@v2+ # cargo-nextest runs the tests: each in a process of its own, with a+ # timeout per test and a JUnit report. See src/.config/nextest.toml.+ - uses: taiki-e/install-action@ab6895351a88b908e96a4697860d4ffea5e57dbe # v2.87.27+ with:+ tool: [email protected]+ # Saved even when the job fails: a cache saved only by a green run+ # is never saved while the job is red, and every red run then+ # compiles from nothing.+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: src+ cache-on-failure: true# Slint's Skia renderer links fontconfig and freetype; cpal links# ALSA; bindgen needs libclang; whisper.cpp is built by cmake; the@@ -67,7 +88,7 @@ jobs:- name: FFmpeg development librariesrun: |set -euo pipefail- curl -fsSL --retry 5 --retry-all-errors -o ffmpeg.tar.xz "$FFMPEG_LINUX"+ scripts/ci/fetch.sh ffmpeg-linux64 ffmpeg.tar.xzmkdir -p ffmpeg-devtar -xf ffmpeg.tar.xz -C ffmpeg-dev --strip-components=1echo "FFMPEG_DIR=$PWD/ffmpeg-dev" >> "$GITHUB_ENV"@@ -80,9 +101,9 @@ jobs:run: |set -euo pipefailstem="onnxruntime-linux-x64-$ORT_VERSION"- curl -fsSL --retry 5 --retry-all-errors -o onnxruntime.tgz \- "https://github.com/microsoft/onnxruntime/releases/download/v$ORT_VERSION/$stem.tgz"+ scripts/ci/fetch.sh onnxruntime-linux-x64 onnxruntime.tgztar -xzf onnxruntime.tgz+ test -f "$stem/lib/libonnxruntime.so.$ORT_VERSION"ln -sf "libonnxruntime.so.$ORT_VERSION" "$stem/lib/libonnxruntime.so"{echo "ORT_LIB_LOCATION=$PWD/$stem/lib"@@ -94,14 +115,19 @@ jobs:# below fetch theirs. Left to its own build script, sherpa-onnx-sys# unpacks it into target/sherpa-onnx-prebuilt, which rust-cache does# not keep while the build script's run is kept: a cache hit then- # links against a folder that is not there.+ # links against a folder that is not there. The version is the+ # lockfile's, and deps.lock must pin that version's file.- name: sherpa-onnx librariesrun: |set -euo pipefailversion=$(grep -A1 'name = "sherpa-onnx-sys"' src/Cargo.lock | sed -n 's/^version = "\(.*\)"/\1/p')+ name=sherpa-onnx-linux-x64-static-lib+ case "$(scripts/ci/fetch.sh --url "$name")" in+ */v"$version"/*) ;;+ *) echo "scripts/ci/deps.lock pins $name for another version than Cargo.lock's $version" >&2; exit 1 ;;+ esacstem="sherpa-onnx-v$version-linux-x64-static-lib"- curl -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 \- "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"+ scripts/ci/fetch.sh "$name" sherpa.tar.bz2mkdir -p sherpatar -xjf sherpa.tar.bz2 -C sherpatest -d "sherpa/$stem/lib"@@ -123,25 +149,69 @@ jobs:# before tests: they finish in a fraction of the time and catch the# most.- run: cargo clippy --workspace --all-targets -- -D warnings+ timeout-minutes: 25+ working-directory: src++ # The tests are compiled in a step of their own, so a slow compile+ # and a slow test are told apart by which step ran out of time. One+ # job at a time and no debug info, to stay inside the runner's+ # memory; the run below finds everything already built.+ - name: Build the tests+ timeout-minutes: 25working-directory: src+ env:+ CARGO_PROFILE_DEV_DEBUG: "0"+ run: cargo nextest run --workspace --no-run --build-jobs 1# CONCAT_REQUIRE_GPU: the GPU tests fail rather than skip when no- # adapter is usable, so the parity suite is known to have run.- - run: cargo test --workspace -j 1+ # adapter is usable, so the parity suite is known to have run. The+ # report and the log are kept when it fails (the last step).+ - name: Run the tests+ timeout-minutes: 20working-directory: srcenv:CARGO_PROFILE_DEV_DEBUG: "0"CONCAT_REQUIRE_GPU: "1"+ run: |+ status=0+ cargo nextest run --workspace --build-jobs 1 --profile ci 2>&1 | tee nextest-workspace.log || status=$?+ mv target/nextest/ci/junit.xml target/nextest/ci/junit-workspace.xml 2>/dev/null || true+ exit "$status"+ # nextest does not run the examples in the documentation; cargo does.+ - name: Run the documentation's examples+ timeout-minutes: 10+ working-directory: src+ env:+ CARGO_PROFILE_DEV_DEBUG: "0"+ run: cargo test --workspace --doc -j 1# The performance table's quick scenarios against their budgets, in# the `quick` profile: planning, undo, the document, publishing.# The media scenarios need a machine that is not a shared runner.- run: cargo run -p concat-perf --profile quick -- --check --quick+ timeout-minutes: 10working-directory: src# The gRPC transport is a feature nothing else in the workspace turns# on, so it is linted and tested here on its own. See concat-server.- run: cargo clippy -p concat-server -p concat-cli --features grpc --all-targets -- -D warnings+ timeout-minutes: 10working-directory: src- - run: cargo test -p concat-server --features grpc+ - name: Run the gRPC tests+ timeout-minutes: 10working-directory: src+ run: |+ status=0+ cargo nextest run -p concat-server --features grpc --profile ci 2>&1 | tee nextest-grpc.log || status=$?+ mv target/nextest/ci/junit.xml target/nextest/ci/junit-grpc.xml 2>/dev/null || true+ exit "$status"++ - name: Keep the test report (failure only)+ if: failure()+ uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2+ with:+ name: test-report-linux+ path: |+ src/target/nextest/ci/*.xml+ src/nextest-*.log+ if-no-files-found: ignore# The engine crates that carry no native library build for the web. This# job is what keeps that true: a std::thread or a blocking wait that@@ -150,22 +220,24 @@ jobs:web:name: Engine on wasm32runs-on: ubuntu-latest+ timeout-minutes: 30steps:- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}- name: The pinned toolchainid: toolchainrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}components: clippytargets: wasm32-unknown-unknown- - uses: Swatinem/rust-cache@v2+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: srckey: wasm32+ cache-on-failure: true- run: >cargo clippy-p concat-core -p concat-project -p concat-effects -p concat-render -p concat-text@@ -196,80 +268,172 @@ jobs:sherpa: win-x64verb: Testsruns-on: ${{ matrix.os }}- # A compile is minutes and the Windows suite under an hour; a job past- # this is stuck, and a stuck job is what kept main red for five days.- timeout-minutes: 120+ # Windows' steps that can run long have timeouts of their own, 115+ # minutes between them, so whichever is stuck is named by its own+ # timeout. A stuck job kept main red for a week once: the tests' one+ # step held the compile and every test, ran into its ninety minutes+ # every time, and said nothing about which test it was waiting on.+ timeout-minutes: 135env:- FFMPEG_WINDOWS: https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-n8.1-latest-win64-gpl-shared-8.1.zip+ # No debug info in the test binaries, as on Linux: smaller, faster+ # to link, and nothing reads it here.+ CARGO_PROFILE_DEV_DEBUG: "0"+ ORT_VERSION: 1.28.0steps:- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}- name: The pinned toolchainid: toolchainshell: bashrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}+ - uses: taiki-e/install-action@ab6895351a88b908e96a4697860d4ffea5e57dbe # v2.87.27+ if: runner.os == 'Windows'+ with:+ tool: [email protected]- name: FFmpeg (macOS)if: runner.os == 'macOS'run: |set -euo pipefailbrew install ffmpeg cmakeecho "FFMPEG_DIR=$(brew --prefix ffmpeg)" >> "$GITHUB_ENV"- - name: FFmpeg and LLVM (Windows)+ # LLVM's own installer, at the version deps.lock pins, into a folder+ # of its own: the runner image carries an LLVM of its own, at+ # whatever version the image was made with, which this neither+ # upgrades nor depends on.+ - name: LLVM (Windows)if: runner.os == 'Windows'+ timeout-minutes: 10shell: pwshrun: |$ErrorActionPreference = "Stop"- choco install llvm -y --no-progress- "C:\Program Files\LLVM\bin" | Out-File -FilePath $env:GITHUB_PATH -Append- "LIBCLANG_PATH=C:\Program Files\LLVM\bin" | Out-File -FilePath $env:GITHUB_ENV -Append- curl.exe -fsSL --retry 5 --retry-all-errors -o ffmpeg-win.zip "$env:FFMPEG_WINDOWS"+ & "$env:ProgramFiles\Git\bin\bash.exe" scripts/ci/fetch.sh llvm-win64 llvm.exe+ if ($LASTEXITCODE -ne 0) { throw "the LLVM download failed" }+ Start-Process -Wait -FilePath .\llvm.exe -ArgumentList "/S", "/D=C:\llvm"+ if (-not (Test-Path C:\llvm\bin\libclang.dll)) { throw "LLVM did not install to C:\llvm" }+ Remove-Item llvm.exe+ "C:\llvm\bin" | Out-File -FilePath $env:GITHUB_PATH -Append+ "LIBCLANG_PATH=C:\llvm\bin" | Out-File -FilePath $env:GITHUB_ENV -Append+ - name: FFmpeg (Windows)+ if: runner.os == 'Windows'+ shell: pwsh+ run: |+ $ErrorActionPreference = "Stop"+ & "$env:ProgramFiles\Git\bin\bash.exe" scripts/ci/fetch.sh ffmpeg-win64 ffmpeg-win.zip+ if ($LASTEXITCODE -ne 0) { throw "the FFmpeg download failed" }Expand-Archive ffmpeg-win.zip -DestinationPath ffdl$dir = (Get-ChildItem ffdl -Directory | Select-Object -First 1).FullName"FFMPEG_DIR=$dir" | Out-File -FilePath $env:GITHUB_ENV -Append"$dir\bin" | Out-File -FilePath $env:GITHUB_PATH -Append+ # ONNX Runtime the way the Windows bundle links it (build-app.yml says+ # why): Microsoft's shared build, through ORT_LIB_LOCATION, so the+ # tests run against the runtime the app ships rather than the static+ # one ort-sys would download. On the path for the test binaries.+ - name: ONNX Runtime (Windows)+ if: runner.os == 'Windows'+ shell: pwsh+ run: |+ $ErrorActionPreference = "Stop"+ & "$env:ProgramFiles\Git\bin\bash.exe" scripts/ci/fetch.sh onnxruntime-win-x64 onnxruntime.zip+ if ($LASTEXITCODE -ne 0) { throw "the ONNX Runtime download failed" }+ Expand-Archive onnxruntime.zip -DestinationPath .+ $lib = "$PWD\onnxruntime-win-x64-$env:ORT_VERSION\lib"+ if (-not (Test-Path "$lib\onnxruntime.lib")) { throw "$lib\onnxruntime.lib is not there" }+ "ORT_LIB_LOCATION=$lib" | Out-File -FilePath $env:GITHUB_ENV -Append+ "ORT_PREFER_DYNAMIC_LINK=1" | Out-File -FilePath $env:GITHUB_ENV -Append+ $lib | Out-File -FilePath $env:GITHUB_PATH -Append# The prebuilt sherpa-onnx library, as build-app.yml fetches it:- # static on macOS, the import libraries alone on Windows.+ # static on macOS, the import libraries alone on Windows, whose DLLs+ # go on the path for the test binaries - all but sherpa's own+ # onnxruntime.dll, so the one the tests load is the one above.- name: sherpa-onnx librariesshell: bashrun: |set -euo pipefailversion=$(grep -A1 'name = "sherpa-onnx-sys"' src/Cargo.lock | sed -n 's/^version = "\(.*\)"/\1/p')if [ "$RUNNER_OS" = "Windows" ]; then- stem="sherpa-onnx-v$version-${{ matrix.sherpa }}-shared-MT-Release-lib"+ kind=shared-MT-Release-libelse- stem="sherpa-onnx-v$version-${{ matrix.sherpa }}-static-lib"+ kind=static-libfi- echo "Downloading $stem"- curl -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"+ name="sherpa-onnx-${{ matrix.sherpa }}-$kind"+ stem="sherpa-onnx-v$version-${{ matrix.sherpa }}-$kind"+ case "$(scripts/ci/fetch.sh --url "$name")" in+ */v"$version"/*) ;;+ *) echo "scripts/ci/deps.lock pins $name for another version than Cargo.lock's $version" >&2; exit 1 ;;+ esac+ scripts/ci/fetch.sh "$name" sherpa.tar.bz2mkdir -p sherpapython3 -c "import tarfile; t = tarfile.open('sherpa.tar.bz2'); t.extractall('sherpa', **({'filter': 'data'} if hasattr(tarfile, 'data_filter') else {}))"test -d "sherpa/$stem/lib"if [ "$RUNNER_OS" = "Windows" ]; thenmkdir -p sherpa-libs sherpa-dlls- cp "sherpa/$stem/lib/"*.lib sherpa-libs/- # The DLLs too: the test binaries load them as they start.- cp "sherpa/$stem/lib/"*.dll sherpa-dlls/+ cp "sherpa/$stem/lib/"sherpa-onnx-*.lib sherpa-libs/+ cp "sherpa/$stem/lib/"sherpa-onnx-*.dll sherpa-dlls/echo "SHERPA_ONNX_LIB_DIR=$(pwd -W)\\sherpa-libs" >> "$GITHUB_ENV"echo "$(pwd -W)\\sherpa-dlls" >> "$GITHUB_PATH"elseecho "SHERPA_ONNX_LIB_DIR=$PWD/sherpa/$stem/lib" >> "$GITHUB_ENV"fi- - uses: Swatinem/rust-cache@v2+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: srckey: check-${{ matrix.os }}+ cache-on-failure: true- run: cargo check --workspace --all-targetsif: runner.os != 'Windows'+ timeout-minutes: 45working-directory: src- # One job at a time and no debug info, as on Linux. The timeout is- # for a hang: concat-media's tests once hung here for good.- - run: cargo test --workspace -j 1++ # Windows: compile, then run, then the one scenario that has hung+ # here on its own. Each step has its own timeout, so a red run says+ # whether the compile was slow, a test failed, or a test hung, and a+ # test that hangs is killed by nextest after its slow-timeout and+ # reported by name (src/.config/nextest.toml). Two compile jobs: the+ # runner has the memory for two, and one took most of an hour.+ - name: Build the tests (Windows)+ id: build-testsif: runner.os == 'Windows'- timeout-minutes: 90+ timeout-minutes: 60working-directory: src- env:- CARGO_PROFILE_DEV_DEBUG: "0"+ shell: bash+ run: cargo nextest run --workspace --no-run --build-jobs 2+ - name: Run the tests (Windows)+ if: runner.os == 'Windows'+ timeout-minutes: 30+ working-directory: src+ shell: bash+ run: |+ status=0+ cargo nextest run --workspace --build-jobs 2 --profile ci \+ -E 'not test(=every_edit_still_exports)' 2>&1 | tee nextest-workspace.log || status=$?+ mv target/nextest/ci/junit.xml target/nextest/ci/junit-workspace.xml 2>/dev/null || true+ exit "$status"+ # every_edit_still_exports passes on Linux in under a minute and has+ # hung on Windows. Alone, with its output shown as it comes: each+ # export names itself before it starts and once it is written, so+ # the last line before a hang is the edit, and the half of it, that+ # never finished. Runs whether or not the rest passed.+ - name: The editing session alone (Windows)+ if: ${{ !cancelled() && runner.os == 'Windows' && steps.build-tests.outcome == 'success' }}+ timeout-minutes: 15+ working-directory: src+ shell: bash+ run: |+ status=0+ cargo nextest run -p concat-host --test export --build-jobs 2 --profile ci --no-capture \+ -E 'test(=every_edit_still_exports)' 2>&1 | tee nextest-every-edit.log || status=$?+ mv target/nextest/ci/junit.xml target/nextest/ci/junit-every-edit.xml 2>/dev/null || true+ exit "$status"+ - name: Keep the test report (failure only)+ if: failure() && runner.os == 'Windows'+ uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2+ with:+ name: test-report-windows+ path: |+ src/target/nextest/ci/*.xml+ src/nextest-*.log+ if-no-files-found: ignorediff --git a/.github/workflows/mobile.yml b/.github/workflows/mobile.ymlindex 758a6116..e36d2cac 100644--- a/.github/workflows/mobile.yml+++ b/.github/workflows/mobile.yml@@ -22,8 +22,9 @@ name: Build for phones## The APK is built under a key made for the run, and a release (the `sign`# input, which release.yml alone passes) signs it over again with the-# repository's key, ANDROID_KEYSTORE (the keystore, base64) and-# ANDROID_KEYSTORE_PASSWORD, in a step of its own after the build: the key+# repository's key, ANDROID_KEYSTORE (the keystore, base64),+# ANDROID_KEYSTORE_PASSWORD, and optionally ANDROID_KEY_ALIAS and+# ANDROID_KEY_PASSWORD, in a step of its own after the build: the key# never sits in the environment of the build and the several hundred build# scripts it runs, and a pull request's run never sees it at all. The .ipa# is ad-hoc signed: a tester signs it with their own Apple ID on the way@@ -45,15 +46,22 @@ on:required: falseANDROID_KEYSTORE_PASSWORD:required: false+ ANDROID_KEY_ALIAS:+ description: The key's alias in the keystore; the keystore's first alias when unset+ required: false+ ANDROID_KEY_PASSWORD:+ description: The key's own password; the keystore's when unset+ required: falsepull_request:workflow_dispatch:# Keyed by the calling workflow too, so a call from a release never shares-# a group with a pull request's run. A tag's release is never cancelled;-# everything else is superseded by the next push.+# a group with a pull request's run. Only a pull request's run is+# superseded by the next push to it; a release's, or one started by hand,+# runs to the end.concurrency:group: mobile-${{ github.workflow }}-${{ github.ref }}- cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/') }}+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}env:CARGO_TERM_COLOR: always@@ -69,7 +77,7 @@ jobs:steps:# The ref release.yml names, which is its tag: without one a run# started by hand builds main and publishes it under the tag's name.- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}- name: Read the version@@ -83,11 +91,11 @@ jobs:- name: The pinned toolchainid: toolchainrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}targets: aarch64-linux-android- - uses: taiki-e/cache-cargo-install-action@v2+ - uses: taiki-e/cache-cargo-install-action@7447f04c51f2ba27ca35e7f1e28fab848c5b3ba7 # v2.3.1with:tool: cargo-apk@@ -103,7 +111,7 @@ jobs:# build after it.- name: FFmpeg for Android (cached)id: ffmpeg- uses: actions/cache/restore@v4+ uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0with:path: src/vendor/ffmpeg/aarch64-linux-androidkey: ffmpeg-aarch64-linux-android-${{ hashFiles('src/scripts/ffmpeg-mobile.sh') }}@@ -112,7 +120,7 @@ jobs:run: src/scripts/ffmpeg-mobile.sh aarch64-linux-android- name: FFmpeg for Android (save)if: steps.ffmpeg.outputs.cache-hit != 'true'- uses: actions/cache/save@v4+ uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0with:path: src/vendor/ffmpeg/aarch64-linux-androidkey: ${{ steps.ffmpeg.outputs.cache-primary-key }}@@ -120,7 +128,7 @@ jobs:- name: sherpa-onnx for Androidrun: src/scripts/sherpa-mobile.sh aarch64-linux-android- - uses: Swatinem/rust-cache@v2+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: srckey: aarch64-linux-android@@ -155,11 +163,20 @@ jobs:# removed once the signature is on. apksigner replaces the run's# signature rather than adding to it. A release with no key set keeps# the run's key and says so in the run's annotations.+ #+ # apksigner is told everything it would otherwise stop and ask for:+ # which key (ANDROID_KEY_ALIAS, or the keystore's first alias, which+ # keytool lists), and the key's password (ANDROID_KEY_PASSWORD, or+ # the keystore's, which is what a key made with the keystore has).+ # A prompt on a runner is a step that waits for its timeout.- name: Sign with the repository's keyif: inputs.sign+ timeout-minutes: 10env:KEYSTORE_B64: ${{ secrets.ANDROID_KEYSTORE }}KEYSTORE_PASSWORD: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}+ KEY_ALIAS: ${{ secrets.ANDROID_KEY_ALIAS }}+ KEY_PASSWORD: ${{ secrets.ANDROID_KEY_PASSWORD || secrets.ANDROID_KEYSTORE_PASSWORD }}run: |set -euo pipefailif [ -z "$KEYSTORE_B64" ]; then@@ -168,10 +185,22 @@ jobs:fikeystore=$RUNNER_TEMP/release.keystoreecho "$KEYSTORE_B64" | base64 --decode > "$keystore"+ alias=$KEY_ALIAS+ if [ -z "$alias" ]; then+ alias=$(keytool -list -keystore "$keystore" -storepass:env KEYSTORE_PASSWORD 2>/dev/null \+ | awk -F, '/PrivateKeyEntry/ { print $1; exit }')+ if [ -z "$alias" ]; then+ rm -f "$keystore"+ echo "::error::the keystore holds no private key, or its password is wrong; set ANDROID_KEY_ALIAS"+ exit 1+ fi+ echo "ANDROID_KEY_ALIAS is not set; signing with the keystore's first key, $alias"+ fisdk=${ANDROID_HOME:-$ANDROID_SDK_ROOT}- tools=$(ls -d "$sdk"/build-tools/*/ | sort -V | tail -1)+ tools=$(find "$sdk/build-tools" -mindepth 1 -maxdepth 1 -type d | sort -V | tail -1)apk=src/target/release/apk/Concat.apk- "$tools/apksigner" sign --ks "$keystore" --ks-pass env:KEYSTORE_PASSWORD "$apk"+ "$tools/apksigner" sign --ks "$keystore" --ks-pass env:KEYSTORE_PASSWORD \+ --ks-key-alias "$alias" --key-pass env:KEY_PASSWORD "$apk" < /dev/nullrm -f "$keystore""$tools/apksigner" verify --print-certs "$apk"@@ -197,7 +226,7 @@ jobs:cp src/target/release/apk/Concat.apk "stage/Concat-$VERSION-android-arm64.apk"ls -l stage- - uses: actions/upload-artifact@v7+ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2with:name: Concat-android-arm64path: stage/*@@ -208,7 +237,7 @@ jobs:runs-on: macos-15timeout-minutes: 90steps:- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.ref }}- name: Read the version@@ -220,7 +249,7 @@ jobs:- name: The pinned toolchainid: toolchainrun: echo "channel=$(sed -n 's/^channel = "\(.*\)"/\1/p' src/rust-toolchain.toml)" >> "$GITHUB_OUTPUT"- - uses: dtolnay/rust-toolchain@master+ - uses: dtolnay/rust-toolchain@e2a55d2ffb04f378e9626c28d38b36d230d1e12f # master (v1), 2026-10-09with:toolchain: ${{ steps.toolchain.outputs.channel }}targets: aarch64-apple-ios@@ -232,7 +261,7 @@ jobs:# build after it.- name: FFmpeg for iOS (cached)id: ffmpeg- uses: actions/cache/restore@v4+ uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0with:path: src/vendor/ffmpeg/aarch64-apple-ioskey: ffmpeg-aarch64-apple-ios-${{ hashFiles('src/scripts/ffmpeg-mobile.sh') }}@@ -241,7 +270,7 @@ jobs:run: src/scripts/ffmpeg-mobile.sh aarch64-apple-ios- name: FFmpeg for iOS (save)if: steps.ffmpeg.outputs.cache-hit != 'true'- uses: actions/cache/save@v4+ uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0with:path: src/vendor/ffmpeg/aarch64-apple-ioskey: ${{ steps.ffmpeg.outputs.cache-primary-key }}@@ -249,7 +278,7 @@ jobs:- name: sherpa-onnx for iOSrun: src/scripts/sherpa-mobile.sh aarch64-apple-ios- - uses: Swatinem/rust-cache@v2+ - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2with:workspaces: srckey: aarch64-apple-ios@@ -294,7 +323,7 @@ jobs:(cd payload && ditto -c -k --keepParent Payload "../stage/Concat-$VERSION-ios-arm64.ipa")ls -l stage- - uses: actions/upload-artifact@v7+ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2with:name: Concat-ios-arm64path: stage/*diff --git a/.github/workflows/models.yml b/.github/workflows/models.ymlindex c40f5529..54e42aa6 100644--- a/.github/workflows/models.yml+++ b/.github/workflows/models.yml@@ -39,10 +39,13 @@ jobs:mirror:name: Fill and publishruns-on: ubuntu-latest+ # Two gigabytes down and two up, which takes a runner well under an+ # hour; past three, something is stuck.+ timeout-minutes: 180permissions:contents: writesteps:- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1- name: The tableid: table@@ -91,7 +94,7 @@ jobs:echo "patch=true" >> "$GITHUB_OUTPUT"fi- - uses: actions/upload-artifact@v7+ - uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2if: steps.recorded.outputs.patch == 'true'with:name: recorded-digestsdiff --git a/.github/workflows/nix.yml b/.github/workflows/nix.ymlindex 57ae1a0c..453f8114 100644--- a/.github/workflows/nix.yml+++ b/.github/workflows/nix.yml@@ -23,19 +23,25 @@ on:- .github/workflows/nix.ymlworkflow_dispatch:+# Only a pull request's run is superseded by the next push to it; a run on+# main is the record of whether main builds.concurrency:group: nix-${{ github.ref }}- cancel-in-progress: true+ cancel-in-progress: ${{ github.event_name == 'pull_request' }}jobs:build:name: nix buildruns-on: ubuntu-latest+ # A cold build of the whole workspace and its native dependencies.+ timeout-minutes: 120+ # Read only: the job builds what is checked in and writes nothing back+ # (the lock is refreshed by hand; see the top of this file).permissions:- contents: write+ contents: readsteps:- - uses: actions/checkout@v7- - uses: DeterminateSystems/nix-installer-action@v22+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ - uses: DeterminateSystems/nix-installer-action@ef8a148080ab6020fd15196c2084a2eea5ff2d25 # v22# Refreshes nothing: a lock that would change here is a lock the# commit should have carried.diff --git a/.github/workflows/release.yml b/.github/workflows/release.ymlindex 4f9f1ae8..86ee3707 100644--- a/.github/workflows/release.yml+++ b/.github/workflows/release.yml@@ -14,10 +14,18 @@ name: Release# (.deb, .rpm, .AppImage, Arch) and the Windows installers carry the# FFmpeg runtime libraries beside the binary; the Android .apk is signed# with the repository's key, and the iOS .ipa is signed by the tester on-# the way onto the phone. Nothing is built until CI has passed on the tag.+# the way onto the phone. Nothing is built until CI has passed on the tag,+# and nothing is published until every bundle has been launched where it+# was built: build-app.yml starts each one (`--version`, `--smoke`, and on+# x86_64 under an emulated Ivy Bridge) before it uploads it, and a bundle+# that does not start fails its job, which `publish` needs.+#+# The release is created as a draft. A person looks it over - the notes,+# the files - and publishes it from the Releases page; nothing reaches+# the updater or the downloads page before then.## A release that is not a pre-release also goes to the Microsoft Store,-# once it is published here, when the repository is set up for it; see+# once its draft is made here, when the repository is set up for it; see# the store job.on:push:@@ -36,16 +44,33 @@ jobs:check:name: Tag matches the versionruns-on: ubuntu-latest+ timeout-minutes: 10outputs:tag: ${{ steps.tag.outputs.tag }}version: ${{ steps.tag.outputs.version }}steps:- - uses: actions/checkout@v7+ # The tag a person typed is checked before anything uses it: as the+ # ref to check out, in a script, or in a file name.+ - name: The tag is a version tag+ env:+ TAG: ${{ inputs.tag || github.ref_name }}+ run: |+ if ! [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.]+)?$ ]]; then+ echo "::error::'$TAG' is not a version tag (v1.2.3, or v1.2.3-alpha.1)"+ exit 1+ fi+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ inputs.tag || github.ref }}+ # The tag comes in through the environment, never pasted into the+ # script: a value in ${{ }} is spliced into the shell's text before+ # bash reads it, so a tag typed into the form by hand could have run+ # as a command.- id: tag+ env:+ TAG: ${{ inputs.tag || github.ref_name }}run: |- tag="${{ inputs.tag || github.ref_name }}"+ tag="$TAG"version=$(sed -n 's/^version = "\(.*\)"/\1/p' src/Cargo.toml | head -1)# The tag is the version, or the version with a pre-release# suffix: v0.2.1 and v0.2.1-alpha.22 both belong to 0.2.1.@@ -95,11 +120,12 @@ jobs:name: Publish releaseneeds: [check, build, phones]runs-on: ubuntu-latest+ timeout-minutes: 30permissions:contents: writesteps:# The bundles only: the Store's packages are the store job's.- - uses: actions/download-artifact@v8+ - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2with:path: bundlespattern: Concat-*@@ -107,7 +133,7 @@ jobs:# The tree the release is cut from: the notes are read out of its# history, and the manifest out of its model table.- - uses: actions/checkout@v7+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1with:ref: ${{ needs.check.outputs.tag }}fetch-depth: 0@@ -118,11 +144,14 @@ jobs:# fetched from. An updater or a package script reads this instead of# guessing at asset names. See scripts/models.py.- name: Write the manifest+ env:+ VERSION: ${{ needs.check.outputs.version }}+ TAG: ${{ needs.check.outputs.tag }}run: |set -euo pipefailpython3 source/scripts/models.py release-manifest \- --version "${{ needs.check.outputs.version }}" \- --tag "${{ needs.check.outputs.tag }}" \+ --version "$VERSION" \+ --tag "$TAG" \--bundles bundles \--out bundles/manifest.jsoncat bundles/manifest.json@@ -134,22 +163,25 @@ jobs:ls -lsha256sum -- * > SHA256SUMS- # The notes are the tag's section of CHANGELOG.md, or the commits- # since the previous tag where it has none; see the script.+ # The notes are the commits since the previous tag; see the script.- name: Write the notesid: notes+ env:+ TAG: ${{ needs.check.outputs.tag }}run: |set -euo pipefailcd source- tag="${{ needs.check.outputs.tag }}"+ tag="$TAG"previous=$(git describe --tags --abbrev=0 --match 'v*' "$tag^" 2>/dev/null || git rev-list --max-parents=0 HEAD | tail -1).github/release-notes.sh "$previous" "$tag" "${tag#v}" > "$RUNNER_TEMP/notes.md"cat "$RUNNER_TEMP/notes.md"- - uses: softprops/action-gh-release@v3+ # A draft: a person publishes it. See the top of this file.+ - uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3with:tag_name: ${{ needs.check.outputs.tag }}name: Concat ${{ needs.check.outputs.tag }}+ draft: trueprerelease: ${{ contains(needs.check.outputs.tag, '-') }}body_path: ${{ runner.temp }}/notes.mdfail_on_unmatched_files: true@@ -178,8 +210,9 @@ jobs:needs: [check, publish]if: ${{ !contains(needs.check.outputs.tag, '-') && vars.MSSTORE_PRODUCT_ID != '' }}runs-on: windows-2022+ timeout-minutes: 30steps:- - uses: actions/download-artifact@v8+ - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2with:path: storepattern: Store-windows-*@@ -199,7 +232,7 @@ jobs:& $makeappx.FullName bundle /o /bv "$env:VERSION.0" /d store /p "Concat-$env:VERSION-windows.msixbundle"if ($LASTEXITCODE -ne 0) { throw "makeappx failed with $LASTEXITCODE" }- - uses: microsoft/setup-msstore-cli@v1+ - uses: microsoft/setup-msstore-cli@3b2ec9136230357ce9abc02b9d056626fb248d3f # v1- name: Submitshell: pwshdiff --git a/scripts/ci/deps.lock b/scripts/ci/deps.locknew file mode 100644index 00000000..6ebd1950--- /dev/null+++ b/scripts/ci/deps.lock@@ -0,0 +1,59 @@+# The prebuilt dependencies CI downloads, each pinned to one file and the+# sha256 of its bytes. scripts/ci/fetch.sh is the only thing that reads+# this, and it refuses a file whose digest is not the one written here.+#+# name url sha256+#+# To bump one: change the URL, run `scripts/ci/fetch.sh NAME /tmp/x`, and+# put the digest it reports in place of the old one. Where a version is+# also named elsewhere, change it there too; the comment above each group+# says where.++# FFmpeg 8.1, BtbN's GPL shared builds. A month's last autobuild, not+# `latest`: BtbN keeps those, where the daily ones are deleted after a+# fortnight, and `latest` changes under a build every day.+ffmpeg-linux64 https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-09-30-13-08/ffmpeg-n8.1.3-9-g29e619e767-linux64-gpl-shared-8.1.tar.xz 76649c19a7a4615b367251ba4a8adc643f65831278a00e7a0c38398f7f06484b+ffmpeg-linuxarm64 https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-09-30-13-08/ffmpeg-n8.1.3-9-g29e619e767-linuxarm64-gpl-shared-8.1.tar.xz d5b52fcb4f8d86d997bb75f6c4b383ca0be3f4cd9efd94ab3b4f27b22984f91c+ffmpeg-win64 https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-09-30-13-08/ffmpeg-n8.1.3-9-g29e619e767-win64-gpl-shared-8.1.zip dfe81c3aa0a546ee81980b1b824486dd0fe0a1d452f00cb55135379a74be7bcf+ffmpeg-winarm64 https://github.com/BtbN/FFmpeg-Builds/releases/download/autobuild-2026-09-30-13-08/ffmpeg-n8.1.3-9-g29e619e767-winarm64-gpl-shared-8.1.zip 31f35037625a8084085248fb47de2958678601ee9ab54e087234693616359c4b++# ONNX Runtime, Microsoft's own builds. 1.28.0 is the version ort+# 2.0.0-rc.13 is built against (ORT_VERSION in ci.yml and build-app.yml);+# 1.23.2 is the last Microsoft made for Intel Macs (ORT_INTEL_MAC_VERSION).+# The Windows ones are the CPU builds; build-app.yml says why.+onnxruntime-linux-x64 https://github.com/microsoft/onnxruntime/releases/download/v1.28.0/onnxruntime-linux-x64-1.28.0.tgz a3e1b79d7bb1bf09696ce675f49e4064e6c81f6202b8225624fff0e93f8d6407+onnxruntime-linux-aarch64 https://github.com/microsoft/onnxruntime/releases/download/v1.28.0/onnxruntime-linux-aarch64-1.28.0.tgz e15ff8b5d85afe6c144d97c6fd432254bf76a219daaf17658087d6ecb3e8f0bb+onnxruntime-win-x64 https://github.com/microsoft/onnxruntime/releases/download/v1.28.0/onnxruntime-win-x64-1.28.0.zip abef733dacbe2f571547a7150b479b5cb9cc0df22f96c24983a42cadb1b4f8bc+onnxruntime-win-arm64 https://github.com/microsoft/onnxruntime/releases/download/v1.28.0/onnxruntime-win-arm64-1.28.0.zip cbe4547463ece092b505c3581376ed5896d22b5429f39d5e645e425ecdd369ad+onnxruntime-osx-x86_64 https://github.com/microsoft/onnxruntime/releases/download/v1.23.2/onnxruntime-osx-x86_64-1.23.2.tgz d10359e16347b57d9959f7e80a225a5b4a66ed7d7e007274a15cae86836485a6++# sherpa-onnx's prebuilt libraries. The version is the one src/Cargo.lock+# resolves for sherpa-onnx-sys; the workflows check the two agree.+sherpa-onnx-linux-x64-static-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-linux-x64-static-lib.tar.bz2 d1be7a69ac2b30120058d8302e624239a3064085383cfa47994a14fdc44c32d6+sherpa-onnx-linux-aarch64-static-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-linux-aarch64-static-lib.tar.bz2 743b4debabab2e9f654e393da7263bd9795aabed60ae80c83023054083415d14+sherpa-onnx-osx-arm64-static-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-osx-arm64-static-lib.tar.bz2 126daa2e8c09a4c5d54dc985722c43bd22f598adc56445905b377454b1b27e38+sherpa-onnx-osx-x64-static-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-osx-x64-static-lib.tar.bz2 8d8db6199af0119b16f6e2b01c5548b90c4392a462388dd59491e8c471283cca+sherpa-onnx-win-x64-shared-MT-Release-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-win-x64-shared-MT-Release-lib.tar.bz2 7a362a9f339a2937e7f8806e904f3e8a031a7774982eea08bca8336b2c257529+sherpa-onnx-win-arm64-shared-MT-Release-lib https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.7/sherpa-onnx-v1.13.7-win-arm64-shared-MT-Release-lib.tar.bz2 2afa23a479fe15729e1ea9a9ed242454b1d9fccf749103742071684baa91a232++# The Linux packagers: nfpm (NFPM_VERSION in build-app.yml) for the .deb+# and .rpm, and appimagetool with the AppImage runtime it embeds, both a+# named release rather than `continuous`. The runtime is handed to+# appimagetool with --runtime-file, so it fetches nothing of its own.+nfpm-x86_64 https://github.com/goreleaser/nfpm/releases/download/v2.47.0/nfpm_2.47.0_Linux_x86_64.tar.gz 0660ca602b2d2d2ae4781a06c692b3eeb9d437ffea05b831d76e41f4a3188783+nfpm-arm64 https://github.com/goreleaser/nfpm/releases/download/v2.47.0/nfpm_2.47.0_Linux_arm64.tar.gz 1c0f5f2999b9a974bfb04fdb0cc3306096de530ac5dbb25d739cc5f5219c919c+appimagetool-x86_64 https://github.com/AppImage/appimagetool/releases/download/1.9.1/appimagetool-x86_64.AppImage ed4ce84f0d9caff66f50bcca6ff6f35aae54ce8135408b3fa33abfc3cb384eb0+appimagetool-aarch64 https://github.com/AppImage/appimagetool/releases/download/1.9.1/appimagetool-aarch64.AppImage f0837e7448a0c1e4e650a93bb3e85802546e60654ef287576f46c71c126a9158+appimage-runtime-x86_64 https://github.com/AppImage/type2-runtime/releases/download/20251108/runtime-x86_64 2fca8b443c92510f1483a883f60061ad09b46b978b2631c807cd873a47ec260d+appimage-runtime-aarch64 https://github.com/AppImage/type2-runtime/releases/download/20251108/runtime-aarch64 00cbdfcf917cc6c0ff6d3347d59e0ca1f7f45a6df1a428a0d6d8a78664d87444++# LLVM for Windows: libclang for bindgen, clang-cl on arm64, and+# llvm-objdump for scripts/ci/isa-scan.sh.+llvm-win64 https://github.com/llvm/llvm-project/releases/download/llvmorg-21.1.0/LLVM-21.1.0-win64.exe e751be8019b7500b100456c915b4b3c339d8d88b81b3aad49796d39f700e7ef3+llvm-woa64 https://github.com/llvm/llvm-project/releases/download/llvmorg-21.1.0/LLVM-21.1.0-woa64.exe 3f181aecc83d0eb40fb966ce18b97a3602871dbff83fc91745e7e0fd6661a7f7++# Intel's Software Development Emulator, which the smoke tests run the+# x86_64 bundles under posing as an Ivy Bridge (AVX, no AVX2): the CPU the+# bundles promise to start on. See scripts/ci/smoke-*.+sde-win https://downloadmirror.intel.com/924984/sde-external-10.13.1-2026-07-28-win.tar.xz 74e626ede09b0baa5011fc9e51b58627ea92c3fc0bae5fd7db34b490f335f651+sde-lin https://downloadmirror.intel.com/924984/sde-external-10.13.1-2026-07-28-lin.tar.xz 94e97d623fec54385686e1e7ba65ebc9941748c05ee451423948334892bf2b50diff --git a/scripts/ci/fetch.sh b/scripts/ci/fetch.shnew file mode 100755index 00000000..b067cdd1--- /dev/null+++ b/scripts/ci/fetch.sh@@ -0,0 +1,79 @@+#!/usr/bin/env bash+# Downloads one of the prebuilt dependencies scripts/ci/deps.lock names and+# checks it against the digest written there, so a build only ever links+# or ships bytes somebody has looked at once. A file whose digest does not+# match is deleted and the script fails, naming both digests: a release+# that was replaced upstream, or a download that went wrong, stops the+# build rather than ending up in a bundle.+#+# scripts/ci/fetch.sh NAME OUT download NAME to the file OUT+# scripts/ci/fetch.sh --url NAME print the URL deps.lock has for NAME+#+# OUT already there with the right digest is kept, so a rerun on a warm+# machine downloads nothing. Runs on Linux, macOS and Git's bash on+# Windows: sha256sum where there is one, shasum where there is not.+#+# To bump a dependency, change its row and put the new file's digest in+# it; scripts/ci/fetch.sh prints the digest it got when they differ.+set -euo pipefail++here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)+lock="$here/deps.lock"++usage() {+ echo "usage: $0 NAME OUT | $0 --url NAME" >&2+ exit 2+}++# The row for a name: its URL and digest, space-separated.+row() {+ awk -v name="$1" '+ /^[[:space:]]*(#|$)/ { next }+ $1 == name { print $2, $3; found = 1; exit }+ END { exit !found }+ ' "$lock"+}++digest() {+ if command -v sha256sum >/dev/null 2>&1; then+ sha256sum "$1" | awk '{ print $1 }'+ else+ shasum -a 256 "$1" | awk '{ print $1 }'+ fi+}++[ $# -eq 2 ] || usage++if [ "$1" = "--url" ]; then+ found=$(row "$2") || { echo "fetch.sh: no row named $2 in $lock" >&2; exit 1; }+ echo "${found%% *}"+ exit 0+fi++name=$1+out=$2+found=$(row "$name") || { echo "fetch.sh: no row named $name in $lock" >&2; exit 1; }+url=${found%% *}+want=${found##* }++if [ -f "$out" ] && [ "$(digest "$out")" = "$want" ]; then+ echo "fetch.sh: $name is already at $out"+ exit 0+fi++echo "fetch.sh: $name from $url"+mkdir -p "$(dirname "$out")"+curl -fsSL --retry 5 --retry-all-errors --connect-timeout 30 -o "$out" "$url"++got=$(digest "$out")+if [ "$got" != "$want" ]; then+ rm -f "$out"+ {+ echo "fetch.sh: $name does not match deps.lock"+ echo " url: $url"+ echo " expected: $want"+ echo " got: $got"+ } >&2+ exit 1+fi+echo "fetch.sh: $name verified (sha256 $got)"diff --git a/scripts/ci/isa-allow.txt b/scripts/ci/isa-allow.txtnew file mode 100644index 00000000..d0ff2fc3--- /dev/null+++ b/scripts/ci/isa-allow.txt@@ -0,0 +1,63 @@+# What scripts/ci/isa-scan.sh lets each shipped x86_64 file hold beyond+# the AVX baseline the bundles promise. One row per file name pattern (a+# shell glob, first match wins); then either `any`, for a library that+# picks its kernels by asking the CPU at run time and so carries code an+# older CPU never reaches, or five limits - avx2, fma, bmi, mulx/rorx,+# avx512; see scripts/ci/isa-scan.sh for what each counts - each a count+# (fail past it), `~` and a count (warn past it), or `-` (no limit). A file+# with no row may hold none at all.+#+# pattern avx2 fma bmi mulx/rorx avx512++# FFmpeg: every codec, filter and scaler picks its SIMD by av_get_cpu_flags.+avcodec-*.dll any+avdevice-*.dll any+avfilter-*.dll any+avformat-*.dll any+avutil-*.dll any+postproc-*.dll any+swresample-*.dll any+swscale-*.dll any+libavcodec.so* any+libavdevice.so* any+libavfilter.so* any+libavformat.so* any+libavutil.so* any+libpostproc.so* any+libswresample.so* any+libswscale.so* any++# Microsoft's ONNX Runtime: MLAS chooses its kernels by CPUID. Its general+# code is built for the baseline (1.28.0 holds no BMI instructions at all).+onnxruntime.dll any+onnxruntime_providers_shared.dll any+libonnxruntime.so* any++# sherpa-onnx's Windows DLLs: built with the static MSVC runtime, whose+# maths functions carry FMA3 twins chosen at run time.+sherpa-onnx-c-api.dll any+sherpa-onnx-cxx-api.dll any++# The MSVC runtime, shipped beside the binary: memcpy and friends pick+# their AVX2 paths at run time.+vcruntime140*.dll any+msvcp140*.dll any+concrt140.dll any++# Concat itself. It holds dispatched code of its own - memchr, aho-corasick+# and ring choose AVX2 or BMI2 paths at run time (ring's hand-written+# cryptography is where mulx and rorx come from), and Skia carries its+# Haswell and Skylake-X stages beside the baseline ones - so the vector+# classes are not limited. The compiler-made BMI column is the one that+# tells: pyke's static ONNX Runtime, linked into concat.exe in 0.2.5 and+# 0.2.6 (#313), brought some 2,700 of them. On Linux the binary also links+# sherpa-onnx statically, with an ONNX Runtime of its own inside.+#+# PROVISIONAL: these BMI limits are estimates, not measurements, so for+# now they warn rather than fail; the emulated Ivy Bridge in the smoke+# tests is what fails a build that would crash. The first run on CI+# prints each binary's counts. Once it has, set the limit a margin above+# the count shown, without the `~`, so the next library compiled for+# Haswell fails the build.+concat.exe - - ~1000 - -+concat - - ~1000 - -diff --git a/scripts/ci/isa-scan.sh b/scripts/ci/isa-scan.shnew file mode 100755index 00000000..b74d2768--- /dev/null+++ b/scripts/ci/isa-scan.sh@@ -0,0 +1,183 @@+#!/usr/bin/env bash+# Scans x86_64 binaries for instructions an Ivy Bridge does not have, so a+# bundle that would die at launch with an illegal instruction on a 2012+# machine (#193, #313) fails its build instead.+#+# scripts/ci/isa-scan.sh FILE_OR_FOLDER...+#+# Every PE (.exe, .dll) and ELF file found is disassembled with+# llvm-objdump, and each instruction from a set the x86_64 baseline the+# bundles promise (AVX, nothing newer; see GGML_* in build-app.yml) does not+# include is counted, by class:+#+# avx2 256-bit integer vector work, permutes, broadcasts, gathers+# fma fused multiply-add (FMA3)+# bmi the BMI1 and BMI2 bit operations a compiler emits for ordinary+# shifts and masks when told the CPU has them (shlx, shrx,+# sarx, bzhi, andn, bextr, blsi, blsr, blsmsk, pdep, pext):+# the sign of code compiled for Haswell as a whole rather+# than one dispatched function. #313's crash was a shlx.+# mulx/rorx the two BMI2 instructions hand-written cryptography uses on+# its fast paths (ring's, Go's), behind a CPU check; counted+# apart so they do not drown the column above+# avx512 anything touching a zmm or mask register+#+# Not every one of those is a fault waiting to happen. A library that asks+# the CPU what it has at run time and picks its kernels by the answer+# carries AVX2 code it never runs on an older machine: FFmpeg's codecs and+# scalers, ONNX Runtime's MLAS, sherpa-onnx. And a statically linked binary+# carries such code too - memchr, aho-corasick, ring and Skia all dispatch+# this way - so a count of zero is not what to expect from concat itself.+# scripts/ci/isa-allow.txt says, per file, how much of each class is+# expected: `any` for a library that dispatches (not even disassembled),+# or a limit per class for a binary that holds some dispatched code among+# its own - a number, past which the scan fails; `~` and a number, past+# which it warns; or `-`, no limit. A file with no row may hold none.+#+# What the count catches is the size of the problem #313 was: pyke's+# static ONNX Runtime 1.28.0 for Windows, compiled for Haswell, holds some+# 2,700 compiler-made BMI instructions (1,559 shlx, 783 shrx, 273 sarx);+# Microsoft's onnxruntime.dll of the same version holds none. What the+# count cannot do is say whether an instruction runs, which is why the+# build also launches the bundle under Intel's emulator posing as an Ivy+# Bridge (scripts/ci/smoke-*). The two together are the check.+#+# llvm-objdump is found as $LLVM_OBJDUMP, as llvm-objdump on the path, or as+# the newest llvm-objdump-<version> on the path, which is how Ubuntu installs+# it. Exits 1 when a file is over a hard limit, 2 on a usage error.+set -euo pipefail++here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)+allow="$here/isa-allow.txt"++if [ $# -eq 0 ]; then+ echo "usage: $0 FILE_OR_FOLDER..." >&2+ exit 2+fi++objdump=${LLVM_OBJDUMP:-}+if [ -z "$objdump" ]; then+ if command -v llvm-objdump >/dev/null 2>&1; then+ objdump=llvm-objdump+ else+ objdump=$(compgen -c llvm-objdump- | sort -t- -k3 -n | tail -1 || true)+ fi+fi+if [ -z "$objdump" ] || ! command -v "$objdump" >/dev/null 2>&1; then+ echo "isa-scan: no llvm-objdump; set LLVM_OBJDUMP or install LLVM" >&2+ exit 2+fi++# The allowance for a file, as "avx2 fma bmi mulx/rorx avx512" limits, or+# `any`: the first row whose pattern matches its name, or all zero.+allowance() {+ local base=$1 pattern rest+ while read -r pattern rest; do+ case "$pattern" in '' | '#'*) continue ;; esac+ # shellcheck disable=SC2254 # the pattern is a glob on purpose+ case "$base" in $pattern) echo "${rest%%#*}"; return ;; esac+ done < "$allow"+ echo 0 0 0 0 0+}++# Whether a file is an x86_64 PE or ELF, by its magic bytes and machine+# field, without needing `file` (Git's bash on Windows has none).+is_x86_64() {+ local f=$1 magic+ magic=$(od -An -tx1 -N4 "$f" 2>/dev/null | tr -d ' \n')+ case "$magic" in+ 7f454c46)+ # ELF: e_machine at 18, 0x3e is x86-64.+ [ "$(od -An -tx1 -j18 -N1 "$f" | tr -d ' \n')" = 3e ]+ ;;+ 4d5a*)+ # PE: the header's offset at 0x3c, then "PE\0\0" and the machine,+ # 0x8664 for x86-64.+ local off+ off=$(od -An -tu4 -j60 -N4 "$f" | tr -d ' \n')+ [ "$(od -An -tx1 -j$((off + 4)) -N2 "$f" | tr -d ' \n')" = 6486 ]+ ;;+ *) return 1 ;;+ esac+}++# Counts per class for one file, as "avx2 fma bmi mulx/rorx avx512", and+# the ten most frequent offending mnemonics after them on separate lines.+count() {+ "$objdump" -d --no-show-raw-insn "$1" 2>/dev/null | awk '+ # The mnemonic is the first word after the address; AT&T syntax,+ # which is llvm-objdump'"'"'s default for x86.+ /^ *[0-9a-f]+:/ {+ line = $0+ sub(/^ *[0-9a-f]+:[ \t]*/, "", line)+ split(line, w, /[ \t]+/)+ m = w[1]+ if (m == "" || m ~ /^(rep|lock|notrack|data16|cs|ds)$/) m = w[2]+ cls = ""+ if (line ~ /%zmm|%k[0-7]/) cls = "avx512"+ else if (m ~ /^vf(n)?m(add|sub|addsub|subadd)(132|213|231)/) cls = "fma"+ else if (m ~ /^(andn|bextr|blsi|blsmsk|blsr|bzhi|pdep|pext|sarx|shlx|shrx)[lq]?$/) cls = "bmi"+ else if (m ~ /^(mulx|rorx)[lq]?$/) cls = "asm"+ else if (m ~ /^(vpbroadcast[bwdq]|vbroadcasti128|vperm2i128|vpermd|vpermq|vpermps|vpermpd|vinserti128|vextracti128|vpmaskmov[dq]|vpsllv[dq]|vpsrlv[dq]|vpsravd|vpgather|vgather|vpblendd)/) cls = "avx2"+ else if (m ~ /^vp/ && line ~ /%ymm/ && m !~ /^(vpermil|vptest|vperm2f128)/) cls = "avx2"+ if (cls != "") { n[cls]++; seen[m]++ }+ }+ END {+ printf "%d %d %d %d %d\n", n["avx2"], n["fma"], n["bmi"], n["asm"], n["avx512"]+ for (k in seen) printf "%d %s\n", seen[k], k+ }' | { read -r head; echo "$head"; sort -rn | head -10; }+}++files=()+for arg in "$@"; do+ if [ -d "$arg" ]; then+ while IFS= read -r -d '' f; do files+=("$f"); done \+ < <(find "$arg" -type f \( -name '*.exe' -o -name '*.dll' -o -name '*.so' -o -name '*.so.*' -o -perm -u+x \) -print0)+ else+ files+=("$arg")+ fi+done++status=0+row() { printf '%-36s %10s %10s %10s %10s %10s %s\n' "$@"; }+row file avx2 fma bmi mulx/rorx avx512 verdict+for f in "${files[@]}"; do+ is_x86_64 "$f" || continue+ base=$(basename "$f")+ read -r -a limits <<< "$(allowance "$base")"+ if [ "${limits[0]}" = any ]; then+ row "$base" - - - - - "dispatches at run time; not scanned"+ continue+ fi+ result=$(count "$f")+ read -r -a counts <<< "$(head -1 <<< "$result")"+ verdict=ok+ shown=()+ for i in 0 1 2 3 4; do+ limit=${limits[$i]:-0}+ shown+=("${counts[$i]}/$limit")+ case "$limit" in+ -) ;;+ "~"*)+ if [ "${counts[$i]}" -gt "${limit#"~"}" ] && [ "$verdict" = ok ]; then verdict=WARN; fi+ ;;+ *)+ if [ "${counts[$i]}" -gt "$limit" ]; then verdict=OVER; status=1; fi+ ;;+ esac+ done+ row "$base" "${shown[@]}" "$verdict"+ if [ "$verdict" != ok ]; then+ tail -n +2 <<< "$result" | sed 's/^/ /'+ fi+ if [ "$verdict" = WARN ] && [ -n "${GITHUB_ACTIONS:-}" ]; then+ echo "::warning::isa-scan: $base holds more post-Ivy Bridge instructions than scripts/ci/isa-allow.txt expects (${shown[*]}); see that file"+ fi+done++if [ "$status" -ne 0 ]; then+ echo+ echo "isa-scan: a file holds more post-Ivy Bridge instructions than scripts/ci/isa-allow.txt allows it." >&2+ echo "A library compiled for Haswell (AVX2, FMA, BMI2) as a whole crashes the app on older CPUs; see #313." >&2+fi+exit "$status"diff --git a/scripts/ci/smoke-linux.sh b/scripts/ci/smoke-linux.shnew file mode 100644index 00000000..5e2c32ba--- /dev/null+++ b/scripts/ci/smoke-linux.sh@@ -0,0 +1,157 @@+#!/usr/bin/env bash+# Launches a built Linux bundle the way a person would, and fails if it+# does not start. Nothing else in the build ever runs what it packages:+# a library left out of lib/, a dependency the .deb forgot to name, or an+# instruction an older CPU lacks all used to reach a release unnoticed.+#+# smoke-linux.sh [--sde SDE64] DIR+# The staged folder (DIR/concat beside DIR/lib), on this machine:+# every library resolves from the folder alone; `concat --version`+# exits 0; `concat --smoke` (start-up through the first frame of the+# window, then out) exits 0 under Xvfb, and does not so much as try+# to open the NVIDIA driver's CUDA or NVENC libraries, which nothing+# should touch before a person asks for a hardware export. With+# --sde, `--version` again under Intel's emulator posing as an Ivy+# Bridge (AVX, no AVX2), the oldest CPU the x86_64 bundle promises.+#+# smoke-linux.sh --container IMAGE DEB APPIMAGE+# The packages, in a clean IMAGE (ubuntu:22.04, debian:12) through+# docker: the .deb installed by apt, which pulls in exactly the+# dependencies it declares and nothing else, then `--version` and+# `--smoke` from the installed package and from the AppImage.+#+# Runs in CI after staging (build-app.yml), and locally against a folder+# or packages built the same way. Needs xvfb-run; strace if it is there.+set -euo pipefail++here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)++say() { printf '\n== %s\n' "$*"; }+fail() {+ if [ -n "${GITHUB_ACTIONS:-}" ]; then echo "::error::$*"; else echo "smoke: $*" >&2; fi+ exit 1+}++# A fresh home for each run, so the start-up is a first start-up: no+# settings, no recent projects, nothing cached from a run before.+fresh_home() {+ local home+ home=$(mktemp -d)+ mkdir -p "$home/.config" "$home/.cache" "$home/.local/share"+ echo "$home"+}++# Runs a command with a time limit, prints what it wrote, and fails with+# what it is told when the command does not exit 0.+expect_ok() {+ local what=$1 secs=$2 out status=0+ shift 2+ out=$(timeout --kill-after=10 "$secs" "$@" 2>&1) || status=$?+ printf '%s\n' "$out" | tail -n 60+ if [ "$status" -eq 124 ] || [ "$status" -eq 137 ]; then+ fail "$what: still running after ${secs}s"+ elif [ "$status" -ne 0 ]; then+ fail "$what: exited with $status"+ fi+}++host() {+ local sde="" dir+ if [ "${1:-}" = "--sde" ]; then sde=$2; shift 2; fi+ [ $# -eq 1 ] || { echo "usage: $0 [--sde SDE64] DIR" >&2; exit 2; }+ dir=$(cd "$1" && pwd)+ local bin="$dir/concat"+ [ -x "$bin" ] || fail "$bin is not there"++ # The build's own environment points at the FFmpeg it linked; the+ # folder has to stand without it.+ unset LD_LIBRARY_PATH++ say "every library resolves from the folder"+ local report+ report=$(ldd "$bin" || true)+ if grep -i 'not found' <<< "$report"; then+ echo "$report"+ fail "the binary does not resolve every library from its own folder"+ fi++ say "concat --version"+ HOME=$(fresh_home) expect_ok "concat --version" 60 "$bin" --version++ say "concat --smoke"+ local trace+ trace=$(mktemp)+ local tracer=()+ if command -v strace >/dev/null 2>&1; then+ tracer=(strace -f -qq -e "trace=open,openat" -o "$trace")+ else+ echo "no strace: the start-up is run but its file opens are not checked"+ fi+ HOME=$(fresh_home) expect_ok "concat --smoke" 180 \+ xvfb-run -a -s "-screen 0 1920x1080x24" "${tracer[@]}" "$bin" --smoke+ if [ ${#tracer[@]} -gt 0 ]; then+ local cuda+ cuda=$(grep -E 'lib(cuda|nvcuvid|nvidia-encode)\.so' "$trace" || true)+ if [ -n "$cuda" ]; then+ echo "$cuda"+ fail "concat --smoke tried to load the NVIDIA driver's CUDA or NVENC libraries at start-up"+ fi+ echo "no CUDA or NVENC library opened at start-up"+ fi++ if [ -n "$sde" ]; then+ say "concat --version on an emulated Ivy Bridge"+ # The emulator stops the program at the first instruction the chip+ # it poses as does not have, and says which.+ HOME=$(fresh_home) expect_ok "concat --version under SDE -ivb" 600 "$sde" -ivb -- "$bin" --version+ fi+}++container() {+ [ $# -eq 3 ] || { echo "usage: $0 --container IMAGE DEB APPIMAGE" >&2; exit 2; }+ local image=$1 deb appimage+ deb=$(cd "$(dirname "$2")" && pwd)/$(basename "$2")+ appimage=$(cd "$(dirname "$3")" && pwd)/$(basename "$3")+ [ -f "$deb" ] || fail "$deb is not there"+ [ -f "$appimage" ] || fail "$appimage is not there"+ say "in a clean $image"+ docker run --rm \+ -v "$here/smoke-linux.sh:/ci/smoke-linux.sh:ro" \+ -v "$deb:/pkg/concat.deb:ro" \+ -v "$appimage:/pkg/concat.AppImage:ro" \+ "$image" bash /ci/smoke-linux.sh --inside /pkg/concat.deb /pkg/concat.AppImage+}++# Inside the container. Xvfb and Mesa's software drivers stand in for+# the desktop's display and GPU, and one font for the desktop's fonts;+# everything else the program needs comes from the package's own+# dependencies, so a library it forgot to name is missing here too.+inside() {+ local deb=$1 appimage=$2+ export DEBIAN_FRONTEND=noninteractive+ say "installing the .deb and what it depends on"+ apt-get update -qq+ apt-get install -y -qq --no-install-recommends \+ "$deb" xvfb xauth libgl1-mesa-dri mesa-vulkan-drivers fonts-dejavu-core > /dev/null++ say "concat --version (the package)"+ HOME=$(fresh_home) expect_ok "concat --version from the .deb" 60 concat --version+ say "concat --smoke (the package)"+ HOME=$(fresh_home) expect_ok "concat --smoke from the .deb" 180 \+ xvfb-run -a -s "-screen 0 1920x1080x24" concat --smoke++ # No FUSE in a container: the AppImage unpacks itself and runs.+ export APPIMAGE_EXTRACT_AND_RUN=1+ say "concat --version (the AppImage)"+ HOME=$(fresh_home) expect_ok "concat --version from the AppImage" 120 "$appimage" --version+ say "concat --smoke (the AppImage)"+ HOME=$(fresh_home) expect_ok "concat --smoke from the AppImage" 240 \+ xvfb-run -a -s "-screen 0 1920x1080x24" "$appimage" --smoke+}++case "${1:-}" in+ --container) shift; container "$@" ;;+ --inside) shift; inside "$@" ;;+ *) host "$@" ;;+esac+say "ok"diff --git a/scripts/ci/smoke-macos.sh b/scripts/ci/smoke-macos.shnew file mode 100644index 00000000..bb616f4d--- /dev/null+++ b/scripts/ci/smoke-macos.sh@@ -0,0 +1,72 @@+#!/usr/bin/env bash+# Launches the built macOS bundle the way a person would, and fails if it+# does not start: nothing else in the build ever runs what it packages.+#+# smoke-macos.sh Concat.app+#+# The signature holds over the whole bundle (an ad-hoc one, which is what+# a fork's build gets, passes and is reported); no binary in it loads a+# library from Homebrew or anywhere else outside the bundle; `concat+# --version` exits 0; and `concat --smoke` - start-up through the first+# frame of the window, then out - exits 0. Runs in CI after signing+# (build-app.yml), and locally against a bundle built the same way.+set -euo pipefail++say() { printf '\n== %s\n' "$*"; }+note() {+ if [ -n "${GITHUB_ACTIONS:-}" ]; then echo "::warning::$*"; else echo "smoke: $*"; fi+}+fail() {+ if [ -n "${GITHUB_ACTIONS:-}" ]; then echo "::error::$*"; else echo "smoke: $*" >&2; fi+ exit 1+}++# A time limit without coreutils: perl's alarm, which the exec keeps.+# A run that overstays it dies of SIGALRM, status 142.+expect_ok() {+ local what=$1 secs=$2 out status=0+ shift 2+ out=$(perl -e 'alarm shift; exec @ARGV or die "exec: $!"' "$secs" "$@" 2>&1) || status=$?+ printf '%s\n' "$out" | tail -n 60+ if [ "$status" -eq 142 ]; then+ fail "$what: still running after ${secs}s"+ elif [ "$status" -ne 0 ]; then+ fail "$what: exited with $status"+ fi+}++[ $# -eq 1 ] || { echo "usage: $0 Concat.app" >&2; exit 2; }+app=$1+bin="$app/Contents/MacOS/concat"+[ -x "$bin" ] || fail "$bin is not there"++say "the signature"+codesign --verify --deep --strict --verbose=2 "$app" || fail "the bundle's signature does not verify"+details=$(codesign -dv "$app" 2>&1 || true)+if grep -q 'Signature=adhoc' <<< "$details"; then+ note "the bundle is ad-hoc signed: it opens, with Gatekeeper's warning, and is not notarised"+else+ grep -E '^(Authority|TeamIdentifier)=' <<< "$details" || true+fi++say "nothing loads from outside the bundle"+leaks=""+while IFS= read -r -d '' f; do+ if [[ $(file -b "$f") == *Mach-O* ]]; then+ found=$(otool -L "$f" | tail -n +2 | grep -E '/opt/homebrew|/usr/local|/opt/local' || true)+ if [ -n "$found" ]; then leaks+="$f:"$'\n'"$found"$'\n'; fi+ fi+done < <(find "$app/Contents" -type f -print0)+if [ -n "$leaks" ]; then+ printf '%s' "$leaks"+ fail "a binary in the bundle loads a library from outside it"+fi+echo "every load path is the system's or the bundle's"++say "concat --version"+HOME=$(mktemp -d) expect_ok "concat --version" 60 "$bin" --version++say "concat --smoke"+HOME=$(mktemp -d) expect_ok "concat --smoke" 180 "$bin" --smoke++say "ok"diff --git a/scripts/ci/smoke-windows.ps1 b/scripts/ci/smoke-windows.ps1new file mode 100644index 00000000..bf053236--- /dev/null+++ b/scripts/ci/smoke-windows.ps1@@ -0,0 +1,161 @@+# Launches the staged Windows bundle the way a person would, and fails if+# it does not start. Nothing else in the build ever ran what it packaged,+# and 0.2.6 shipped installers whose program would not open: a DLL that+# was not in the folder, an old DirectML.dll found in System32 instead of+# a new one (#303), an instruction an Ivy Bridge does not have (#313).+#+# scripts/ci/smoke-windows.ps1 -Bundle stage\Concat-<version>-windows-<arch> [-Sde path\to\sde.exe]+#+# 1. Every DLL any program or library in the folder imports is either in+# the folder or one Windows itself provides. A DLL that only happens to+# be on this machine - the Visual C++ runtime, DirectML, ONNX Runtime,+# which Windows 11 carries a copy of its own - counts as missing.+# 2. `concat.exe --version` exits 0 with PATH cut down to Windows' own+# folders, so every DLL comes from the folder or from System32 - as on+# a machine with nothing else installed. It touches no GPU: it proves+# the program and every DLL it loads at start load and initialise.+# 3. `concat.exe --smoke` exits 0 the same way: the start-up runs through+# the first frame of the window, then the program leaves.+# 4. With -Sde, `--version` again under Intel's emulator posing as an Ivy+# Bridge (AVX, nothing newer), which stops the program at the first+# instruction that chip does not have. x86_64 only.+#+# concat.exe is a GUI program: it has no console, so it is judged by its+# exit code and not by what it prints. Windows' error dialogs are turned+# off for the programs this starts, so a missing DLL is an exit code here+# rather than a dialog waiting for a click on a runner nobody is watching.+#Requires -Version 7+param(+ [Parameter(Mandatory = $true)][string]$Bundle,+ [string]$Sde = ""+)+$ErrorActionPreference = "Stop"++$bundle = (Resolve-Path $Bundle).Path+$exe = Join-Path $bundle "concat.exe"+if (-not (Test-Path $exe)) { throw "$exe is not there" }++function Say($text) { Write-Host ""; Write-Host "== $text" }+function Fail($text) {+ if ($env:GITHUB_ACTIONS) { Write-Host "::error::$text" }+ throw $text+}++# No error dialogs for this process or the ones it starts: a child takes+# its error mode from its parent. SEM_FAILCRITICALERRORS (the loader's+# "the code execution cannot proceed") and SEM_NOGPFAULTERRORBOX (the+# crash dialog).+Add-Type -Namespace Smoke -Name Native -MemberDefinition @"+[DllImport("kernel32.dll")] public static extern uint SetErrorMode(uint mode);+"@+[void][Smoke.Native]::SetErrorMode(0x0001 -bor 0x0002)++# What a status a program died with means, for the ones a broken bundle+# dies with.+$meanings = @{+ [int]0xC0000135 = "STATUS_DLL_NOT_FOUND: a DLL it imports is nowhere to be found"+ [int]0xC0000139 = "STATUS_ENTRYPOINT_NOT_FOUND: a DLL it found is not the one it was built against"+ [int]0xC000007B = "STATUS_INVALID_IMAGE_FORMAT: a DLL for another architecture, or a damaged one"+ [int]0xC000001D = "STATUS_ILLEGAL_INSTRUCTION: an instruction this CPU does not have"+ [int]0xC0000005 = "STATUS_ACCESS_VIOLATION: a crash"+ [int]0xC0000409 = "STATUS_STACK_BUFFER_OVERRUN: a fail-fast, which is how a Rust panic aborts"+ [int]0xC0E90002 = "a code integrity block (Smart App Control) on an unsigned or untrusted file"+}++# Starts a program with the given PATH and waits for it, up to a limit;+# fails unless it exits 0. Its output is shown when there is any.+function Expect-Ok($what, $file, [string[]]$arguments, $path, $seconds) {+ $info = New-Object System.Diagnostics.ProcessStartInfo+ $info.FileName = $file+ foreach ($a in $arguments) { [void]$info.ArgumentList.Add($a) }+ $info.UseShellExecute = $false+ $info.RedirectStandardOutput = $true+ $info.RedirectStandardError = $true+ # Not the bundle: the program has to find its DLLs beside itself, not+ # in the folder it was started from.+ $info.WorkingDirectory = [IO.Path]::GetTempPath()+ $info.Environment["PATH"] = $path+ $process = [System.Diagnostics.Process]::Start($info)+ $stdout = $process.StandardOutput.ReadToEndAsync()+ $stderr = $process.StandardError.ReadToEndAsync()+ if (-not $process.WaitForExit($seconds * 1000)) {+ $process.Kill($true)+ Fail "${what}: still running after $seconds s"+ }+ $process.WaitForExit()+ $out = ($stdout.Result + $stderr.Result).Trim()+ if ($out) { $out -split "`n" | Select-Object -Last 60 | ForEach-Object { Write-Host " $_" } }+ $code = $process.ExitCode+ if ($code -ne 0) {+ $meaning = $meanings[[int]$code]+ Fail ("${what}: exited with 0x{0:X8}{1}" -f $code, $(if ($meaning) { " - $meaning" } else { "" }))+ }+ Write-Host " exited 0"+}++# ── 1. Every import is in the folder or Windows' own ───────────────────+Say "every DLL the bundle imports is in it, or is Windows' own"+$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"+$dumpbin = & $vswhere -latest -products * -find "VC\Tools\MSVC\**\bin\Host*\*\dumpbin.exe" | Select-Object -First 1+if (-not $dumpbin) { throw "dumpbin.exe is not in the Visual Studio installation" }++# DLLs a clean Windows does not have, or has in a version of its own that+# is not the one the bundle was built against. One of these resolved from+# System32 on the runner is a DLL missing on somebody's machine.+$mustShip = @(+ "vcruntime140.dll", "vcruntime140_1.dll", "msvcp140.dll", "msvcp140_1.dll",+ "msvcp140_2.dll", "msvcp140_atomic_wait.dll", "msvcp140_codecvt_ids.dll",+ "concrt140.dll", "vccorlib140.dll", "vcomp140.dll",+ "directml.dll", "onnxruntime.dll", "onnxruntime_providers_shared.dll"+)+$system = Join-Path $env:SystemRoot "System32"+$inBundle = @{}+Get-ChildItem $bundle -Recurse -File | ForEach-Object { $inBundle[$_.Name.ToLowerInvariant()] = $true }+$missing = @()+foreach ($pe in Get-ChildItem $bundle -Recurse -File -Include *.exe, *.dll) {+ $lines = & $dumpbin /nologo /dependents $pe.FullName+ if ($LASTEXITCODE -ne 0) { throw "dumpbin failed on $($pe.Name)" }+ # The names are the indented lines under "Image has the following+ # (delay load) dependencies:", up to the summary.+ $reading = $false+ foreach ($line in $lines) {+ if ($line -match "Image has the following .*dependencies") { $reading = $true; continue }+ if ($line -match "^\s*Summary") { $reading = $false; continue }+ if (-not $reading) { continue }+ $dll = $line.Trim()+ if (-not $dll -or $dll -notmatch "\.dll$") { continue }+ $name = $dll.ToLowerInvariant()+ if ($inBundle.ContainsKey($name)) { continue }+ # API sets: names the loader resolves to whatever implements them.+ if ($name -match "^(api|ext)-ms-") { continue }+ if ($mustShip -contains $name) { $missing += "$($pe.Name) -> $dll (must ship with the program)"; continue }+ if (-not (Test-Path (Join-Path $system $dll))) { $missing += "$($pe.Name) -> $dll (not in the folder or in System32)" }+ }+}+if ($missing.Count -gt 0) {+ $missing | ForEach-Object { Write-Host " $_" }+ Fail "the bundle imports DLLs it does not carry"+}+Write-Host " every import resolves to the folder or to Windows"++# Windows' own folders and nothing else.+$bare = @($system, $env:SystemRoot, (Join-Path $system "Wbem"), (Join-Path $system "WindowsPowerShell\v1.0")) -join ";"++# ── 2. --version ───────────────────────────────────────────────────────+Say "concat.exe --version, with only Windows on the path"+Expect-Ok "concat.exe --version" $exe @("--version") $bare 60++# ── 3. --smoke ─────────────────────────────────────────────────────────+Say "concat.exe --smoke, with only Windows on the path"+Expect-Ok "concat.exe --smoke" $exe @("--smoke") $bare 180++# ── 4. On an emulated Ivy Bridge ───────────────────────────────────────+if ($Sde) {+ Say "concat.exe --version on an emulated Ivy Bridge"+ $sdeExe = (Resolve-Path $Sde).Path+ # The emulator's own folder too: it loads its pieces from beside itself.+ $path = (Split-Path $sdeExe) + ";" + $bare+ Expect-Ok "concat.exe --version under SDE -ivb" $sdeExe @("-ivb", "--", $exe, "--version") $path 600+}++Say "ok"diff --git a/src/.config/nextest.toml b/src/.config/nextest.tomlnew file mode 100644index 00000000..24181fc1--- /dev/null+++ b/src/.config/nextest.toml@@ -0,0 +1,40 @@+# cargo-nextest's settings for this workspace. CI runs the tests through+# nextest rather than `cargo test` for three things `cargo test` does not+# give it: every test in a process of its own, so a process-wide setting+# one test changes (the hardware decode preference, say) never reaches+# another; a timeout per test, so a test that hangs is killed and named+# rather than holding the whole step until the runner gives up on it; and+# a JUnit report the job keeps when it fails.+#+# Locally `cargo nextest run` uses the default profile, which is nextest's+# own defaults; `--profile ci` is what CI runs.++[profile.ci]+# Every test runs, whatever fails first: a red run should list everything+# that is red, not the first thing.+fail-fast = false+# A test is reported as slow after a minute and killed after five, which+# is far past what any test takes on Linux (the longest, the export+# scenarios, finish in under one). Killed, it is reported as a timeout+# under its own name.+slow-timeout = { period = "60s", terminate-after = 5 }+# Failures again at the end, after the rest of the output has scrolled+# past them.+final-status-level = "fail"+failure-output = "immediate-final"++[profile.ci.junit]+# Written to target/nextest/ci/junit.xml, which the CI job uploads when it+# fails. The output of a failed test is kept in the report; a passing+# one's is not.+path = "junit.xml"+store-success-output = false+store-failure-output = true++# The scripted editing session exports some thirty times, and on Windows'+# software adapter each export is slower than on Linux's lavapipe: it is+# given longer before it counts as stuck. CI also runs it on its own with+# its output shown, so a hang names the edit it stopped at; see ci.yml.+[[profile.ci.overrides]]+filter = "test(=every_edit_still_exports)"+slow-timeout = { period = "60s", terminate-after = 12 }diff --git a/src/crates/concat-host/tests/export.rs b/src/crates/concat-host/tests/export.rsindex d08af09f..eaef5b7e 100644--- a/src/crates/concat-host/tests/export.rs+++ b/src/crates/concat-host/tests/export.rs@@ -563,6 +563,12 @@ impl Studio {fn export_at(&mut self, label: &str, rate: Option<(i64, i64)>) -> Exported {self.count += 1;let label = format!("{:02} {label}", self.count).replace('/', "-");+ // Each export names itself before it starts and again once the+ // file is written, so a run that hangs (every_edit_still_exports+ // on Windows CI, at the time of writing) says which edit and which+ // half: the render, or the reopen and read-back after it. Shown+ // with --nocapture, or under a failure.+ eprintln!("{label}: exporting");let settings = self.session.settings();let spec = ExportSpec {output: self@@ -593,6 +599,7 @@ impl Studio {let cancel = AtomicBool::new(false);let written = export::run(&request, &cancel, |_| {}).unwrap_or_else(|error| panic!("{label}: the export failed: {error}"));+ eprintln!("{label}: written, reading it back");self.session.save(None).expect("saves the project");let reopened = Session::open(self.session.path(), self.session.settings())@@ -1781,18 +1788,56 @@ fn the_edges_export_too() {/// Hardware decode preferred, the way the app always sets it: the/// export decodes its sources on the platform's device where there is one-/// and reads the same picture and sound back. The preference is the-/// process's, so the other scenarios running alongside share it for the-/// moment, and must not mind.+/// and reads the same picture and sound back.+///+/// A platform can name a device the machine cannot open: Windows' CI+/// runner has WARP, a software Direct3D adapter with no video decoder, so+/// D3D11VA falls back to the CPU there, as it should, and the reader never+/// reports the device. The device is tried first on its own, by name and+/// without touching the preference; where it does not decode, the test+/// skips and says so, unless the job says the GPU tests must run+/// (`CONCAT_REQUIRE_GPU`, as cards.rs reads it), in which case it fails.+///+/// The preference is the process's. Under nextest every test is a process+/// of its own and nothing else sees it; under `cargo test` the scenarios+/// running alongside share it while this one runs, and the guard puts back+/// whatever it was before, so none of them is left with it after.#[test]fn an_export_decodes_on_the_hardware_when_preferred() {- struct Preferred;- impl Drop for Preferred {+ let scratch = Scratch::new("hardware");+ // HEVC, not the H.264 the other scenarios use: 8-bit H.264 is the one+ // stream the preference leaves on the CPU, which decodes it faster+ // (concat_media::hardware::hardware_wins).+ let sources = Sources::make_in(scratch.path(), VideoCodec::Hevc);++ if let Some(device) = HwDevice::platform_default() {+ let probe = DecodeOptions {+ hardware: concat_media::HwPolicy::Device(device),+ ..DecodeOptions::default()+ };+ let mut reader = Decoder::open(&sources.peek, &probe).expect("opens");+ reader.next_frame().expect("decodes").expect("a frame");+ if reader.hardware() != Some(device) {+ let why = format!(+ "{} is this platform's device, but it does not decode HEVC on this machine \+ (a software adapter, or no video decoder)",+ device.label()+ );+ if std::env::var_os("CONCAT_REQUIRE_GPU").is_some() {+ panic!("{why}, and CONCAT_REQUIRE_GPU is set");+ }+ eprintln!("{why}, and CONCAT_REQUIRE_GPU is not set; skipping");+ return;+ }+ }++ struct Restore(Option<HwDevice>);+ impl Drop for Restore {fn drop(&mut self) {- concat_media::set_hardware_decode(false);+ concat_media::hardware::set_hardware_device(self.0);}}- let _preferred = Preferred;+ let _restore = Restore(concat_media::hardware::hardware_device());concat_media::set_hardware_decode(true);assert_eq!(concat_media::hardware_decode(),@@ -1800,11 +1845,6 @@ fn an_export_decodes_on_the_hardware_when_preferred() {"on where the platform has a device, off where it has none");- let scratch = Scratch::new("hardware");- // HEVC, not the H.264 the other scenarios use: 8-bit H.264 is the one- // stream the preference leaves on the CPU, which decodes it faster- // (concat_media::hardware::hardware_wins).- let sources = Sources::make_in(scratch.path(), VideoCodec::Hevc);// A reader opened the way the engine opens them follows the preference// onto the device, on a machine that has one.let mut reader = Decoder::open(&sources.peek, &DecodeOptions::default()).expect("opens");diff --git a/src/crates/concat-speech/Cargo.toml b/src/crates/concat-speech/Cargo.tomlindex f04cc400..700db998 100644--- a/src/crates/concat-speech/Cargo.toml+++ b/src/crates/concat-speech/Cargo.toml@@ -73,8 +73,9 @@ whisper-rs = { version = "0.16", features = ["metal"] }# runtime in place of the plain one.ort = { workspace = true, optional = true, features = ["coreml"] }-# DirectML, so Chatterbox runs on any Direct3D 12 GPU; the same runtime-# build concat-vision's cutouts use.+# DirectML, so Chatterbox runs on any Direct3D 12 GPU where the runtime+# has the provider. The shipped Windows bundle links Microsoft's CPU build+# instead, so there it runs on the CPU; concat-vision's manifest says why.[target.'cfg(target_os = "windows")'.dependencies]ort = { workspace = true, optional = true, features = ["directml"] }diff --git a/src/crates/concat-vision/Cargo.toml b/src/crates/concat-vision/Cargo.tomlindex cdaae5d5..6b864989 100644--- a/src/crates/concat-vision/Cargo.toml+++ b/src/crates/concat-vision/Cargo.toml@@ -26,6 +26,16 @@ ort = { workspace = true, optional = true }[target.'cfg(target_vendor = "apple")'.dependencies]ort = { workspace = true, optional = true, features = ["coreml"] }+# DirectML on Windows asks for the provider; whether it is there is the+# runtime's to say. The bundles and CI link Microsoft's CPU build of ONNX+# Runtime dynamically instead of the static DirectML one ort-sys would+# download - that one is compiled for Haswell and kept the app from+# starting on older CPUs (#313), and it imports a DirectML.dll many+# machines only have an old copy of (#303). See build-app.yml's ONNX+# Runtime (Windows) step. With the CPU build the provider fails to+# register and ort carries on with the CPU one. The feature stays because+# a build with no ORT_LIB_LOCATION (a developer's) downloads by feature,+# and pyke publishes no Windows build without one.[target.'cfg(target_os = "windows")'.dependencies]ort = { workspace = true, optional = true, features = ["directml"] }diff --git a/src/rust-toolchain.toml b/src/rust-toolchain.tomlindex 36c87b3f..1d511a0f 100644--- a/src/rust-toolchain.toml+++ b/src/rust-toolchain.toml@@ -1,6 +1,9 @@-# Pinned so a checkout in two years builds the same way it does today.+# Pinned so a checkout in two years builds the same way it does today, to+# the patch release: "1.93" alone is whichever 1.93.x is newest on the day,+# so two runs a week apart could build with different compilers. CI reads+# the channel from here (the workflows' "The pinned toolchain" steps).[toolchain]-channel = "1.93"+channel = "1.93.1"components = ["rustfmt", "clippy"]# The engine crates that carry no native library build for the web too, and# CI checks that they keep doing so; see README.md.