Skip to content

The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.

Fetch sherpa-onnx outside target/, key the cache on the build environment, and build ggml portable

Jareerc6d79535f5e5
diff --git a/.github/workflows/build-app.yml b/.github/workflows/build-app.yml
index b1a6428c..79c116a4 100644
--- a/.github/workflows/build-app.yml
+++ b/.github/workflows/build-app.yml
@@ -55,6 +55,13 @@ env:
# Oldest macOS the bundles run on. Apple silicon Macs all run newer than
# this; it is the Intel floor.
MACOSX_DEPLOYMENT_TARGET: "12.0"
+ # ggml (whisper.cpp's kernels) tunes itself to the machine it is built
+ # on unless told otherwise, and these bundles run on other machines: a
+ # runner with AVX-512 or SVE would leave a binary that faults on a CPU
+ # without them. Off, ggml compiles to the portable baseline instead -
+ # AVX2 with FMA and F16C on x86_64, NEON on ARM. whisper-rs-sys forwards
+ # every GGML_* variable to CMake as a define.
+ GGML_NATIVE: "OFF"
jobs:
app:
@@ -63,14 +70,18 @@ jobs:
fail-fast: false
matrix:
include:
+ # `ffmpeg` and `sherpa` name the target the way each project's
+ # release archives do.
- platform: macos
arch: arm64
os: macos-15
target: aarch64-apple-darwin
+ sherpa: osx-arm64
- platform: macos
arch: x86_64
os: macos-15-intel
target: x86_64-apple-darwin
+ sherpa: osx-x64
# 22.04, not latest: the binary links the build machine's glibc,
# so the oldest supported runner is what keeps it runnable on
# older distributions.
@@ -79,21 +90,25 @@ jobs:
os: ubuntu-22.04
target: x86_64-unknown-linux-gnu
ffmpeg: linux64
+ sherpa: linux-x64
- platform: linux
arch: aarch64
os: ubuntu-22.04-arm
target: aarch64-unknown-linux-gnu
ffmpeg: linuxarm64
+ sherpa: linux-aarch64
- platform: windows
arch: x86_64
os: windows-2022
target: x86_64-pc-windows-msvc
ffmpeg: win64
+ sherpa: win-x64
- platform: windows
arch: aarch64
os: windows-11-arm
target: aarch64-pc-windows-msvc
ffmpeg: winarm64
+ sherpa: win-arm64
runs-on: ${{ matrix.os }}
# The slowest target (Intel macOS, fat LTO) finishes in under half an
# hour. A job still running past this is stuck, not slow.
@@ -101,10 +116,6 @@ jobs:
steps:
- uses: actions/checkout@v7
- uses: dtolnay/rust-toolchain@stable
- - uses: Swatinem/rust-cache@v2
- with:
- workspaces: engine
- key: ${{ matrix.target }}
- name: Read the version
id: version
@@ -179,7 +190,12 @@ jobs:
# rest of the build; the cc crate hands it the Visual Studio
# environment, and the cmake crate passes it to CMake - but only
# under a Ninja generator, since the Visual Studio generator chooses
- # the compiler itself. Ninja ships on the runner image.
+ # the compiler itself. Ninja ships on the runner image. Under Ninja
+ # the cmake crate also writes CMAKE_CXX_FLAGS itself, in place of
+ # CMake's defaults, and those defaults are where /EHsc came from;
+ # clang-cl, unlike cl, rejects `try` outright without it (ggml's
+ # gguf reader uses exceptions). CXXFLAGS travels through the cc
+ # crate into those flags.
- name: FFmpeg development libraries (Windows)
if: matrix.platform == 'windows'
shell: pwsh
@@ -191,6 +207,7 @@ jobs:
Start-Process -Wait -FilePath .\llvm.exe -ArgumentList "/S"
"CC=clang-cl" | Out-File -FilePath $env:GITHUB_ENV -Append
"CXX=clang-cl" | Out-File -FilePath $env:GITHUB_ENV -Append
+ "CXXFLAGS=/EHsc" | Out-File -FilePath $env:GITHUB_ENV -Append
"CMAKE_GENERATOR=Ninja" | Out-File -FilePath $env:GITHUB_ENV -Append
} else {
choco install llvm -y --no-progress
@@ -204,14 +221,39 @@ jobs:
"FFMPEG_DIR=$dir" | Out-File -FilePath $env:GITHUB_ENV -Append
"$dir\bin" | Out-File -FilePath $env:GITHUB_PATH -Append
- # sherpa-onnx links shared on Windows (see concat-speech's manifest for
- # why static cannot work here), from a directory holding only the
- # import libraries. The separation matters: when the sys crate's build
- # script sees DLLs beside the .lib files it tries to copy them into
- # the "profile directory", located by searching OUT_DIR's ancestors
- # for one named after $PROFILE - which panics under the custom `app`
- # profile, whose directory is not called "release". The DLLs are kept
- # apart and the packaging step ships them beside the binary.
+ # ── sherpa-onnx ────────────────────────────────────────────────────
+ # The prebuilt library the sys crate would otherwise fetch for
+ # itself, fetched here instead and handed over through
+ # SHERPA_ONNX_LIB_DIR. Left to the crate, the download lands inside
+ # target/, where rust-cache's cleanup does not keep it; a restored
+ # cache then replays the build script's answer without the files
+ # it points at, and the crate fails to link the moment anything
+ # makes it recompile. Outside target/, the files are there on every
+ # run. The version is the one the lockfile resolved, so a bump
+ # cannot desynchronise.
+ - name: sherpa-onnx static libraries (macOS and Linux)
+ if: matrix.platform != 'windows'
+ shell: bash
+ run: |
+ set -euo pipefail
+ version=$(grep -A1 'name = "sherpa-onnx-sys"' engine/Cargo.lock | sed -n 's/^version = "\(.*\)"/\1/p')
+ stem="sherpa-onnx-v$version-${{ matrix.sherpa }}-static-lib"
+ echo "Downloading $stem"
+ curl -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"
+ mkdir -p sherpa
+ tar -xjf sherpa.tar.bz2 -C sherpa
+ test -d "sherpa/$stem/lib"
+ echo "SHERPA_ONNX_LIB_DIR=$PWD/sherpa/$stem/lib" >> "$GITHUB_ENV"
+
+ # On Windows it links shared (see concat-speech's manifest for why
+ # static cannot work there), from a directory holding only the
+ # import libraries. The separation matters: when the sys crate's
+ # build script sees DLLs beside the .lib files it tries to copy them
+ # into the "profile directory", located by searching OUT_DIR's
+ # ancestors for one named after $PROFILE - which panics under the
+ # custom `app` profile, whose directory is not called "release". The
+ # DLLs are kept apart and the packaging step ships them beside the
+ # binary.
#
# The archive is .tar.bz2, and Windows' own tar has no bzip2 of its
# own: it pipes through an external decoder, and on the x64 image
@@ -223,10 +265,8 @@ jobs:
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
- # The version the lockfile resolved, so a bump cannot desynchronise.
$version = (Select-String -Path engine\Cargo.lock -Pattern 'name = "sherpa-onnx-sys"' -Context 0,1).Context.PostContext[0] -replace '.*"(.*)".*', '$1'
- $arch = if ("${{ matrix.arch }}" -eq "aarch64") { "win-arm64" } else { "win-x64" }
- $stem = "sherpa-onnx-v$version-$arch-shared-MT-Release-lib"
+ $stem = "sherpa-onnx-v$version-${{ matrix.sherpa }}-shared-MT-Release-lib"
Write-Host "Downloading $stem"
curl.exe -fsSL --retry 5 --retry-all-errors -o sherpa.tar.bz2 "https://github.com/k2-fsa/sherpa-onnx/releases/download/v$version/$stem.tar.bz2"
Write-Host "Unpacking"
@@ -239,6 +279,17 @@ jobs:
"SHERPA_DLL_DIR=$PWD\sherpa-dlls" | Out-File -FilePath $env:GITHUB_ENV -Append
# ── Build ──────────────────────────────────────────────────────────
+ # The cache comes after the environment is set, not before: its key
+ # is made from the compiler and linker variables in force (RUSTFLAGS,
+ # CC, CXX, CMAKE_*, and GGML_* added to the default set), so a change
+ # to any of them is a new key rather than a stale target/ served
+ # under the old one.
+ - uses: Swatinem/rust-cache@v2
+ with:
+ workspaces: engine
+ key: ${{ matrix.target }}
+ env-vars: CARGO CC CFLAGS CXX CMAKE RUST GGML
+
# The shipping profile: fat LTO, panic=abort, stripped. See the
# workspace Cargo.toml for what each knob costs.
#