Skip to content

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

fix(ci): vendor OpenSSL via cargo + drop x86_64 macOS from wheel matrix

chopratejas1314842b1901
diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json
index 4f7a7cefc..2b88649d8 100644
--- a/.claude-plugin/marketplace.json
+++ b/.claude-plugin/marketplace.json
@@ -5,14 +5,14 @@
},
"metadata": {
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
- "version": "0.20.14"
+ "version": "0.20.16"
},
"plugins": [
{
"name": "headroom",
"source": "./plugins/headroom-agent-hooks",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
- "version": "0.20.14",
+ "version": "0.20.16",
"author": {
"name": "Headroom Contributors",
"url": "https://github.com/chopratejas/headroom"
diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json
index 4f7a7cefc..2b88649d8 100644
--- a/.github/plugin/marketplace.json
+++ b/.github/plugin/marketplace.json
@@ -5,14 +5,14 @@
},
"metadata": {
"description": "Headroom marketplace for Claude Code and GitHub Copilot CLI plugins.",
- "version": "0.20.14"
+ "version": "0.20.16"
},
"plugins": [
{
"name": "headroom",
"source": "./plugins/headroom-agent-hooks",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
- "version": "0.20.14",
+ "version": "0.20.16",
"author": {
"name": "Headroom Contributors",
"url": "https://github.com/chopratejas/headroom"
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 61bebd0a3..738586c4c 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -178,15 +178,29 @@ jobs:
fail-fast: false
matrix:
include:
+ # Use manylinux_2_28 explicitly (instead of `auto`, which resolved
+ # to manylinux2014 / CentOS 7 / OpenSSL 1.0.2k — too old for
+ # `openssl-sys 0.9`). manylinux_2_28 is AlmaLinux 8 / glibc 2.28,
+ # the same baseline our e2e Dockerfiles use post-#360. The
+ # `openssl/vendored` Cargo feature (added in headroom-proxy)
+ # compiles OpenSSL from source so the system version doesn't
+ # matter, but pinning the floor keeps us off CentOS 7's ancient
+ # gcc/glibc surface anyway.
- os: ubuntu-latest
target: x86_64-unknown-linux-gnu
- manylinux: auto
+ manylinux: 2_28
- os: ubuntu-latest
target: aarch64-unknown-linux-gnu
manylinux: 2_28
- - os: macos-15-intel
- target: x86_64-apple-darwin
- manylinux: ""
+ # NOTE: `macos-15-intel` (x86_64-apple-darwin) is intentionally
+ # NOT in the matrix. `ort-sys 2.0.0-rc.12` (transitive via our
+ # ML compression backend) does not provide prebuilt ONNX Runtime
+ # binaries for `x86_64-apple-darwin`, and building ORT from
+ # source would add CMake + ~5 minutes per build. Apple Silicon
+ # macOS is fully covered below; Intel-mac users install the
+ # platform-independent sdist (also produced by this matrix).
+ # Tracked as a follow-up: switch to `ort-tract` or upstream a
+ # request for x86_64 macOS prebuilts.
- os: macos-14
target: aarch64-apple-darwin
manylinux: ""
@@ -206,47 +220,29 @@ jobs:
run: |
python scripts/version-sync.py --version ${{ needs.detect-version.outputs.npm_version }}
- # macOS Intel runner does NOT have OpenSSL on the openssl-sys
- # default lookup path; aarch64 finds it via /opt/homebrew/opt/openssl@3
- # but x86_64 runners use /usr/local/Cellar which openssl-sys does
- # not auto-discover. Resolve OPENSSL_DIR explicitly so openssl-sys
- # (transitive via fastembed → hf-hub → ureq → native-tls) builds
- # on both Apple silicon and Intel.
- - name: Install OpenSSL (macOS)
- if: runner.os == 'macOS'
- shell: bash
- run: |
- brew install openssl@3 pkg-config
- openssl_dir="$(brew --prefix openssl@3)"
- {
- echo "OPENSSL_DIR=$openssl_dir"
- echo "OPENSSL_LIB_DIR=$openssl_dir/lib"
- echo "OPENSSL_INCLUDE_DIR=$openssl_dir/include"
- echo "PKG_CONFIG_PATH=$openssl_dir/lib/pkgconfig"
- } >> "$GITHUB_ENV"
-
- name: Build wheels
uses: PyO3/maturin-action@v1
with:
target: ${{ matrix.target }}
args: --release --out dist --interpreter python3.10 python3.11 python3.12 python3.13
manylinux: ${{ matrix.manylinux }}
- # cibuildwheel/maturin-action spins up its own manylinux
- # container; the e2e Dockerfiles' yum install openssl-devel
- # is not inherited. Install build-time native deps inside
- # whatever distro the manylinux image happens to be (RHEL
- # family uses yum, Debian-family musllinux uses apt). Without
- # this, openssl-sys (pulled by fastembed → hf-hub → ureq →
- # native-tls) fails: "pkg-config exited with status code 1 —
- # Package openssl was not found". Only runs for Linux builds;
- # macOS sets up via the brew step above.
+ # OpenSSL is vendored at the cargo level via headroom-proxy's
+ # `openssl = { features = ["vendored"] }` dep — that compiles
+ # OpenSSL from source as part of the cargo build, so we no
+ # longer need to install system OpenSSL in the manylinux
+ # container or via Homebrew on macOS. We DO still install
+ # `perl-IPC-Cmd` because OpenSSL's vendored Configure script
+ # uses it (without it the build fails with "Can't locate
+ # IPC/Cmd.pm"). The `if`/`elif` covers RHEL-family manylinux
+ # images; we don't run on Debian-family musllinux today, but
+ # the apt branch makes it forward-compatible.
before-script-linux: |
set -euo pipefail
if command -v yum >/dev/null 2>&1; then
- yum install -y openssl-devel pkgconfig perl-IPC-Cmd
+ yum install -y perl-IPC-Cmd
elif command -v apt-get >/dev/null 2>&1; then
apt-get update
- apt-get install -y --no-install-recommends libssl-dev pkg-config
+ apt-get install -y --no-install-recommends libipc-cmd-perl
else
echo "::error::No supported package manager (yum/apt-get) found in manylinux container" >&2
exit 1
diff --git a/Cargo.lock b/Cargo.lock
index 121f92ffb..70effc6ec 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -1865,6 +1865,7 @@ dependencies = [
"humantime",
"hyper",
"hyper-util",
+ "openssl",
"pin-project-lite",
"proptest",
"reqwest",
@@ -2926,6 +2927,15 @@ version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7c87def4c32ab89d880effc9e097653c8da5d6ef28e6b539d313baaacfbafcbe"
+[[package]]
+name = "openssl-src"
+version = "300.6.0+3.6.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "a8e8cbfd3a4a8c8f089147fd7aaa33cf8c7450c4d09f8f80698a0cf093abeff4"
+dependencies = [
+ "cc",
+]
+
[[package]]
name = "openssl-sys"
version = "0.9.114"
@@ -2934,6 +2944,7 @@ checksum = "13ce1245cd07fcc4cfdb438f7507b0c7e4f3849a69fd84d52374c66d83741bb6"
dependencies = [
"cc",
"libc",
+ "openssl-src",
"pkg-config",
"vcpkg",
]
diff --git a/crates/headroom-proxy/Cargo.toml b/crates/headroom-proxy/Cargo.toml
index c35039ecf..6277c7095 100644
--- a/crates/headroom-proxy/Cargo.toml
+++ b/crates/headroom-proxy/Cargo.toml
@@ -24,6 +24,18 @@ tracing = { workspace = true }
tracing-subscriber = { version = "0.3", features = ["json", "env-filter", "fmt"] }
reqwest = { version = "0.12", default-features = false, features = ["stream", "rustls-tls", "http2"] }
tokio-tungstenite = { version = "0.24", default-features = false, features = ["connect", "rustls-tls-webpki-roots"] }
+# `hf-hub` (transitive via `fastembed`) hard-codes `native-tls` as a default
+# feature; Cargo's feature unification then enables `openssl-sys` for the
+# whole workspace despite our `reqwest`/`tokio-tungstenite`/`tokio-rustls`
+# preferences. The wheel build matrix breaks on every Linux + Intel-macOS
+# target where the manylinux container's system OpenSSL is too old or
+# absent (manylinux2014 ships 1.0.2k; aarch64 cross sysroot has none).
+# Enabling `openssl/vendored` instructs `openssl-sys` to compile OpenSSL
+# from source as part of the cargo build, removing the system-OpenSSL
+# dependency entirely. ~30s extra one-time build cost; works on every
+# target uniformly. The crate is referenced here only to enable the
+# feature via Cargo's unification — we don't use the API directly.
+openssl = { version = "0.10", features = ["vendored"] }
clap = { workspace = true, features = ["derive", "env"] }
serde = { workspace = true }
serde_json = { workspace = true }
diff --git a/plugins/headroom-agent-hooks/.claude-plugin/plugin.json b/plugins/headroom-agent-hooks/.claude-plugin/plugin.json
index 012dde945..4d8e4f107 100644
--- a/plugins/headroom-agent-hooks/.claude-plugin/plugin.json
+++ b/plugins/headroom-agent-hooks/.claude-plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "headroom",
- "version": "0.20.14",
+ "version": "0.20.16",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"author": {
"name": "Headroom Contributors",
diff --git a/plugins/headroom-agent-hooks/.github/plugin/plugin.json b/plugins/headroom-agent-hooks/.github/plugin/plugin.json
index c737a36dc..f91358182 100644
--- a/plugins/headroom-agent-hooks/.github/plugin/plugin.json
+++ b/plugins/headroom-agent-hooks/.github/plugin/plugin.json
@@ -1,6 +1,6 @@
{
"name": "headroom",
- "version": "0.20.14",
+ "version": "0.20.16",
"description": "Headroom startup hooks for Claude Code and GitHub Copilot CLI.",
"author": {
"name": "Headroom Contributors",
diff --git a/tests/test_release_workflows.py b/tests/test_release_workflows.py
index df3880507..cda9f5393 100644
--- a/tests/test_release_workflows.py
+++ b/tests/test_release_workflows.py
@@ -69,34 +69,104 @@ def test_ci_commitlint_skips_default_github_merge_commits() -> None:
assert "!startsWith(github.event.head_commit.message, 'Merge pull request ')" in content
-def test_build_wheels_installs_openssl_devel_on_linux_via_before_script() -> None:
- """The wheel matrix uses PyO3/maturin-action's manylinux container,
- which does not inherit our e2e Dockerfiles' yum installs. Without an
- explicit before-script-linux, openssl-sys (transitive via fastembed
- → hf-hub → ureq → native-tls) fails to find pkg-config'd OpenSSL
- and the Linux x86_64/aarch64 wheels never build.
+def test_headroom_proxy_vendors_openssl() -> None:
+ """`hf-hub` (transitive via `fastembed`) hard-codes `native-tls` as a
+ default feature, forcing `openssl-sys` for the entire workspace
+ despite our `reqwest`/`tokio-tungstenite`/`tokio-rustls` preferences.
+ The wheel matrix breaks on every Linux + Intel-macOS target where
+ the manylinux container's system OpenSSL is too old (manylinux2014
+ ships 1.0.2k) or the cross-compile sysroot has none (aarch64).
+ Enabling the `vendored` Cargo feature on `openssl` instructs
+ `openssl-sys` to compile OpenSSL from source — works on every
+ target uniformly.
+ """
+ cargo_toml = (ROOT / "crates" / "headroom-proxy" / "Cargo.toml").read_text(encoding="utf-8")
+
+ # Guard against a future refactor that re-removes the vendored dep.
+ assert 'openssl = { version = "0.10", features = ["vendored"] }' in cargo_toml
+
+
+def test_build_wheels_installs_perl_ipc_cmd_for_vendored_openssl() -> None:
+ """OpenSSL's vendored `Configure` script needs `IPC::Cmd`. Without
+ it the build fails with `Can't locate IPC/Cmd.pm`. The before-script
+ must install it on whichever distro the manylinux container runs
+ (RHEL family today; defensive apt branch for future Debian-family
+ musllinux builds).
"""
content = (ROOT / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
assert "before-script-linux:" in content
- assert "yum install -y openssl-devel" in content
- # apt-get fallback for non-RHEL manylinux variants
- assert "libssl-dev pkg-config" in content
+ assert "perl-IPC-Cmd" in content # yum (RHEL family)
+ assert "libipc-cmd-perl" in content # apt (Debian family fallback)
+
+
+def test_build_wheels_does_not_set_openssl_dir() -> None:
+ """`OPENSSL_DIR` (and the related `OPENSSL_LIB_DIR` /
+ `OPENSSL_INCLUDE_DIR`) DEFEATS the `vendored` feature: openssl-sys
+ will use the system OpenSSL at that path instead of compiling from
+ source. The earlier macOS hot-fix exported these env vars; with
+ vendored enabled they must NOT be set, otherwise we silently
+ regress to the system-OpenSSL path that broke originally.
+ """
+ content = (ROOT / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
+
+ # Locate the build-wheels job and assert it does not export OPENSSL_DIR.
+ bw_start = content.index("\n build-wheels:")
+ bw_end = content.index("\n collect-dist:")
+ body = content[bw_start:bw_end]
+
+ assert "OPENSSL_DIR" not in body, (
+ "build-wheels must not set OPENSSL_DIR — it disables openssl-sys's "
+ "vendored feature and reintroduces the system-OpenSSL dependency."
+ )
-def test_build_wheels_resolves_openssl_dir_explicitly_on_macos() -> None:
- """macOS Intel runners (`macos-15-intel`) keep Homebrew under
- `/usr/local/Cellar`, which `openssl-sys` does not auto-discover.
- aarch64 runners use `/opt/homebrew/opt/openssl@3` which IS on the
- default path. Setting OPENSSL_DIR explicitly fixes Intel without
- regressing aarch64.
+def test_build_wheels_matrix_excludes_intel_macos() -> None:
+ """`ort-sys 2.0.0-rc.12` (transitive via the ML compression backend)
+ has no prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`.
+ Building ORT from source would add CMake + ~5 minutes per build.
+ Apple Silicon macOS is fully covered; Intel-mac users install from
+ the platform-independent sdist this matrix also produces.
+
+ We assert against the actual matrix entry shape (`target: <triple>`
+ on a non-comment line) so explanatory comments mentioning the
+ excluded triple don't false-positive.
"""
content = (ROOT / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8")
- assert "Install OpenSSL (macOS)" in content
- assert "if: runner.os == 'macOS'" in content
- assert "brew install openssl@3" in content
- assert 'echo "OPENSSL_DIR=$openssl_dir"' in content
+ bw_start = content.index("\n build-wheels:")
+ bw_end = content.index("\n collect-dist:")
+ body = content[bw_start:bw_end]
+
+ matrix_targets: list[str] = []
+ for raw in body.splitlines():
+ stripped = raw.lstrip()
+ # Skip YAML comments — only look at real matrix-entry lines.
+ if stripped.startswith("#"):
+ continue
+ if stripped.startswith("target:"):
+ # `target: x86_64-apple-darwin` → `x86_64-apple-darwin`
+ matrix_targets.append(stripped.split(":", 1)[1].strip())
+
+ assert "aarch64-apple-darwin" in matrix_targets, "Apple Silicon must stay in the matrix"
+ assert "x86_64-unknown-linux-gnu" in matrix_targets
+ assert "aarch64-unknown-linux-gnu" in matrix_targets
+
+ # Intel macOS must NOT be a matrix entry — re-add only after switching
+ # off ort-sys (e.g., to ort-tract) or adding a CMake-from-source step.
+ assert "x86_64-apple-darwin" not in matrix_targets, (
+ f"x86_64-apple-darwin must not be a wheel-matrix target; got {matrix_targets}"
+ )
+
+ # The runner OS itself shouldn't appear as a configured `os:` either.
+ matrix_os: list[str] = []
+ for raw in body.splitlines():
+ stripped = raw.lstrip()
+ if stripped.startswith("#"):
+ continue
+ if stripped.startswith("os:"):
+ matrix_os.append(stripped.split(":", 1)[1].strip())
+ assert "macos-15-intel" not in matrix_os
def test_npm_publish_jobs_do_not_download_dist_artifact() -> None: