Skip to content

An open-source, privacy-first, self-hosted knowledge workspace where humans and AI agents work together 开源、隐私优先、自托管的知识工作空间,让人与智能体在此协作

:zap: Pipeline Windows packaging and verify signing before release builds

Danield2a756807b35
diff --git a/docs/RELEASE-VERIFICATION.zh-CN.md b/docs/RELEASE-VERIFICATION.zh-CN.md
index b775678664..5b2d910cfb 100644
--- a/docs/RELEASE-VERIFICATION.zh-CN.md
+++ b/docs/RELEASE-VERIFICATION.zh-CN.md
@@ -130,11 +130,11 @@ python -X utf8 scripts/build-release.py --appx
python -X utf8 scripts/build-release.py --appx --execute
```
-各平台安装包按 Windows、Linux、Android、鸿蒙的顺序构建。选择 Windows 时,先准备 Windows 资源,再同时编译 AMD64、ARM64 和同时选中的 Android 内核,默认最多三个任务并行;随后串行打包和签名,Android 阶段复用本次已编译的 AAR。可传入 `--kernel-jobs 2` 限制为两个任务,或用 `--kernel-jobs 1` 恢复这部分串行编译。
+各平台安装包按 Windows、Linux、Android、鸿蒙的顺序构建。选择 Windows 时,先准备 Windows 资源,再同时编译 AMD64、ARM64 和同时选中的 Android 内核,默认最多三个任务并行。AMD64 内核完成并验证后立即开始打包,无需等待 Android;ARM64 内核就绪且 AMD64 打包完成后,再打包 ARM64。Windows 安装包及签名始终串行,Android 编译可在其间继续运行,Android 阶段复用本次已编译的 AAR。可传入 `--kernel-jobs 2` 限制为两个任务,或用 `--kernel-jobs 1` 恢复这部分串行编译。
Linux 前端仅构建一次,AMD64、ARM64 内核并行编译,各自使用独立输出目录;Electron 安装包仍串行生成。同时选择 Linux 和鸿蒙时,Linux 安装包生成期间并行编译鸿蒙 ARM64 内核,Linux 阶段等待该编译完成,鸿蒙阶段验证并复用本次内核和头文件后打包。仅选择鸿蒙时仍在鸿蒙阶段编译内核。编译失败会阻止后续阶段;中断时回收后台编译进程。
-并行编译日志分别保存在本次临时构建目录的 `kernel-logs/` 中,主日志记录任务启动、结束和失败。编译失败或按 Ctrl+C 时,停止队列并回收仍在运行的编译进程;未开始的任务不再执行。Windows 资源准备阶段包含并行内核编译,完成后才进入各架构安装包阶段。
+并行编译日志分别保存在本次临时构建目录的 `kernel-logs/` 中,主日志记录任务启动、结束和失败。编译、打包失败或按 Ctrl+C 时,停止队列并回收仍在运行的编译进程;未开始的任务不再执行。Windows 资源准备阶段仅记录资源生成,各架构安装包阶段可与剩余内核编译重叠。
包含 Windows 时,先隐藏输入一次签名 PIN;预检验证通过后自动完成后续签名。仅查看计划或构建其他平台时不询问 PIN。WSL 的 SSH 认证仍在预检阶段完成,预检通过后再离开电脑。
@@ -361,7 +361,7 @@ python -X utf8 scripts/verify-release.py check D:/releases/siyuan --version 3.8.
### YubiKey PIN
-默认通过 Windows 智能卡驱动和 Authenticode 接口自动签名,支持 CNG 的 RSA、ECDSA 签名密钥。启动时隐藏输入一次 PIN,预检先验证密钥可用,再进行耗时构建;PIN 仅在进程内存中暂存,经签名子进程的标准输入传递,不写入文件或日志。每个文件使用 SHA256 签名,并检查证书、签名和时间戳;仅设备通信中断自动重试一次,PIN 错误立即停止。
+默认通过 Windows 智能卡驱动和 Authenticode 接口自动签名,支持 CNG 的 RSA、ECDSA 签名密钥。启动时隐藏输入一次 PIN,预检将已有的 AMD64 elevator EXE 复制到临时目录,验证 PIN 后对副本执行 SHA256 签名、时间戳请求和验签,通过后才开始耗时构建;原文件保持不变,临时副本在成功或失败后均删除。PIN 仅在进程内存中暂存,经签名子进程的标准输入传递,不写入文件或日志。每个发布文件仍单独签名并检查证书和时间戳;预检通过不保证后续网络或设备状态不变。失败时仅输出阶段、错误码和签名状态,不输出凭据或异常原文;仅设备通信中断自动重试一次,PIN 错误立即停止。
自动执行时可由启动程序通过进程环境变量 `SIYUAN_SIGNING_PIN` 提供 PIN,无需把密码写进命令或脚本。仍需保持签名设备连接;脚本不改变 PIN 或触摸策略,需要触摸的密钥不能完全无人值守。若驱动不支持自动认证,可添加 `--signing-ui` 沿用系统弹窗。
diff --git a/scripts/build-release.py b/scripts/build-release.py
index 7cfbf44cff..c33ef9ddf3 100644
--- a/scripts/build-release.py
+++ b/scripts/build-release.py
@@ -29,7 +29,7 @@ PLATFORMS = ("windows", "linux", "android", "harmony")
ACTIVE_RECORD = None
-def compile_kernels(commands, directory, jobs=2):
+def compile_kernels(commands, directory, jobs=2, on_complete=None):
"""Run isolated compiler processes; keep all record updates on the calling thread."""
processes = []
directory.mkdir(parents=True, exist_ok=True)
@@ -65,6 +65,8 @@ def compile_kernels(commands, directory, jobs=2):
if code:
raise BuildError(f"内核编译失败:{name}({code});日志:{path}")
print(f"内核编译完成:{name}", flush=True)
+ if on_complete is not None:
+ on_complete(name)
if pending:
time.sleep(0.1)
finally:
@@ -328,14 +330,14 @@ def read_signing_pin(value=None):
return value
-def check_signing_pin(pin, thumbprint):
+def check_signing_pin(pin, thumbprint, files=()):
# 认证请求仅通过标准输入传递,不经命令行、配置文件或发布日志。
env = dict(os.environ)
for key in ("SIYUAN_SIGNING_PIN", "SIYUAN_RELEASE_SIGN_PIN", "PSModulePath"):
env.pop(key, None)
command = ["powershell.exe", "-NoProfile", "-NonInteractive", "-ExecutionPolicy", "Bypass", "-File",
str(ROOT / "scripts/release-sign-windows.ps1")]
- request = json.dumps({"pin": pin, "thumbprint": thumbprint, "files": []}) + "\n"
+ request = json.dumps({"pin": pin, "thumbprint": thumbprint, "files": [str(path) for path in files]}) + "\n"
for attempt in range(2):
try:
result = subprocess.run(command, cwd=ROOT, env=env, input=request, encoding="utf-8", errors="replace",
@@ -346,9 +348,9 @@ def check_signing_pin(pin, thumbprint):
if result.returncode == 0:
return
if result.returncode != 20 or attempt:
- diagnostic = re.search(r'SIGNING_DIAGNOSTIC:(request|certificate|authentication):((?:0x[0-9a-fA-F]{8},?)+)', result.stderr or '')
- if diagnostic:
- print(diagnostic.group(0), flush=True)
+ diagnostics = re.findall(r'SIGNING_DIAGNOSTIC:(?:request|certificate|authentication|signing|verification):(?:0x[0-9a-fA-F]{8},?)+|SIGNING_STATUS:\d+:\d+', result.stderr or '')
+ for diagnostic in diagnostics:
+ print(diagnostic, flush=True)
raise BuildError("Windows 签名认证失败,请检查 PIN、设备和证书;PIN 错误不会自动重试")
# 仅重试已识别的设备通信中断,不重试 PIN 错误。
time.sleep(1)
@@ -494,8 +496,12 @@ class Builder:
if "windows" in self.args.platforms:
self.thumbprint = certificate_thumbprint(self.args.certificate_subject, self.args.certificate_sha1)
if self.signing_pin is not None:
- check_signing_pin(self.signing_pin, self.thumbprint)
- print(f"Windows 签名证书:{self.thumbprint};PIN 已验证,后续自动签名", flush=True)
+ with tempfile.TemporaryDirectory(prefix="siyuan-signing-probe-") as temporary:
+ probe = Path(temporary) / "signing-probe.exe"
+ shutil.copy2(ROOT / "app/elevator/elevator-amd64.exe", probe)
+ check_signing_pin(self.signing_pin, self.thumbprint, [probe])
+ verify_signature(probe, self.thumbprint)
+ print(f"Windows 签名证书:{self.thumbprint};临时 EXE 签名、时间戳和验签已通过,后续自动签名", flush=True)
else:
print(f"Windows 签名证书:{self.thumbprint};签名时请按系统提示输入 YubiKey PIN", flush=True)
if {"linux", "harmony"} & set(self.args.platforms):
@@ -540,23 +546,36 @@ class Builder:
def windows(self):
with self.stage("windows_tools"):
self.windows_tools()
- if self.args.kernel_jobs > 1:
- commands = []
- for arch in ("amd64", "arm64"):
- kernel, env = self.windows_kernel(arch)
- # Each Go invocation also limits its package parallelism.
- command = ["go", "build", "-p", str(max(1, (os.cpu_count() or 2) // self.args.kernel_jobs)),
- "-tags", "fts5 sqlcipher", "-ldflags=-s -w", "-o", kernel, "."]
- commands.append((arch, command, ROOT / "kernel", env))
- if "android" in self.args.platforms:
- aar, env, command = self.android_kernel()
- env["GOMAXPROCS"] = str(max(1, (os.cpu_count() or 2) // self.args.kernel_jobs))
- commands.insert(1, ("android", command, ROOT / "kernel", env))
- compile_kernels(commands, self.work / "kernel-logs", jobs=self.args.kernel_jobs)
- for arch in ("amd64", "arm64"):
- self.check_kernel(self.work / "kernels" / arch / "SiYuan-Kernel.exe", arch)
- self.kernels_prebuilt = True
- self.android_prebuilt = "android" in self.args.platforms
+ if self.args.kernel_jobs > 1:
+ commands = []
+ for arch in ("amd64", "arm64"):
+ kernel, env = self.windows_kernel(arch)
+ command = ["go", "build", "-p", str(max(1, (os.cpu_count() or 2) // self.args.kernel_jobs)),
+ "-tags", "fts5 sqlcipher", "-ldflags=-s -w", "-o", kernel, "."]
+ commands.append((arch, command, ROOT / "kernel", env))
+ if "android" in self.args.platforms:
+ aar, env, command = self.android_kernel()
+ env["GOMAXPROCS"] = str(max(1, (os.cpu_count() or 2) // self.args.kernel_jobs))
+ commands.insert(1, ("android", command, ROOT / "kernel", env))
+ ready = set()
+ packaged = set()
+ def completed(name):
+ if name == "android":
+ self.android_prebuilt = True
+ return
+ self.check_kernel(self.work / "kernels" / name / "SiYuan-Kernel.exe", name)
+ ready.add(name)
+ for arch, config in (("amd64", "electron-builder.yml"), ("arm64", "electron-builder-arm64.yml")):
+ if arch in packaged:
+ continue
+ if arch not in ready:
+ break
+ self.kernels_prebuilt = True
+ with self.stage("windows_" + arch):
+ self.windows_arch(arch, config)
+ packaged.add(arch)
+ compile_kernels(commands, self.work / "kernel-logs", jobs=self.args.kernel_jobs, on_complete=completed)
+ return
for arch, config in (("amd64", "electron-builder.yml"), ("arm64", "electron-builder-arm64.yml")):
with self.stage("windows_" + arch):
self.windows_arch(arch, config)
diff --git a/scripts/release-sign-windows.cjs b/scripts/release-sign-windows.cjs
index d1e7c2fdd1..da439bbd15 100644
--- a/scripts/release-sign-windows.cjs
+++ b/scripts/release-sign-windows.cjs
@@ -23,12 +23,13 @@ async function sign(options, retry = true) {
path.join(__dirname, "release-sign-windows.ps1")],
{windowsHide: true, env, stdio: ["pipe", "pipe", "pipe"]});
let output = "";
+ let diagnosticBuffer = "";
const timeout = setTimeout(() => {
child.kill();
reject(new Error("Hardware signing timed out"));
}, 120000);
child.stdout.on("data", data => { output += data.toString("utf8"); });
- child.stderr.on("data", () => {});
+ child.stderr.on("data", data => { diagnosticBuffer = (diagnosticBuffer + data.toString("utf8")).slice(-4096); });
child.stdin.on("error", () => {});
child.on("error", () => {
clearTimeout(timeout);
@@ -42,7 +43,9 @@ async function sign(options, retry = true) {
} else if (code === 20 && retry) {
setTimeout(() => { sign(options, false).then(resolve, reject); }, 1000);
} else {
- reject(new Error("Hardware signing failed; check the PIN, token, certificate and timestamp connection"));
+ const diagnostics = diagnosticBuffer.match(/SIGNING_DIAGNOSTIC:(?:request|certificate|authentication|signing|verification):(?:0x[0-9a-f]{8},?)+|SIGNING_STATUS:\d+:\d+/gi) || [];
+ reject(new Error("Hardware signing failed; check the PIN, token, certificate and timestamp connection" +
+ (diagnostics.length ? "; " + diagnostics.join("; ") : "")));
}
});
child.stdin.end(JSON.stringify({pin: signingPin, thumbprint, files: [options.path]}) + "\n");
diff --git a/scripts/release-sign-windows.ps1 b/scripts/release-sign-windows.ps1
index 00ed569595..12f9719441 100644
--- a/scripts/release-sign-windows.ps1
+++ b/scripts/release-sign-windows.ps1
@@ -85,7 +85,10 @@ try {
[SiYuanReleaseKey]::Authenticate($certificate.Handle, $payload.pin, $oid -eq '1.2.840.113549.1.1.1')
$payload.pin = $null
foreach ($file in $payload.files) {
+ $stage = "signing"
$signature = Set-AuthenticodeSignature -LiteralPath $file -Certificate $certificate -HashAlgorithm SHA256 -TimestampServer "http://timestamp.digicert.com"
+ $stage = 'verification'
+ [Console]::Error.WriteLine('SIGNING_STATUS:' + [int]$signature.Status + ':' + [int]($null -ne $signature.TimeStamperCertificate))
if ($signature.Status -ne "Valid" -or $signature.SignerCertificate.Thumbprint -ne $certificate.Thumbprint -or
$null -eq $signature.TimeStamperCertificate) {
throw "Signature or timestamp verification failed"
diff --git a/scripts/test_build_release.py b/scripts/test_build_release.py
index 556b684d35..0f6e3dee5a 100644
--- a/scripts/test_build_release.py
+++ b/scripts/test_build_release.py
@@ -76,10 +76,16 @@ class BuildTests(unittest.TestCase):
def test_windows_queue_includes_android_and_serializes_packaging(self):
self.args.kernel_jobs = 3
packed = []
- def compile(commands, directory, jobs):
+ def compile(commands, directory, jobs, on_complete):
self.assertEqual(jobs, 3)
self.assertEqual([item[0] for item in commands], ["amd64", "android", "arm64"])
self.assertEqual(len({str(item[1][item[1].index("-o") + 1]) for item in commands}), 3)
+ on_complete("amd64")
+ self.assertEqual(packed, ["amd64"])
+ self.assertFalse(getattr(self.builder, "android_prebuilt", False))
+ on_complete("arm64")
+ self.assertEqual(packed, ["amd64", "arm64"])
+ on_complete("android")
with patch.object(self.builder, "windows_tools"), patch.object(self.builder, "check_kernel"), \
patch.object(self.builder, "android_kernel", return_value=(self.root / "kernel.aar", {},
["gomobile", "bind", "-o", self.root / "kernel.aar"])), \
@@ -117,6 +123,45 @@ class BuildTests(unittest.TestCase):
self.assertNotIn("SIYUAN_SIGNING_PIN", run.call_args.kwargs["env"])
self.assertNotIn("PSModulePath", run.call_args.kwargs["env"])
+ def test_preflight_signs_and_verifies_temporary_copy(self):
+ self.args.platforms = ["windows"]
+ self.builder.signing_pin = "private-pin"
+ original = self.write(self.root / "app/elevator/elevator-amd64.exe", b"original executable")
+ probes = []
+ def sign(pin, thumbprint, files):
+ self.assertEqual(pin, "private-pin")
+ probe = files[0]
+ probes.append(probe)
+ self.assertNotEqual(probe, original)
+ self.assertEqual(probe.read_bytes(), original.read_bytes())
+ probe.write_bytes(b"signed executable")
+ def verify(path, thumbprint):
+ self.assertEqual(path.read_bytes(), b"signed executable")
+ with patch.object(build, "ROOT", self.root), patch.object(build, "source_preflight"), \
+ patch.object(build, "certificate_thumbprint", return_value="A" * 40), \
+ patch.object(build, "check_signing_pin", side_effect=sign), \
+ patch.object(build, "verify_signature", side_effect=verify):
+ self.builder.preflight()
+ self.assertEqual(original.read_bytes(), b"original executable")
+ self.assertFalse(probes[0].exists())
+
+ def test_preflight_signing_failure_is_propagated_and_copy_removed(self):
+ self.args.platforms = ["windows"]
+ self.builder.signing_pin = "private-pin"
+ self.write(self.root / "app/elevator/elevator-amd64.exe")
+ probes = []
+ def fail(pin, thumbprint, files):
+ probes.extend(files)
+ raise build.BuildError("signing probe failed")
+ with patch.object(build, "ROOT", self.root), patch.object(build, "source_preflight"), \
+ patch.object(build, "certificate_thumbprint", return_value="A" * 40), \
+ patch.object(build, "check_signing_pin", side_effect=fail), \
+ patch.object(build, "verify_signature") as verify:
+ with self.assertRaisesRegex(build.BuildError, "signing probe failed"):
+ self.builder.preflight()
+ verify.assert_not_called()
+ self.assertFalse(probes[0].exists())
+
def test_signing_preflight_retries_only_device_communication(self):
for codes in ((20, 0), (20, 20), (1,)):
results = [subprocess.CompletedProcess([], code, stdout="secret", stderr="secret") for code in codes]
diff --git a/scripts/test_release_sign_windows.cjs b/scripts/test_release_sign_windows.cjs
index 0f8916e160..b33f6133d4 100644
--- a/scripts/test_release_sign_windows.cjs
+++ b/scripts/test_release_sign_windows.cjs
@@ -31,7 +31,7 @@ function load(codes = [0, 0]) {
maximum = Math.max(maximum, active);
setImmediate(() => {
active -= 1;
- child.stderr.emit("data", Buffer.from("test-secret"));
+ child.stderr.emit("data", Buffer.from("test-secret SIGNING_DIAGNOSTIC:signing:0x8010006e SIGNING_STATUS:3:0"));
child.emit("close", codes.shift());
});
};
@@ -56,6 +56,7 @@ async function main() {
const errors = await Promise.allSettled([failed.sign(options("first.exe")), failed.sign(options("second.exe"))]);
assert.equal(failed.calls.length, 1);
assert.ok(errors.every(result => result.status === "rejected" && !result.reason.message.includes("test-secret")));
+ assert.ok(errors.every(result => result.reason.message.includes("SIGNING_DIAGNOSTIC:signing:0x8010006e")));
const retry = load([20, 0]);
await retry.sign(options("retry.exe"));
assert.equal(retry.calls.length, 2);