AI-native knowledge base & Evernote/Obsidian alternative with instant sync & native MCP. Free on Cloudflare or self-hosted via Docker.
HTML ai-agentcloudflarecloudflare-d1cloudflare-r2cloudflare-workersdockerelectronevernoteevernote-alternativeknowledge-basemcpnote-takingobsidian-alternativeoffline-firstpersonal-knowledge-basepwaself-hostedsqlitetiptapweb-clipper
fix(extension): verify review source archives before store submission
tianma-if588a23553b81
diff --git a/.github/workflows/extension-build.yml b/.github/workflows/extension-build.ymlindex 3039b38d..bbdc9525 100644--- a/.github/workflows/extension-build.yml+++ b/.github/workflows/extension-build.yml@@ -4,10 +4,16 @@ on:pull_request:paths:- "apps/extension/**"+ - "apps/web/src/components/ui/**"+ - "apps/web/src/lib/utils.ts"+ - "tsconfig.json"+ - "bunfig.toml"+ - "patches/**"- "package.json"- "bun.lock"- "scripts/publish-web-clipper.mjs"- "scripts/publish-web-clipper.test.mjs"+ - "scripts/verify-extension-source.mjs"- ".github/workflows/extension-build.yml"- ".github/workflows/extension-store.yml"push:@@ -15,10 +21,16 @@ on:- mainpaths:- "apps/extension/**"+ - "apps/web/src/components/ui/**"+ - "apps/web/src/lib/utils.ts"+ - "tsconfig.json"+ - "bunfig.toml"+ - "patches/**"- "package.json"- "bun.lock"- "scripts/publish-web-clipper.mjs"- "scripts/publish-web-clipper.test.mjs"+ - "scripts/verify-extension-source.mjs"- ".github/workflows/extension-build.yml"- ".github/workflows/extension-store.yml"workflow_dispatch:@@ -31,13 +43,15 @@ jobs:if: github.repository == 'tianma-if/edgeever'name: Build Chromium and Firefoxruns-on: ubuntu-latest- timeout-minutes: 10+ timeout-minutes: 20steps:- name: Checkoutuses: actions/checkout@v5- name: Set up Bunuses: oven-sh/setup-bun@v2+ with:+ bun-version-file: package.json- name: Install dependenciesrun: bun install --frozen-lockfile@@ -56,3 +70,6 @@ jobs:- name: Lint Firefox extensionrun: bun run lint:extension:firefox++ - name: Rebuild Firefox from the review source archive+ run: bun scripts/verify-extension-source.mjsdiff --git a/.github/workflows/extension-store.yml b/.github/workflows/extension-store.ymlindex 3f70c4f3..0196f7a6 100644--- a/.github/workflows/extension-store.yml+++ b/.github/workflows/extension-store.yml@@ -58,6 +58,8 @@ jobs:- name: Set up Bunuses: oven-sh/setup-bun@v2+ with:+ bun-version-file: package.json- name: Install dependenciesrun: bun install --frozen-lockfilediff --git a/apps/extension/SOURCE_BUILD.md b/apps/extension/SOURCE_BUILD.mdindex 7015c41d..ae8ce12d 100644--- a/apps/extension/SOURCE_BUILD.md+++ b/apps/extension/SOURCE_BUILD.md@@ -3,11 +3,12 @@This archive is the reviewable source for one Firefox Add-ons submission.The archive root is `edgeever-source`. It contains the extension source,the root package manifest, the Bun lockfile, workspace package manifests,+the Web UI component sources imported by the extension,and the patch files `bun install --frozen-lockfile` requires.Requirements:-- Bun 1.3.14 or later+- The Bun version in the root `package.json` `packageManager` field (currently 1.3.14)- Node.js 20 or later, required by `web-ext`From the archive root:@@ -24,3 +25,7 @@ The Firefox package is written to `apps/extension/web-ext-artifacts`.The extension reads a page only after the user chooses a clip action, thensends that content to the EdgeEver instance configured in the extensionsettings. Review steps are in `apps/extension/FIREFOX_STORE_LISTING.md`.++Before store submission, the packaging command extracts this exact archive into+a clean temporary directory, installs with the frozen lockfile, and rebuilds+Firefox. Every rebuilt file must match the submission build by SHA-256.diff --git a/docs/extension-store.md b/docs/extension-store.mdindex 8ef39f80..a7c050d5 100644--- a/docs/extension-store.md+++ b/docs/extension-store.md@@ -2,8 +2,8 @@Chrome, Edge, and Firefox listings are submitted by the official repositoryworkflow **Submit Web Clipper**. This is separate from `bun run release`.-A product release does not upload the browser extension, and a store review-does not change a GitHub Release.+A stable GitHub Release triggers the store workflow asynchronously. Store+submission does not block the GitHub Release, and review does not change it.Edge users install the Chrome Web Store item`gjadpfmanienmlofajibkfkkpfdkclgo`. One Chrome submission covers that item.@@ -15,6 +15,13 @@ It does not approve the listing. Users stay on the last approved version untilChrome and Mozilla each finish review. An uploaded version number cannot bereused. Increase `apps/extension/package.json` before submitting changes.+Before either store is contacted, the command extracts the Firefox review source+ZIP into a clean directory, installs with the frozen lockfile, and rebuilds it.+Every output file must match the submission build by SHA-256; a mismatch stops+the command. The archive includes the shared Web UI components the extension+imports. Build CI watches these dependencies and runs the same rebuild check.+Both workflows use the Bun version from the checked-out root `package.json`.+## Run itOn https://github.com/tianma-if/edgeever/actions/workflows/extension-store.yml,diff --git a/docs/extension-store.zh-CN.md b/docs/extension-store.zh-CN.mdindex 351f4895..d9d76bb9 100644--- a/docs/extension-store.zh-CN.md+++ b/docs/extension-store.zh-CN.md@@ -1,8 +1,8 @@# 网页剪藏插件商店提交Chrome、Edge 和 Firefox 的上架由官方仓库工作流 **Submit Web Clipper** 提交。-它和 `bun run release` 分开。产品正式发布不会上传浏览器插件,商店审核也不会-改变 GitHub Release。+它和 `bun run release` 分开。正式 GitHub Release 发布后会异步触发商店工作流;+商店提交不阻塞 GitHub Release,审核结果也不会改变已发布的 Release。Edge 用户安装的是 Chrome 网上应用店商品`gjadpfmanienmlofajibkfkkpfdkclgo`。一次 Chrome 提交覆盖这个商品。Firefox@@ -13,6 +13,12 @@ Edge 用户安装的是 Chrome 网上应用店商品最后一个已通过的版本,直到 Chrome 和 Mozilla 分别审核完成。已经上传过的版本号不能重复使用。提交改动前要先提高 `apps/extension/package.json` 里的版本。+调用任何商店接口前,命令会把 Firefox 审核源码 ZIP 解压到干净目录,使用冻结的+锁文件安装依赖并重新构建。每个产物文件的 SHA-256 必须与提交版本一致,否则停止+提交。源码包包含插件引用的 Web 共享 UI 组件;构建 CI 会监听这些依赖的变化,+并执行相同的重建检查。两个工作流均使用检出代码中根目录 `package.json` 指定的+Bun 版本。+## 运行打开 https://github.com/tianma-if/edgeever/actions/workflows/extension-store.yml,diff --git a/scripts/publish-web-clipper.mjs b/scripts/publish-web-clipper.mjsindex 6d3011f4..d2d04748 100644--- a/scripts/publish-web-clipper.mjs+++ b/scripts/publish-web-clipper.mjs@@ -1,5 +1,5 @@import { spawnSync } from "node:child_process";-import { createHmac, createSign, randomUUID } from "node:crypto";+import { createHash, createHmac, createSign, randomUUID } from "node:crypto";import {appendFileSync,cpSync,@@ -36,6 +36,13 @@ const ROOT_SOURCE_FILES = ["bunfig.toml","tsconfig.json",];+export const SHARED_EXTENSION_SOURCE_FILES = [+ "apps/web/src/components/ui/button.tsx",+ "apps/web/src/components/ui/input.tsx",+ "apps/web/src/components/ui/button-tooltip.tsx",+ "apps/web/src/components/ui/tooltip.tsx",+ "apps/web/src/lib/utils.ts",+];const usage = `Usage:bun run publish:extension -- [options]@@ -547,7 +554,7 @@ const waitForAmoUpload = async (client, uuid) => {throw new Error("Timed out waiting for Firefox Add-ons validation.");};-const copySourceArchive = (repoRoot, destinationRoot) => {+export const copySourceArchive = (repoRoot, destinationRoot) => {mkdirSync(destinationRoot, { recursive: true });for (const file of ROOT_SOURCE_FILES) {cpSync(join(repoRoot, file), join(destinationRoot, file));@@ -570,6 +577,72 @@ const copySourceArchive = (repoRoot, destinationRoot) => {mkdirSync(dirname(target), { recursive: true });cpSync(join(repoRoot, manifest), target);}+ for (const file of SHARED_EXTENSION_SOURCE_FILES) {+ const target = join(destinationRoot, file);+ mkdirSync(dirname(target), { recursive: true });+ cpSync(join(repoRoot, file), target);+ }+};++export const createSourceArchive = (repoRoot, sourceZip) => {+ const staging = mkdtempSync(join(tmpdir(), "edgeever-web-clipper-source-"));+ try {+ copySourceArchive(repoRoot, join(staging, "edgeever-source"));+ rmSync(sourceZip, { force: true });+ runCommand("zip", ["-q", "-r", "-X", sourceZip, "edgeever-source", "-x", "*.DS_Store"], { cwd: staging });+ } finally {+ rmSync(staging, { recursive: true, force: true });+ }+};++const buildFileHashes = (directory, prefix = "") => {+ const files = new Map();+ for (const entry of readdirSync(join(directory, prefix), { withFileTypes: true })) {+ const path = join(prefix, entry.name);+ if (entry.isDirectory()) {+ for (const [file, hash] of buildFileHashes(directory, path)) files.set(file, hash);+ } else {+ files.set(path, createHash("sha256").update(readFileSync(join(directory, path))).digest("hex"));+ }+ }+ return files;+};++export const assertIdenticalBuildDirectories = (referenceDirectory, rebuiltDirectory) => {+ const expected = buildFileHashes(referenceDirectory);+ const rebuilt = buildFileHashes(rebuiltDirectory);+ const different = [...new Set([...expected.keys(), ...rebuilt.keys()])]+ .filter((file) => expected.get(file) !== rebuilt.get(file)).sort();+ if (different.length) throw new Error(`Source archive rebuild differs: ${different.join(", ")}`);+ if (!expected.size) throw new Error("Cannot verify an empty extension build");+ return expected.size;+};++export const sourceRebuildEnvironment = (environment = process.env) => ({+ ...Object.fromEntries(Object.entries(environment).filter(([key]) =>+ !/^(CHROME_WEB_STORE_|AMO_|EDGE_EVER_REVIEW_)/.test(key) && !["GITHUB_TOKEN", "GH_TOKEN"].includes(key))),+ NODE_PATH: "",+ NODE_OPTIONS: "",+});++export const verifySourceArchive = ({ sourceZip, referenceDirectory }) => {+ const staging = mkdtempSync(join(tmpdir(), "edgeever-web-clipper-rebuild-"));+ try {+ runCommand("unzip", ["-q", sourceZip, "-d", staging]);+ const sourceRoot = join(staging, "edgeever-source");+ if (!existsSync(join(sourceRoot, "package.json"))) throw new Error("Source archive is missing its root package.json");+ if (existsSync(join(sourceRoot, "node_modules")) || existsSync(join(sourceRoot, "apps/extension/dist-firefox"))) {+ throw new Error("Source rebuild must start without dependencies or built assets");+ }+ const options = { cwd: sourceRoot, env: sourceRebuildEnvironment() };+ runCommand("bun", ["install", "--frozen-lockfile"], options);+ runCommand("bun", ["run", "build:extension:firefox"], options);+ const count = assertIdenticalBuildDirectories(referenceDirectory, join(sourceRoot, "apps/extension/dist-firefox"));+ console.log(`Firefox source archive independently rebuilt: ${count} files match SHA-256.`);+ return count;+ } finally {+ rmSync(staging, { recursive: true, force: true });+ }};const zipDirectoryContents = (directory, archivePath) => {@@ -630,16 +703,9 @@ const buildPackages = (version) => {"apps/extension/web-ext-artifacts",`edgeever_web_clipper-${version}-source.zip`,);- const staging = mkdtempSync(join(tmpdir(), "edgeever-web-clipper-"));- try {- copySourceArchive(REPO_ROOT, join(staging, "edgeever-source"));- rmSync(sourceZip, { force: true });- runCommand("zip", ["-r", "-X", sourceZip, "edgeever-source", "-x", "*.DS_Store"], {- cwd: staging,- });- } finally {- rmSync(staging, { recursive: true, force: true });- }+ createSourceArchive(REPO_ROOT, sourceZip);+ // This runs before either store is contacted, including in --dry-run mode.+ verifySourceArchive({ sourceZip, referenceDirectory: join(REPO_ROOT, "apps/extension/dist-firefox") });return { version, chromiumZip, firefoxZip, sourceZip };};diff --git a/scripts/publish-web-clipper.test.mjs b/scripts/publish-web-clipper.test.mjsindex 29d92378..7f8adf5e 100644--- a/scripts/publish-web-clipper.test.mjs+++ b/scripts/publish-web-clipper.test.mjs@@ -1,5 +1,5 @@import { createHmac, createVerify, generateKeyPairSync } from "node:crypto";-import { mkdtempSync, readFileSync, writeFileSync } from "node:fs";+import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join } from "node:path";import { describe, expect, test } from "bun:test";@@ -8,10 +8,12 @@ import {amoValidationErrors,assertCredentials,assertOfficialRepository,+ assertIdenticalBuildDirectories,buildApprovalNotes,chromeSubmissionPlan,chromeUploadFinished,compareVersions,+ copySourceArchive,createAmoJwt,createChromeClient,createFirefoxClient,@@ -24,6 +26,8 @@ import {parsePublishArgs,resolveReleaseNotes,shouldCopyExtensionSource,+ SHARED_EXTENSION_SOURCE_FILES,+ sourceRebuildEnvironment,submitChrome,submitFirefox,workspacePackageManifests,@@ -262,6 +266,50 @@ describe("web clipper store command", () => {expect(manifests).not.toContain("apps/extension/package.json");});+ test("review source contains shared UI imports without copying the whole Web app", () => {+ const staging = mkdtempSync(join(tmpdir(), "edgeever-source-test-"));+ try {+ copySourceArchive(new URL("..", import.meta.url).pathname, staging);+ for (const file of SHARED_EXTENSION_SOURCE_FILES) {+ expect(readFileSync(join(staging, file), "utf8")).toBe(readFileSync(new URL(`../${file}`, import.meta.url), "utf8"));+ }+ expect(existsSync(join(staging, "apps/web/src/components/EditorPane.tsx"))).toBe(false);+ expect(existsSync(join(staging, ".env.local"))).toBe(false);+ expect(existsSync(join(staging, "apps/extension/dist"))).toBe(false);+ expect(existsSync(join(staging, "apps/extension/node_modules"))).toBe(false);+ } finally { rmSync(staging, { recursive: true, force: true }); }+ });++ test("source rebuild audit rejects missing, extra and changed artifacts", () => {+ const staging = mkdtempSync(join(tmpdir(), "edgeever-rebuild-test-"));+ const expected = join(staging, "expected"), rebuilt = join(staging, "rebuilt");+ mkdirSync(expected); mkdirSync(rebuilt);+ try {+ writeFileSync(join(expected, "manifest.json"), "manifest");+ expect(() => assertIdenticalBuildDirectories(expected, rebuilt)).toThrow("manifest.json");+ writeFileSync(join(rebuilt, "manifest.json"), "changed");+ expect(() => assertIdenticalBuildDirectories(expected, rebuilt)).toThrow("manifest.json");+ writeFileSync(join(rebuilt, "manifest.json"), "manifest");+ expect(assertIdenticalBuildDirectories(expected, rebuilt)).toBe(1);+ writeFileSync(join(rebuilt, "extra.js"), "extra");+ expect(() => assertIdenticalBuildDirectories(expected, rebuilt)).toThrow("extra.js");+ } finally { rmSync(staging, { recursive: true, force: true }); }+ });++ test("extension CI covers shared UI and verifies the review archive on the official repository", () => {+ const build = readFileSync(new URL("../.github/workflows/extension-build.yml", import.meta.url), "utf8");+ expect(build.match(/"apps\/web\/src\/components\/ui\/\*\*"/g)).toHaveLength(2);+ expect(build.match(/"apps\/web\/src\/lib\/utils.ts"/g)).toHaveLength(2);+ expect(build).toContain("if: github.repository == 'tianma-if/edgeever'");+ expect(build).toContain("run: bun scripts/verify-extension-source.mjs");+ expect(build).toContain("bun-version-file: package.json");+ });++ test("independent builds do not inherit store credentials or external module paths", () => {+ const environment = sourceRebuildEnvironment({ PATH: "/usr/bin", HOME: "/tmp/example", CHROME_WEB_STORE_CLIENT_SECRET: "secret", AMO_JWT_SECRET: "secret", EDGE_EVER_REVIEW_API_TOKEN: "secret", GITHUB_TOKEN: "secret", GH_TOKEN: "secret", NODE_PATH: "/original/node_modules", NODE_OPTIONS: "--require external.js" });+ expect(environment).toEqual({ PATH: "/usr/bin", HOME: "/tmp/example", NODE_PATH: "", NODE_OPTIONS: "" });+ });+test("signs Firefox and Chrome credentials", () => {const amo = createAmoJwt({issuer: "user:1:1",diff --git a/scripts/verify-extension-source.mjs b/scripts/verify-extension-source.mjsnew file mode 100644index 00000000..43aa2e44--- /dev/null+++ b/scripts/verify-extension-source.mjs@@ -0,0 +1,15 @@+import { mkdtempSync, rmSync } from "node:fs";+import { tmpdir } from "node:os";+import { join } from "node:path";+import { fileURLToPath } from "node:url";+import { createSourceArchive, verifySourceArchive } from "./publish-web-clipper.mjs";++const repoRoot = fileURLToPath(new URL("..", import.meta.url));+const staging = mkdtempSync(join(tmpdir(), "edgeever-extension-source-check-"));+try {+ const sourceZip = join(staging, "source.zip");+ createSourceArchive(repoRoot, sourceZip);+ verifySourceArchive({ sourceZip, referenceDirectory: join(repoRoot, "apps/extension/dist-firefox") });+} finally {+ rmSync(staging, { recursive: true, force: true });+}