The open-source agent harness - the runtime layer that turns an LLM into a working agent.
[1/N] Release Workflows: Publish harness from root instead of dist (#188)
Chirag Jain80432817b279
diff --git a/.github/workflows/release.yml b/.github/workflows/release.ymlindex da8598f72..582cf1527 100644--- a/.github/workflows/release.yml+++ b/.github/workflows/release.yml@@ -5,7 +5,7 @@ on:permissions:contents: read- id-token: write # required for npm trusted publishing (OIDC)+ id-token: write # required for trusted publishing (OIDC)jobs:publish:@@ -20,10 +20,6 @@ jobs:node-version: 22registry-url: https://registry.npmjs.org- # OIDC trusted publishing requires npm >= 11.5.1; node 22 bundles npm 10.x.- # npm 11 also refuses prerelease publishes without an explicit --tag.- - run: npm install -g npm@latest-- run: pnpm install --frozen-lockfile- run: pnpm build- run: pnpm test@@ -41,19 +37,21 @@ jobs:fi# Library first: utils pins workspace:* → this concrete version on publish.- # Prereleases (e.g. 0.1.9-rc.1) must pass --tag (npm 11); first preid → dist-tag (rc).+ # Prereleases (e.g. 0.1.9-rc.1) pass --tag explicitly; first preid → dist-tag (rc).+ # --no-git-checks: this job checks out a tag (detached HEAD), so pnpm's default+ # "current branch is main/master" check would otherwise always fail here.- name: Publish @truefoundry/utils-core- working-directory: packages/harness/dist+ working-directory: packages/harnessrun: |VERSION="$(node -p "require('./package.json').version")"if [[ "$VERSION" == *-* ]]; thenTAG="${VERSION#*-}"TAG="${TAG%%.*}"echo "Publishing $VERSION with --tag $TAG"- npm publish --access public --tag "$TAG"+ pnpm publish --access public --tag "$TAG" --no-git-checkselseecho "Publishing $VERSION (latest)"- npm publish --access public+ pnpm publish --access public --no-git-checksfi# TODO(oss): add --provenance once the repo is public. Do NOT add it now:# provenance publicly attests the source repo and commit.diff --git a/RELEASING.md b/RELEASING.mdindex c6eeb95a8..40ce13fb7 100644--- a/RELEASING.md+++ b/RELEASING.md@@ -4,7 +4,7 @@ Two public packages ship from this repo:| Package | Source | How it publishes || ------------------------- | ------------------ | --------------------------------------------------------------- |-| `@truefoundry/utils-core` | `packages/harness` | From staged `packages/harness/dist` (library) |+| `@truefoundry/utils-core` | `packages/harness` | From package root (`dist/`) || `@truefoundry/utils` | `packages/server` | From package root (`dist/`, including `dist/_frontend/` for UI) |The git tag `v*` must match **`packages/harness` (`@truefoundry/utils-core`) version**.@@ -33,21 +33,37 @@ open-source release happens later — every deferred item carries a> tsconfig.build.json), which embeds/references original TypeScript source in> the published tarball — this is a deliberate, accepted trade-off for now.-## Publishing model (openai-node style)--The package publishes **from `packages/harness/dist`**, not the package-folder. The build compiles every `src/**/*.ts` to its own `.js` (CJS) +-`.mjs` (ESM) + `.d.ts` triple and generates `dist/package.json` with-dist-relative paths (`scripts/make-dist-package-json.mjs`), so the tarball-root is the compiled file tree. Consequences:--- Deep imports mirror the source tree with no `dist/` segment:- `@truefoundry/utils-core/core/llm/LLMTypes`, `.../core/runtime/contextUtils`.-- Legacy `moduleResolution: "node"` consumers (the gateway) resolve subpaths- as literal file lookups — no `exports`/`typesVersions` support needed.+## Publishing model++The package publishes from the package root (`packages/harness`), same as+every other package in this repo. The build compiles every `src/**/*.ts` to+its own `.js` (CJS) + `.mjs` (ESM) + `.d.ts` triple under `dist/`, and+`package.json`'s `exports` map points at `./dist/...` paths; `files: ["dist"]`+scopes the published tarball to just the compiled output. Consequences:++- Deep imports keep the same specifier as before, e.g.+ `@truefoundry/utils-core/core/llm/LLMTypes` — the package's own `"./*"`+ export pattern maps that to `./dist/core/llm/LLMTypes.{d.ts,js,mjs}`+ internally, so no consumer using exports-aware resolution (`bundler`,+ `node16`, `nodenext`) needs to change anything. Only a genuinely legacy+ `moduleResolution: "node"`/`"node10"` consumer (which ignores `package.json`+ `exports` entirely and does literal on-disk path lookups) would need to add+ a `/dist/` segment or move to an exports-aware mode.- `require()` and `import` both work (`.js` is CJS, `.mjs` is ESM).- The curated barrels (`.`, `./core`, `./agent-session`) remain the publicAPI; deep imports are the escape hatch for internals.+- `package.json` is published as-is (no staged rewrite), and it also carries a+ `"trueforge-dev"` export condition pointing at `./src/*.ts`, used only+ for dist-free host dev inside this monorepo (`packages/server`'s+ `NODE_OPTIONS='--conditions=trueforge-dev'` scripts). `src/` is **not**+ in the tarball (`files: ["dist"]`), so that condition name must never be a+ string a consumer's tooling activates automatically — it deliberately isn't+ `"development"`, which Vite/webpack default to auto-activating in dev mode+ with no consumer opt-in (Vite's default `resolve.conditions` substitutes the+ literal string `"development"` whenever `NODE_ENV !== 'production'`). A+ genuinely custom condition name like this one is only ever reachable by an+ explicit `--conditions=trueforge-dev` opt-in, which no external+ consumer would ever set — so it ships as truly inert dead weight.## Per-release flow@@ -78,15 +94,16 @@ root is the compiled file tree. Consequences:3. **CI publishes automatically.** `.github/workflows/release.yml` installs,builds, tests, verifies the tag matches `packages/harness/package.json`, then- runs `npm publish` **from `packages/harness/dist`**. (The `@truefoundry/utils`+ runs `pnpm publish` from `packages/harness`. (The `@truefoundry/utils`publish is deferred — see the note above.)- Auth is trusted publishing (OIDC — no `NPM_TOKEN`). Watch the repo Actions tab.+ Auth is trusted publishing (OIDC — no `NPM_TOKEN`). pnpm 11 implements the+ OIDC token exchange natively (no npm CLI involved). Watch the repo Actions tab.- **Dist-tags:** npm 11 requires an explicit `--tag` for prereleases. CI derives- it from the semver prerelease id (`0.2.0-rc.1` → `--tag rc`). Stable releases- publish to `latest`. Install with `npx @truefoundry/utils@rc` or- `@0.2.0-rc.1`; bare `npx @truefoundry/utils` stays on `latest`.+ **Dist-tags:** CI passes an explicit `--tag` for prereleases, derived from the+ semver prerelease id (`0.2.0-rc.1` → `--tag rc`). Stable releases publish to+ `latest`. Install with `npx @truefoundry/utils@rc` or `@0.2.0-rc.1`; bare+ `npx @truefoundry/utils` stays on `latest`.4. **Bump the pinned version in the gateway.** Pin exact versions (no `^`)during the fast 0.x churn:@@ -110,7 +127,7 @@ For tight loops, skip the publish round-trip:```bashpnpm clean && pnpm build && pnpm standalone:start# or pack only:-cd packages/harness && pnpm build && cd dist && pnpm pack+cd packages/harness && pnpm build && pnpm packcd packages/server && pnpm pack```@@ -119,7 +136,7 @@ Publish a real version when CI or teammates need it.## Troubleshooting-- **Publish fails requiring a tag**: prerelease versions need `--tag` (npm 11).+- **Publish fails requiring a tag**: prerelease versions need `--tag`.CI handles this; for local publishes use e.g. `pnpm publish --tag rc`.- **Publish fails with 403/E403**: version already published (npm versionsare immutable — bump and re-tag), or the trusted publisher config doesn't@@ -131,24 +148,24 @@ Publish a real version when CI or teammates need it.`frontend` before `@truefoundry/utils`; the release job fails closed if thecopy is absent.- **OIDC/auth error in the publish step**: trusted publishing requires- npm >= 11.5.1; the workflow upgrades npm globally before publishing —- check that step ran.+ pnpm >= 11.0.7 (native OIDC token exchange; `pnpm/action-setup@v4` reads the+ pinned version from root `packageManager`) and the npmjs.com trusted+ publisher config (repo + workflow filename + ref) matching exactly.## Deferred to the real OSS release — grep for `TODO(oss)`Done: root LICENSE + MIT license fields, CONTRIBUTING, SECURITY,CODE_OF_CONDUCT, issue/PR templates.-| Item | Where |-| ------------------------------------------------------------------------------- | ------------------------------------------------------- |-| Ship README (guard lives in make-dist-package-json.mjs) | packages/harness/scripts |-| Add repository/homepage/bugs/keywords metadata | packages/harness/package.json |-| Add `--provenance` to publish | .github/workflows/release.yml |-| Replace `internal.devtest.truefoundry.tech` URLs | packages/server/src/config |-| Private-repo links in shipped sandbox scripts | packages/harness/src/core/sandbox/scripts/mcp_client.py |-| Sourcemaps/declarationMap decision | packages/harness/tsup.config.ts, tsconfig.build.json |-| CODEOWNERS / CI for external (fork) PRs | .github/ |-| Secret-scan history, enable secret scanning + push protection, flip repo public | GitHub settings |+| Item | Where |+| ---------------------------------------------------------------------------------- | ------------------------------------------------------- |+| Ship README (currently no `README.md` in `packages/harness`, npm page stays blank) | packages/harness |+| Add repository/homepage/bugs/keywords metadata | packages/harness/package.json |+| Add `--provenance` to publish | .github/workflows/release.yml |+| Private-repo links in shipped sandbox scripts | packages/harness/src/core/sandbox/scripts/mcp_client.py |+| Sourcemaps/declarationMap decision | packages/harness/tsup.config.ts, tsconfig.build.json |+| CODEOWNERS / CI for external (fork) PRs | .github/ |+| Secret-scan history, enable secret scanning + push protection, flip repo public | GitHub settings |---@@ -183,7 +200,7 @@ The dispatch commit SHA is the image tag. The workflow:2. Watch the run. The image and chart land in JFrog; the job summary prints theimage URI.-> npm publish of `@truefoundry/utils-core` is a separate pipeline+> Publishing `@truefoundry/utils-core` (via `pnpm publish`) is a separate pipeline> (`release.yml`) triggered by a `vX.Y.Z` GitHub Release tag that must match> `packages/harness/package.json`. It is independent of this image/chart> workflow.diff --git a/packages/harness/AGENTS.md b/packages/harness/AGENTS.mdindex ba0cd3fc0..19a9de949 100644--- a/packages/harness/AGENTS.md+++ b/packages/harness/AGENTS.md@@ -1,3 +1,3 @@-- Workspace source MUST remain ESM so `exports.development` can load `src/` without a build. Published `dist/package.json` MUST remain CommonJS and omit development conditions while exposing CJS `.js` and ESM `.mjs` outputs.+- Workspace source MUST remain ESM so `exports["trueforge-dev"]` can load `src/` without a build. That condition name MUST NOT be `"development"` (or `"production"`) — the package publishes from its own root with `files: ["dist"]`, so `src/` is never in the tarball, and Vite/webpack default to auto-activating a literal `"development"` condition in dev mode with no consumer opt-in, which would break resolution for any such consumer. `dist/` output MUST expose CJS `.js` and ESM `.mjs` outputs alongside `.d.ts`.- Top-level public package surfaces (`core`, `agent-session`, and other exported entrypoints) MAY use optional (`?`) properties for caller ergonomics.- Within this package, records, snapshots, store INPUT types, and other internal contracts MUST NOT use `?`. Absence MUST be an explicit required `| null` or `| undefined` field; prefer `null` for domain absence and `undefined` only when update/patch semantics must distinguish “not provided” from “clear.” Public surfaces adapt by supplying those explicit absences when calling inward. This forces internal object literals to be exhaustive under `tsc`, preventing forgotten fields and silent data loss.diff --git a/packages/harness/package.json b/packages/harness/package.jsonindex 16534048c..e028611de 100644--- a/packages/harness/package.json+++ b/packages/harness/package.json@@ -1,8 +1,8 @@{"name": "@truefoundry/utils-core",- "version": "0.1.9",- "description": "TrueFoundry agent core and session library",- "author": "Truefoundry",+ "version": "0.1.10-rc.1",+ "description": "TrueFoundry Utils Core",+ "author": "TrueFoundry","license": "MIT","private": false,"type": "module",@@ -11,7 +11,7 @@"types": "./dist/index.d.ts","exports": {".": {- "development": {+ "trueforge-dev": {"types": "./src/index.ts","default": "./src/index.ts"},@@ -20,7 +20,7 @@"default": "./dist/index.mjs"},"./core": {- "development": {+ "trueforge-dev": {"types": "./src/core/index.ts","default": "./src/core/index.ts"},@@ -29,7 +29,7 @@"default": "./dist/core/index.mjs"},"./agent-session": {- "development": {+ "trueforge-dev": {"types": "./src/agent-session/index.ts","default": "./src/agent-session/index.ts"},@@ -38,7 +38,7 @@"default": "./dist/agent-session/index.mjs"},"./request-reply": {- "development": {+ "trueforge-dev": {"types": "./src/request-reply/index.ts","default": "./src/request-reply/index.ts"},@@ -55,7 +55,7 @@"default": "./dist/*.js"},"./*": {- "development": {+ "trueforge-dev": {"types": "./src/*.ts","default": "./src/*.ts"},@@ -75,11 +75,11 @@"dev": "pnpm run build:gen:watch","build:js": "tsup","build:dts": "tsc -p tsconfig.build.json --emitDeclarationOnly",- "build:pkg": "node scripts/make-dist-package-json.mjs",+ "build:pkg": "node scripts/write-dist-package-json.mjs","build:check": "node scripts/check-dist.mjs","typecheck": "pnpm run build:gen && tsc --noEmit","test": "pnpm run build:gen && jest --config jest.config.cjs",- "pack:dry": "cd dist && pnpm pack --dry-run"+ "pack:dry": "pnpm pack --dry-run"},"dependencies": {"@ai-sdk/alibaba": "^2.0.24",diff --git a/packages/harness/scripts/check-dist.mjs b/packages/harness/scripts/check-dist.mjsindex fc6ef3d98..e0a945449 100644--- a/packages/harness/scripts/check-dist.mjs+++ b/packages/harness/scripts/check-dist.mjs@@ -1,6 +1,6 @@-// Post-build smoke tests, mirroring the tail of openai-node's build script:-// the staged dist must be loadable via require() and import() at the root,-// the subpath barrels, and a deep file path, with declarations beside them.+// Post-build smoke test: the exact regression class this guards against is+// dist/*.js being silently misinterpreted as ESM (see write-dist-package-json.mjs) —+// require()/import() failing here is how that would first surface.import fs from 'node:fs';import { createRequire } from 'node:module';import path from 'node:path';@@ -40,21 +40,29 @@ if (!fs.existsSync(distPkgPath)) {console.error('FAIL dist/package.json is missing (run build:pkg)');} else {const distPkg = JSON.parse(fs.readFileSync(distPkgPath, 'utf-8'));- const staged = JSON.stringify(distPkg);if (distPkg.type !== 'commonjs') {failures += 1;- console.error('FAIL dist/package.json type must be "commonjs"');- }- if (staged.includes('./dist/')) {- failures += 1;- console.error('FAIL dist/package.json still references ./dist/ paths');- }- if (staged.includes('./src/') || staged.includes('"development"')) {- failures += 1;- console.error('FAIL dist/package.json must not ship development → src/ export conditions');+ console.error('FAIL dist/package.json must set "type": "commonjs" (or dist/*.js resolves as ESM)');}}+// package.json is published as-is (no more staged dist/package.json rewrite), and only+// `dist` ships (files: ["dist"]) — so a literal "development" condition pointing at+// ./src/... would be a live, resolvable-but-broken promise for any consumer whose+// tooling defaults to activating it, not dead weight. Vite's default resolve.conditions+// is ['module', 'browser', 'development|production'], which it substitutes to the literal+// string "development" whenever NODE_ENV !== 'production' — i.e. every Vite dev-mode+// consumer would hit this with zero opt-in. The real condition name must never collide+// with a tool's own auto-activated condition (see packages/harness/package.json).+const rootPkgPath = path.join(pkgRoot, 'package.json');+const rootPkg = JSON.parse(fs.readFileSync(rootPkgPath, 'utf-8'));+if (JSON.stringify(rootPkg.exports ?? {}).includes('"development"')) {+ failures += 1;+ console.error(+ 'FAIL package.json exports must not use a literal "development" condition key (Vite/webpack auto-activate it in dev mode; src/ is not shipped)',+ );+}+if (failures > 0) {console.error(`check-dist: ${failures} failure(s)`);process.exit(1);diff --git a/packages/harness/scripts/make-dist-package-json.mjs b/packages/harness/scripts/make-dist-package-json.mjsdeleted file mode 100644index 3de414695..000000000--- a/packages/harness/scripts/make-dist-package-json.mjs+++ /dev/null@@ -1,57 +0,0 @@-// Generates dist/package.json so the package can be published from dist/,-// making dist the package root: src/core/foo.ts is importable as @truefoundry/utils-core/core/foo-import fs from 'node:fs';-import path from 'node:path';-import { fileURLToPath } from 'node:url';--const pkgRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');-const distDir = path.join(pkgRoot, 'dist');--const pkg = JSON.parse(fs.readFileSync(path.join(pkgRoot, 'package.json'), 'utf-8'));--// dist/ becomes the package root, so ./dist/foo.js is published as ./foo.js.-function stripDistPrefix(value) {- if (typeof value === 'string') return value.replace(/^\.\/dist\//, './');- if (value && typeof value === 'object') {- return Object.fromEntries(Object.entries(value).map(([key, entry]) => [key, stripDistPrefix(entry)]));- }- return value;-}--// Workspace-only: local `tsx` uses exports.development → src/. Published-// packages must not ship those conditions (src/ is not in the tarball).-function omitDevelopmentConditions(value) {- if (Array.isArray(value)) {- return value.map(omitDevelopmentConditions);- }- if (value && typeof value === 'object') {- return Object.fromEntries(- Object.entries(value)- .filter(([key]) => key !== 'development')- .map(([key, entry]) => [key, omitDevelopmentConditions(entry)]),- );- }- return value;-}--const distPkg = {- ...pkg,- type: 'commonjs',- main: stripDistPrefix(pkg.main),- module: stripDistPrefix(pkg.module),- types: stripDistPrefix(pkg.types),- exports: omitDevelopmentConditions(stripDistPrefix(pkg.exports)),- files: ['**/*'],-};-delete distPkg.scripts;-delete distPkg.devDependencies;--// Interim stance: the npm page must stay blank, so no README may ship.-const readmes = fs.readdirSync(distDir).filter(name => /^readme(\.(md|txt|markdown))?$/i.test(name));-if (readmes.length > 0) {- console.error(`Refusing to stage dist: remove ${readmes.join(', ')} so the npm page has no README`);- process.exit(1);-}--fs.writeFileSync(path.join(distDir, 'package.json'), `${JSON.stringify(distPkg, null, 2)}\n`);-console.log('Wrote dist/package.json');diff --git a/packages/harness/scripts/write-dist-package-json.mjs b/packages/harness/scripts/write-dist-package-json.mjsnew file mode 100644index 000000000..ad13ca0ab--- /dev/null+++ b/packages/harness/scripts/write-dist-package-json.mjs@@ -0,0 +1,15 @@+// tsup emits the CJS build as .js (and ESM as .mjs). Node determines a .js+// file's module format from the nearest package.json's "type" field, and the+// workspace package.json is "type": "module" (needed so exports.development+// can load src/ without a build) — so without this override, every dist/*.js+// file would be misread as an ES module and break require(). .mjs files are+// unaffected: Node always treats them as ESM regardless of "type".+import fs from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';++const pkgRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');+const distDir = path.join(pkgRoot, 'dist');++fs.writeFileSync(path.join(distDir, 'package.json'), '{\n "type": "commonjs"\n}\n');+console.log('Wrote dist/package.json ({ "type": "commonjs" })');diff --git a/packages/server/package.json b/packages/server/package.jsonindex ccae42d81..89a53cc2a 100644--- a/packages/server/package.json+++ b/packages/server/package.json@@ -16,13 +16,13 @@"build:gen": "node scripts/generate-catalog.mjs","build:clean": "rm -rf dist","build": "pnpm run build:gen && tsc --noEmit && pnpm run build:clean && tsup && chmod +x dist/cli.js && pnpm run build:frontend-assets",- "dev:serve": "NODE_OPTIONS='--conditions=development' NODE_ENV=development tsx watch --env-file=.env src/main.ts",+ "dev:serve": "NODE_OPTIONS='--conditions=trueforge-dev' NODE_ENV=development tsx watch --env-file=.env src/main.ts","dev": "pnpm run \"/^(build:gen:watch|dev:serve)$/\"",- "dev:no-watch": "NODE_OPTIONS='--conditions=development' NODE_ENV=development tsx --env-file=.env src/main.ts",+ "dev:no-watch": "NODE_OPTIONS='--conditions=trueforge-dev' NODE_ENV=development tsx --env-file=.env src/main.ts","start": "NODE_ENV=production node --env-file=.env dist/main.js","push-sandbox-image-to-daytona": "node scripts/push-sandbox-image-to-daytona.mjs",- "migrate": "NODE_OPTIONS='--conditions=development' STANDALONE=false tsx --env-file=.env src/db/postgres/migrate-cli.ts",- "openapi:write": "pnpm run build:gen && NODE_OPTIONS='--conditions=development' tsx --env-file=.env.test scripts/write-openapi.ts",+ "migrate": "NODE_OPTIONS='--conditions=trueforge-dev' STANDALONE=false tsx --env-file=.env src/db/postgres/migrate-cli.ts",+ "openapi:write": "pnpm run build:gen && NODE_OPTIONS='--conditions=trueforge-dev' tsx --env-file=.env.test scripts/write-openapi.ts","pack:dry": "pnpm pack --dry-run","migrate:legacy-sqlite-path": "node scripts/migrate-legacy-sqlite-path.mjs","predev:no-watch": "pnpm --filter @truefoundry/utils-core build:gen && pnpm run build:gen",@@ -30,7 +30,7 @@"test:eventsub": "jest --config jest.eventsub.config.cjs","test:store:postgres": "jest --config jest.store.postgres.config.cjs","test:store:sqlite": "jest --config jest.store.sqlite.config.cjs",- "test": "pnpm run build:gen && NODE_OPTIONS='--conditions=development' node --env-file=.env.test ./node_modules/jest/bin/jest.js --config jest.unit.config.cjs",+ "test": "pnpm run build:gen && NODE_OPTIONS='--conditions=trueforge-dev' node --env-file=.env.test ./node_modules/jest/bin/jest.js --config jest.unit.config.cjs","typecheck": "pnpm run build:gen && tsc --noEmit && tsc --noEmit -p tests/db/tsconfig.json && tsc --noEmit -p tests/unit/tsconfig.json"},"dependencies": {diff --git a/packages/server/tsconfig.json b/packages/server/tsconfig.jsonindex 23064b887..20ab3d729 100644--- a/packages/server/tsconfig.json+++ b/packages/server/tsconfig.json@@ -3,7 +3,7 @@"compilerOptions": {"types": ["node", "jest"],"noEmit": true,- "customConditions": ["development"],+ "customConditions": ["trueforge-dev"],"paths": {"openai/resources/chat": ["../harness/node_modules/openai/resources/chat/index.d.ts"]}