Skip to content

Beautiful, AI-native markdown IDE and LLM wiki

ci(ok): compute and log the fast soak tier, unarmed (#2957)

Andrew Mikofalvy0c302e5b4c56
diff --git a/.github/scripts/select-beta-to-promote.mjs b/.github/scripts/select-beta-to-promote.mjs
index 6c5e19b8a..7ea3ac6e8 100644
--- a/.github/scripts/select-beta-to-promote.mjs
+++ b/.github/scripts/select-beta-to-promote.mjs
@@ -22,14 +22,54 @@
// DMG must not mis-decide silently. fetchReleaseMeta signals this by returning
// null for 404 and throwing for everything else; selectPromotion lets the throw
// propagate so main() can exit non-zero.
+//
+// SOAK TIERS (currently LOG-ONLY — see below). Alongside the selection above the
+// script answers a second, orthogonal question: would the cut in front of it
+// qualify for a 1h, round-the-clock soak instead of the 24h weekday one?
+// Qualifying means the changeset delta is BOTH patch-only AND carries a linked
+// bug fix. patch-only alone is not selective — it describes roughly three
+// quarters of recent stable transitions, so on its own it is a global soak
+// reduction rather than a bug lane; the bug-linked conjunct is what makes it a
+// lane.
+//
+// This is a DIFFERENT axis from the `tier` field selectPromotion returns. That
+// one asks "may this UNDER-SOAKED beta promote early because its DMG smoked
+// clean"; this one asks "how long should a cut have to soak at all". They never
+// share a variable and are reported as separate outputs: `tier` and `soak_tier`.
+//
+// The soak tier is NOT armed. `FAST_TIER_ARMED` is a literal "false" in the
+// workflow, so `resolveTier` can only ever return "standard" and the dispatched
+// target is always the 24h selection. Everything here computes and logs the tier
+// it WOULD choose. Arming is an owner decision and is documented, with its
+// prerequisites, in RELEASES.md.
import { execFileSync, spawnSync } from 'node:child_process';
import { appendFileSync } from 'node:fs';
import { pathToFileURL } from 'node:url';
+import { computeStablePromotion, realGit } from '../../scripts/compute-stable-version.mjs';
const BETA_TAG_RE = /^v\d+\.\d+\.\d+-beta\.\d+$/;
const STABLE_TAG_RE = /^v\d+\.\d+\.\d+$/;
+// Evaluation-only soak for the fast-tier candidate. It never reaches the
+// promotion selection while the tier is unarmed.
+const FAST_SOAK_SECONDS = 3600;
+// Linear label that marks an issue as a bug fix.
+const BUG_LABEL = 'Bug';
+// Where a mirrored changeset's originating pull request lives. The mirror
+// rewrites SHAs but preserves the squash subject, so the `(#N)` in that subject
+// is a reference into the SOURCE monorepo, not into this repo.
+const DEFAULT_LINK_REPO = 'inkeep/agents-private';
+const LINEAR_API_URL = 'https://api.linear.app/graphql';
+// One URL can carry more than one attachment, and attachments are the union of
+// every carrier (branch, PR link, magic word, manual paste), which is why this
+// asks the attachment graph rather than parsing branch names.
+const ATTACHMENTS_FOR_URL_QUERY = `query AttachmentsForURL($url: String!) {
+ attachmentsForURL(url: $url, first: 20) {
+ nodes { issue { identifier labels(first: 50) { nodes { name } } } }
+ }
+}`;
+
// Filter raw `git tag` output to conforming beta tags, preserving input order.
// Ordering is git's job (`--sort=-version:refname`, newest first) — the same
// resolver promote-stable.yml / release.yml use; this only drops plain vX.Y.Z
@@ -128,6 +168,130 @@ function evaluateSmoke(beta, smokeBeta, log) {
}
}
+// --- soak-tier predicate (evaluation only while unarmed) ---
+
+// Does `candidate` qualify for the fast soak tier? Qualifying requires BOTH:
+// 1. the changeset delta over the latest stable bumps only the patch level
+// 2. at least one changeset in that delta traces to a bug-labelled issue
+//
+// The delta is not recomputed here. `computeDelta` is expected to be
+// `computeStablePromotion`, the same function promote-stable derives the
+// published version from, so "what is in this cut" has exactly one definition.
+//
+// Boundaries arrive as injected named seams so tests need no repo and no
+// network:
+// computeDelta(betaTag) -> the promotion decision for that tag
+// resolveChangesetPrUrl(id) -> pull-request URL | null (may throw)
+// resolveIssuesForUrl(url) -> { issues } | { unresolvable } (may throw)
+//
+// DELIBERATE DEPARTURE FROM THIS FILE'S FAIL-LOUD CONTRACT. Everything above
+// fails loud because a wrong SELECTION ships bad bytes. This predicate inverts
+// that: it decides only how long a cut soaks, so its failure mode costs latency
+// rather than correctness. Every failure, absence, or unresolvability therefore
+// degrades to "does not qualify" — the slower, more conservative tier — with a
+// warning recorded. Nothing here throws, and nothing here defaults to fast.
+export async function evaluateFastTier({
+ candidate,
+ computeDelta,
+ resolveChangesetPrUrl,
+ resolveIssuesForUrl,
+ bugLabel = BUG_LABEL,
+}) {
+ const warnings = [];
+ const verdict = (fields) => ({
+ candidate: candidate || null,
+ bump: null,
+ deltaCount: null,
+ bugLinked: false,
+ linkedIssues: [],
+ warnings,
+ ...fields,
+ });
+
+ // Nothing has cleared even the 1h soak, so there is no cut to classify.
+ if (!candidate) return verdict({ qualifies: false, reason: 'no-fast-candidate' });
+
+ let delta;
+ try {
+ delta = computeDelta(candidate);
+ } catch (err) {
+ warnings.push(`delta-error ${candidate}: ${err.message}`);
+ return verdict({ qualifies: false, reason: 'delta-error' });
+ }
+
+ // Already shipped / empty delta, and first-stable bootstrap, are both real
+ // answers rather than failures — neither describes a patch-only bug fix, and
+ // neither carries a bump to test.
+ if (delta?.skip) return verdict({ qualifies: false, reason: 'delta-skipped' });
+ if (delta?.bootstrap) return verdict({ qualifies: false, reason: 'delta-bootstrap' });
+
+ const bump = delta?.bump ?? null;
+ const deltaIds = Array.isArray(delta?.deltaIds) ? delta.deltaIds : [];
+ const withDelta = (fields) => verdict({ bump, deltaCount: deltaIds.length, ...fields });
+
+ // Bump first, on purpose: it is local and free, while the bug-link hop costs
+ // an API round trip per changeset. A non-patch delta can never qualify, so
+ // there is nothing to learn from resolving its links.
+ if (bump !== 'patch') return withDelta({ qualifies: false, reason: 'bump-not-patch' });
+
+ const wanted = String(bugLabel).toLowerCase();
+ const linkedIssues = [];
+ for (const id of deltaIds) {
+ let url;
+ try {
+ url = resolveChangesetPrUrl(id);
+ } catch (err) {
+ warnings.push(`changeset-pr-error ${id}: ${err.message}`);
+ continue;
+ }
+ if (!url) {
+ warnings.push(`changeset-pr-unresolved ${id}`);
+ continue;
+ }
+
+ let resolved;
+ try {
+ resolved = await resolveIssuesForUrl(url);
+ } catch (err) {
+ warnings.push(`issues-error ${url}: ${err.message}`);
+ continue;
+ }
+ if (resolved?.unresolvable) {
+ warnings.push(`issues-unresolvable ${url}: ${resolved.unresolvable}`);
+ continue;
+ }
+
+ // One URL may resolve to several issues; any one of them carrying the label
+ // makes the cut bug-linked.
+ for (const issue of resolved?.issues ?? []) {
+ const labels = Array.isArray(issue?.labels) ? issue.labels : [];
+ if (labels.some((name) => String(name).toLowerCase() === wanted)) {
+ linkedIssues.push(issue?.identifier || url);
+ }
+ }
+ }
+
+ if (linkedIssues.length === 0) return withDelta({ qualifies: false, reason: 'not-bug-linked' });
+ return withDelta({
+ qualifies: true,
+ reason: 'patch-only-and-bug-linked',
+ bugLinked: true,
+ linkedIssues,
+ });
+}
+
+// Fold the predicate verdict and the arming flag into the soak tier that governs
+// this tick, and pick the target that tier implies.
+//
+// `armed` is the entire difference between "we computed a fast tier" and "a fast
+// tier can shorten a real promotion". While it is false the fast branch is
+// unreachable, so `target` is the 24h selection on every path — which is why the
+// predicate can ship live without changing which betas promote, or when.
+export function resolveTier({ armed, verdict, standardTarget, fastTarget }) {
+ const tier = armed && verdict?.qualifies === true ? 'fast' : 'standard';
+ return { tier, target: tier === 'fast' ? fastTarget : standardTarget };
+}
+
// --- workflow-runtime wiring (real git / gh boundary) ---
// Newest plain vX.Y.Z stable tag's commit SHA ("" if no stable exists yet). The
@@ -194,19 +358,96 @@ function realFetchReleaseMeta(tag) {
}
}
-function main() {
+// A changeset's originating pull request, via the commit that ADDED the file.
+// Copybara rewrites SHAs but keeps the squash subject, and this repo's squash
+// subjects end in `(#N)`, so the trailing reference survives the mirror. It is
+// anchored to the end of the subject so a number quoted mid-title cannot
+// outrank the real one. No `(#N)` (a hand-authored or imported commit) is a
+// clean "no link", not an error.
+export function makeResolveChangesetPrUrl(linkRepo) {
+ return (changesetId) => {
+ const subject = execFileSync(
+ 'git',
+ ['log', '-1', '--diff-filter=A', '--format=%s', '--', `.changeset/${changesetId}.md`],
+ { encoding: 'utf8' },
+ ).trim();
+ const m = /\(#(\d+)\)$/.exec(subject);
+ return m ? `https://github.com/${linkRepo}/pull/${m[1]}` : null;
+ };
+}
+
+// Issues attached to a pull-request URL in Linear.
+//
+// Header is the raw key: `Bearer` is the OAuth form and is rejected for a
+// personal API key. Rate limiting arrives as an HTTP 400 carrying
+// `errors[].extensions.code === "RATELIMITED"` rather than a 429, so the GraphQL
+// error array is inspected before the status code.
+//
+// Returns `{ unresolvable }` for the states that are simply "we cannot answer"
+// (no key configured, a response we do not recognize) and throws for transport
+// and API failures. The caller degrades on both; the split exists so the two
+// read differently in the log.
+export function makeResolveIssuesForUrl(apiKey) {
+ return async (url) => {
+ if (!apiKey) return { unresolvable: 'no-linear-api-key' };
+
+ const res = await fetch(LINEAR_API_URL, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json', authorization: apiKey },
+ body: JSON.stringify({ query: ATTACHMENTS_FOR_URL_QUERY, variables: { url } }),
+ });
+ const body = await res.text();
+
+ let payload;
+ try {
+ payload = JSON.parse(body);
+ } catch {
+ throw new Error(`Linear returned a non-JSON response (HTTP ${res.status}).`);
+ }
+ if (Array.isArray(payload?.errors) && payload.errors.length > 0) {
+ const codes = payload.errors
+ .map((e) => e?.extensions?.code)
+ .filter(Boolean)
+ .join(',');
+ throw new Error(
+ `Linear GraphQL error (HTTP ${res.status}${codes ? `, ${codes}` : ''}): ${payload.errors[0]?.message ?? 'unknown'}`,
+ );
+ }
+ if (!res.ok) throw new Error(`Linear request failed (HTTP ${res.status}).`);
+
+ const nodes = payload?.data?.attachmentsForURL?.nodes;
+ if (!Array.isArray(nodes)) return { unresolvable: 'malformed-response' };
+
+ const issues = [];
+ for (const node of nodes) {
+ const issue = node?.issue;
+ if (!issue) continue;
+ issues.push({
+ identifier: issue.identifier ?? '',
+ labels: (issue.labels?.nodes ?? [])
+ .map((l) => l?.name)
+ .filter((n) => typeof n === 'string'),
+ });
+ }
+ return { issues };
+ };
+}
+
+async function main() {
const soakSeconds = Number(process.env.SOAK_SECONDS || '86400');
+ const armed = process.env.FAST_TIER_ARMED === 'true';
const rawTags = execFileSync('git', ['tag', '--list', 'v*-beta.*', '--sort=-version:refname'], {
encoding: 'utf8',
});
const betaTags = parseBetaTags(rawTags);
const latestStableSha = resolveLatestStableSha();
+ const isAlreadyShipped = makeRealIsAlreadyShipped(latestStableSha);
let result;
try {
result = selectPromotion({
betaTags,
- isAlreadyShipped: makeRealIsAlreadyShipped(latestStableSha),
+ isAlreadyShipped,
fetchReleaseMeta: realFetchReleaseMeta,
soakSeconds,
nowMs: Date.now(),
@@ -219,13 +460,11 @@ function main() {
process.exit(1);
}
- let target = '';
- let tier = '';
- if (result.kind === 'select') {
- target = result.target;
- tier = result.tier;
+ const standardTarget = result.kind === 'select' ? result.target : '';
+ const selectionTier = result.kind === 'select' ? result.tier : '';
+ if (standardTarget) {
console.log(
- `::notice::Eligible: ${target} (unshipped + fully cut + soaked >= ${soakSeconds}s).`,
+ `::notice::Eligible: ${standardTarget} (unshipped + fully cut + soaked >= ${soakSeconds}s).`,
);
} else {
console.log(
@@ -233,15 +472,84 @@ function main() {
);
}
+ // Soak-tier evaluation runs on EVERY tick, including outside business hours,
+ // so the decision and its inputs are observable before anyone relies on them.
+ // The whole block is guarded: the selection above is already final, and a tier
+ // that cannot be computed must cost latency, never the tick.
+ let fastTarget = '';
+ let verdict = {
+ qualifies: false,
+ reason: 'not-evaluated',
+ candidate: null,
+ bump: null,
+ deltaCount: null,
+ bugLinked: false,
+ linkedIssues: [],
+ warnings: [],
+ };
+ try {
+ const fastCandidate = selectPromotion({
+ betaTags,
+ isAlreadyShipped,
+ fetchReleaseMeta: realFetchReleaseMeta,
+ soakSeconds: FAST_SOAK_SECONDS,
+ nowMs: Date.now(),
+ });
+ fastTarget = fastCandidate.kind === 'select' ? fastCandidate.target : '';
+ verdict = await evaluateFastTier({
+ candidate: fastTarget,
+ computeDelta: (betaTag) => computeStablePromotion(betaTag, realGit),
+ resolveChangesetPrUrl: makeResolveChangesetPrUrl(process.env.LINK_REPO || DEFAULT_LINK_REPO),
+ resolveIssuesForUrl: makeResolveIssuesForUrl(process.env.LINEAR_API_KEY),
+ });
+ } catch (err) {
+ verdict = {
+ ...verdict,
+ reason: 'tier-evaluation-error',
+ warnings: [`tier-evaluation-error: ${err.message}`],
+ };
+ }
+
+ const { tier: soakTier, target } = resolveTier({ armed, verdict, standardTarget, fastTarget });
+
+ for (const warning of verdict.warnings) {
+ console.log(`::warning::Soak-tier predicate degraded: ${warning}`);
+ }
+ console.log(
+ `::notice::Soak tier: ${soakTier} (armed=${armed}; would-qualify=${verdict.qualifies}, reason=${verdict.reason}, candidate=${verdict.candidate || 'none'}, bump=${verdict.bump || 'none'}, delta=${verdict.deltaCount ?? 'none'}, bug-linked=${verdict.bugLinked}${verdict.linkedIssues.length > 0 ? ` [${verdict.linkedIssues.join(', ')}]` : ''}).`,
+ );
+ if (!armed) {
+ console.log(
+ `::notice::The fast soak tier is not armed; promotion continues to use the ${soakSeconds}s soak and the target is the 24h selection.`,
+ );
+ }
+
if (process.env.GITHUB_OUTPUT) {
// `fast_tier_candidate` is the seam the macOS smoke leg keys off. Nothing
- // populates it today: `main()` supplies no fast-tier predicate, so
- // selectPromotion never reports the fast tier and the leg never runs. That
- // is the whole of the build-but-do-not-arm posture — arming is a documented
- // operator step in RELEASES.md, not a side effect of merging this.
+ // populates it: it is written empty here and the soak-tier predicate never
+ // feeds it, because the two answer different questions (may an under-soaked
+ // beta promote early on a clean DMG smoke, versus how long should a cut soak
+ // at all). Wiring them together is a separate, deliberate step.
+ //
+ // Every other value is a validated tag, a fixed reason keyword, a boolean,
+ // or a number, so none of them can break the key=value framing. Warnings are
+ // deliberately log-only — they carry API error text.
appendFileSync(
process.env.GITHUB_OUTPUT,
- `target=${target}\ntier=${tier}\nfast_tier_candidate=\n`,
+ [
+ `target=${target}`,
+ `tier=${selectionTier}`,
+ 'fast_tier_candidate=',
+ `soak_tier=${soakTier}`,
+ `fast_armed=${armed}`,
+ `fast_candidate=${verdict.candidate || ''}`,
+ `fast_qualifies=${verdict.qualifies}`,
+ `fast_reason=${verdict.reason}`,
+ `fast_bump=${verdict.bump || ''}`,
+ `fast_delta_count=${verdict.deltaCount ?? ''}`,
+ `fast_bug_linked=${verdict.bugLinked}`,
+ '',
+ ].join('\n'),
);
}
}
@@ -249,5 +557,10 @@ function main() {
// Run main() only as a CLI, not when imported by the test file. Portable across
// node (all ESM versions) and bun — import.meta.main is Node 24+ only.
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
- main();
+ main().catch((err) => {
+ // main() is async only because the bug-link hop is; the fail-loud contract
+ // is unchanged, so a rejection still exits non-zero and retries next tick.
+ console.error(`::error::select-beta-to-promote: ${err.message}`);
+ process.exit(1);
+ });
}
diff --git a/.github/scripts/select-beta-to-promote.test.mjs b/.github/scripts/select-beta-to-promote.test.mjs
index 330860b4a..cefdd8b82 100644
--- a/.github/scripts/select-beta-to-promote.test.mjs
+++ b/.github/scripts/select-beta-to-promote.test.mjs
@@ -1,5 +1,17 @@
-import { describe, expect, test } from 'vitest';
-import { parseBetaTags, selectPromotion } from './select-beta-to-promote.mjs';
+import { execFileSync } from 'node:child_process';
+import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { afterEach, describe, expect, test } from 'vitest';
+import { computeStablePromotion } from '../../scripts/compute-stable-version.mjs';
+import {
+ evaluateFastTier,
+ makeResolveChangesetPrUrl,
+ makeResolveIssuesForUrl,
+ parseBetaTags,
+ resolveTier,
+ selectPromotion,
+} from './select-beta-to-promote.mjs';
// Fixed clock so tests never call Date.now().
const NOW = Date.parse('2026-07-08T20:00:00Z');
@@ -262,3 +274,494 @@ describe('fast tier (FR5a) — DMG-smoke-gated early promotion', () => {
expect(smokeCalls).toEqual([]);
});
});
+
+// --- soak-tier predicate ---
+//
+// A different axis from the DMG-smoke fast tier above. That one asks whether an
+// under-soaked beta may promote early; this one asks how long a cut should have
+// to soak at all. They never share a variable.
+
+const FAST_CANDIDATE = 'v0.30.2-beta.3';
+const STANDARD_TARGET = 'v0.30.1-beta.9';
+const PR_URL = 'https://github.com/inkeep/agents-private/pull/2767';
+const SHA_BETA = 'a'.repeat(40);
+const SHA_STABLE = 'b'.repeat(40);
+
+// A computeStablePromotion-shaped result. Kept structural (not a re-derivation)
+// so the seam is exercised with the same fields the real function returns; the
+// "consumes computeStablePromotion output directly" test below pins that shape
+// against the real implementation.
+function deltaOf({ bump = 'patch', deltaIds = ['fix-a'] } = {}) {
+ return {
+ skip: false,
+ stableVersion: '0.30.2',
+ stableTag: 'v0.30.2',
+ bump,
+ deltaIds,
+ deltaCount: deltaIds.length,
+ betaSha: SHA_BETA,
+ latestStableSha: SHA_STABLE,
+ };
+}
+
+// changeset id -> pull-request URL. An unknown id === the adding commit's
+// subject carried no `(#N)`; the sentinel 'THROW' simulates a git failure.
+function prUrls(map) {
+ return (id) => {
+ if (map[id] === 'THROW') throw new Error('simulated git failure');
+ return map[id] ?? null;
+ };
+}
+
+// URL -> issues. An unknown URL === zero attachments. 'THROW' simulates an
+// HTTP / network / rate-limit failure; 'UNRESOLVABLE' simulates a missing key.
+function issueResolver(map) {
+ return async (url) => {
+ const entry = map[url];
+ if (entry === 'THROW') {
+ throw new Error('Linear GraphQL error (HTTP 400, RATELIMITED): too many requests');
+ }
+ if (entry === 'UNRESOLVABLE') return { unresolvable: 'no-linear-api-key' };
+ return { issues: entry ?? [] };
+ };
+}
+
+const BUG_ISSUE = [{ identifier: 'PRD-7490', labels: ['Bug', 'Performance'] }];
+
+const soakTier = (over) =>
+ evaluateFastTier({
+ candidate: FAST_CANDIDATE,
+ computeDelta: () => deltaOf(),
+ resolveChangesetPrUrl: prUrls({ 'fix-a': PR_URL }),
+ resolveIssuesForUrl: issueResolver({ [PR_URL]: BUG_ISSUE }),
+ ...over,
+ });
+
+// The tier an ARMED workflow would land on. Kept separate from `qualifies` so
+// the three-case matrix reads as a tier decision rather than a boolean.
+const armedTier = (verdict) =>
+ resolveTier({ armed: true, verdict, standardTarget: STANDARD_TARGET, fastTarget: FAST_CANDIDATE })
+ .tier;
+
+describe('evaluateFastTier (soak tier)', () => {
+ test('patch-only delta whose changeset resolves to a bug-labelled issue would take the fast tier', async () => {
+ const r = await soakTier();
+ expect(r.qualifies).toBe(true);
+ expect(r.reason).toBe('patch-only-and-bug-linked');
+ expect(r.bump).toBe('patch');
+ expect(r.bugLinked).toBe(true);
+ expect(r.linkedIssues).toEqual(['PRD-7490']);
+ expect(r.warnings).toEqual([]);
+ expect(armedTier(r)).toBe('fast');
+ });
+
+ test('a minor changeset in the delta stays on the standard tier even with a linked bug', async () => {
+ const r = await soakTier({ computeDelta: () => deltaOf({ bump: 'minor' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('bump-not-patch');
+ expect(r.bump).toBe('minor');
+ expect(armedTier(r)).toBe('standard');
+ });
+
+ test('a patch-only delta with no changeset resolving to any issue stays on the standard tier', async () => {
+ const r = await soakTier({ resolveIssuesForUrl: issueResolver({}) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('not-bug-linked');
+ expect(r.bugLinked).toBe(false);
+ expect(armedTier(r)).toBe('standard');
+ });
+
+ test('an issue without the bug label does not make the cut bug-linked', async () => {
+ const r = await soakTier({
+ resolveIssuesForUrl: issueResolver({
+ [PR_URL]: [{ identifier: 'PRD-1', labels: ['Feature', 'Docs'] }],
+ }),
+ });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('not-bug-linked');
+ });
+
+ test('an unresolvable lookup degrades to standard with a warning naming the cause', async () => {
+ const r = await soakTier({ resolveIssuesForUrl: issueResolver({ [PR_URL]: 'UNRESOLVABLE' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('not-bug-linked');
+ expect(r.warnings).toHaveLength(1);
+ expect(r.warnings[0]).toMatch(/issues-unresolvable/);
+ expect(r.warnings[0]).toMatch(/no-linear-api-key/);
+ });
+
+ test('a throwing lookup degrades to standard with a warning and lets nothing escape', async () => {
+ const r = await soakTier({ resolveIssuesForUrl: issueResolver({ [PR_URL]: 'THROW' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('not-bug-linked');
+ expect(r.warnings).toHaveLength(1);
+ expect(r.warnings[0]).toMatch(/issues-error/);
+ expect(r.warnings[0]).toMatch(/RATELIMITED/);
+ });
+
+ test('a throwing changeset-to-PR resolution degrades rather than failing the tick', async () => {
+ const r = await soakTier({ resolveChangesetPrUrl: prUrls({ 'fix-a': 'THROW' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.warnings[0]).toMatch(/changeset-pr-error fix-a/);
+ });
+
+ test('a major bump in the delta stays on the standard tier', async () => {
+ const r = await soakTier({ computeDelta: () => deltaOf({ bump: 'major' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('bump-not-patch');
+ expect(armedTier(r)).toBe('standard');
+ });
+
+ test('no fast candidate at all yields a distinguishable reason', async () => {
+ const r = await soakTier({ candidate: '' });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('no-fast-candidate');
+ expect(r.candidate).toBeNull();
+ expect(r.bump).toBeNull();
+ });
+
+ test('one URL resolving to several issues is bug-linked when any one carries the label', async () => {
+ const r = await soakTier({
+ resolveIssuesForUrl: issueResolver({
+ [PR_URL]: [
+ { identifier: 'PRD-1', labels: ['Feature'] },
+ { identifier: 'PRD-2', labels: ['Bug'] },
+ { identifier: 'PRD-3', labels: [] },
+ ],
+ }),
+ });
+ expect(r.qualifies).toBe(true);
+ expect(r.linkedIssues).toEqual(['PRD-2']);
+ });
+
+ test('a changeset whose adding commit carries no PR reference contributes no link and does not throw', async () => {
+ const r = await soakTier({ resolveChangesetPrUrl: prUrls({}) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('not-bug-linked');
+ expect(r.warnings).toEqual(['changeset-pr-unresolved fix-a']);
+ });
+
+ test('a skipped delta stays on the standard tier without crashing', async () => {
+ const r = await soakTier({ computeDelta: () => ({ skip: true, reason: 'already shipped' }) });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('delta-skipped');
+ });
+
+ test('a bootstrap delta stays on the standard tier without crashing', async () => {
+ const r = await soakTier({
+ computeDelta: () => ({ skip: false, bootstrap: true, bump: null }),
+ });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('delta-bootstrap');
+ });
+
+ test('a throwing delta computation degrades rather than failing the tick', async () => {
+ const r = await soakTier({
+ computeDelta: () => {
+ throw new Error('git rev-parse failed');
+ },
+ });
+ expect(r.qualifies).toBe(false);
+ expect(r.reason).toBe('delta-error');
+ expect(r.warnings[0]).toMatch(/delta-error/);
+ });
+
+ test('resolving several changesets collects links across all of them', async () => {
+ const otherUrl = 'https://github.com/inkeep/agents-private/pull/2800';
+ const r = await soakTier({
+ computeDelta: () => deltaOf({ deltaIds: ['fix-a', 'fix-b'] }),
+ resolveChangesetPrUrl: prUrls({ 'fix-a': PR_URL, 'fix-b': otherUrl }),
+ resolveIssuesForUrl: issueResolver({
+ [PR_URL]: [{ identifier: 'PRD-1', labels: ['Feature'] }],
+ [otherUrl]: [{ identifier: 'PRD-2', labels: ['bug'] }], // label match is case-insensitive
+ }),
+ });
+ expect(r.deltaCount).toBe(2);
+ expect(r.qualifies).toBe(true);
+ expect(r.linkedIssues).toEqual(['PRD-2']);
+ });
+
+ test('consumes computeStablePromotion output directly, so no second delta implementation exists', async () => {
+ // Drives the predicate with a delta produced by the REAL function that
+ // decides the published version. If that function's returned shape changes,
+ // this fails instead of the two silently disagreeing about what is in a cut.
+ const fakeGit = {
+ revParse: (ref) => (ref.startsWith('v0.30.2-beta') ? SHA_BETA : SHA_STABLE),
+ newestStableTag: () => 'v0.30.1',
+ changesetIds: (sha) =>
+ sha === SHA_BETA ? ['already-shipped', 'fix-a'] : ['already-shipped'],
+ isAncestor: () => false,
+ bumpTypeOf: () => 'patch',
+ };
+ const r = await soakTier({ computeDelta: (tag) => computeStablePromotion(tag, fakeGit) });
+ expect(r.bump).toBe('patch');
+ expect(r.deltaCount).toBe(1);
+ expect(r.qualifies).toBe(true);
+ expect(armedTier(r)).toBe('fast');
+ });
+});
+
+describe('resolveTier (soak tier)', () => {
+ test('an unarmed workflow reports standard and the 24h target even when the predicate qualifies', async () => {
+ const verdict = await soakTier();
+ expect(verdict.qualifies).toBe(true);
+ expect(
+ resolveTier({
+ armed: false,
+ verdict,
+ standardTarget: STANDARD_TARGET,
+ fastTarget: FAST_CANDIDATE,
+ }),
+ ).toEqual({ tier: 'standard', target: STANDARD_TARGET });
+ });
+
+ test('no verdict shape can reach the fast tier while unarmed', () => {
+ const verdicts = [
+ { qualifies: true },
+ { qualifies: false },
+ { qualifies: 'true' },
+ { qualifies: 1 },
+ {},
+ null,
+ ];
+ for (const verdict of verdicts) {
+ expect(
+ resolveTier({
+ armed: false,
+ verdict,
+ standardTarget: STANDARD_TARGET,
+ fastTarget: FAST_CANDIDATE,
+ }),
+ ).toEqual({ tier: 'standard', target: STANDARD_TARGET });
+ }
+ });
+
+ test('an armed workflow takes the fast candidate only on a strict qualifying verdict', () => {
+ expect(
+ resolveTier({
+ armed: true,
+ verdict: { qualifies: true },
+ standardTarget: STANDARD_TARGET,
+ fastTarget: FAST_CANDIDATE,
+ }),
+ ).toEqual({ tier: 'fast', target: FAST_CANDIDATE });
+ // A truthy-but-not-true verdict must not be read as qualifying.
+ expect(
+ resolveTier({
+ armed: true,
+ verdict: { qualifies: 'yes' },
+ standardTarget: STANDARD_TARGET,
+ fastTarget: FAST_CANDIDATE,
+ }),
+ ).toEqual({ tier: 'standard', target: STANDARD_TARGET });
+ });
+});
+
+// --- real boundaries ---
+//
+// Every test above injects the three seams, which is right for the pure
+// predicate but leaves the actual git and HTTP implementations uncovered. These
+// drive the real functions: the git hop against a throwaway repository, the
+// Linear hop against a stubbed global fetch. They are the only tests here that
+// would notice if a real boundary stopped behaving the way the predicate
+// assumes.
+
+const tempRepos = [];
+const realFetch = globalThis.fetch;
+
+afterEach(() => {
+ globalThis.fetch = realFetch;
+ while (tempRepos.length > 0) {
+ rmSync(tempRepos.pop(), { recursive: true, force: true });
+ }
+});
+
+// A repository with one commit per changeset, so the adding-commit lookup has
+// something real to walk. Identity and signing are pinned so this does not
+// depend on the developer's or runner's git config.
+function makeRepoWithChangesets(commits) {
+ const dir = mkdtempSync(join(tmpdir(), 'ok-fast-tier-'));
+ tempRepos.push(dir);
+ const git = (...args) =>
+ execFileSync(
+ 'git',
+ [
+ '-c',
+ 'user.name=Test',
+ '-c',
+ '-c',
+ 'commit.gpgsign=false',
+ ...args,
+ ],
+ { cwd: dir, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] },
+ );
+ git('init', '--quiet', '--initial-branch=main');
+ execFileSync('mkdir', ['-p', join(dir, '.changeset')]);
+ for (const { id, subject } of commits) {
+ writeFileSync(
+ join(dir, '.changeset', `${id}.md`),
+ `---\n'@inkeep/open-knowledge': patch\n---\n\n${id}\n`,
+ );
+ git('add', '--all');
+ git('commit', '--quiet', '-m', subject);
+ }
+ return dir;
+}
+
+describe('makeResolveChangesetPrUrl (real git)', () => {
+ test('resolves a changeset to the pull request named by its adding commit', () => {
+ const dir = makeRepoWithChangesets([
+ { id: 'older-change', subject: 'feat(ok): something earlier (#1000)' },
+ { id: 'fix-the-thing', subject: 'fix(ok): stop the crash (#2767)' },
+ ]);
+ const cwd = process.cwd();
+ try {
+ process.chdir(dir);
+ const resolve = makeResolveChangesetPrUrl('inkeep/agents-private');
+ expect(resolve('fix-the-thing')).toBe('https://github.com/inkeep/agents-private/pull/2767');
+ // Picks the commit that added THAT file, not simply the newest commit.
+ expect(resolve('older-change')).toBe('https://github.com/inkeep/agents-private/pull/1000');
+ } finally {
+ process.chdir(cwd);
+ }
+ });
+
+ test('returns null rather than throwing when the subject carries no PR reference', () => {
+ const dir = makeRepoWithChangesets([
+ { id: 'hand-authored', subject: 'chore: hand-authored commit with no pull request' },
+ // A number that is not the trailing squash reference must not be picked up.
+ { id: 'mid-subject', subject: 'fix: handle (#12) in prose but land without a reference' },
+ ]);
+ const cwd = process.cwd();
+ try {
+ process.chdir(dir);
+ const resolve = makeResolveChangesetPrUrl('inkeep/agents-private');
+ expect(resolve('hand-authored')).toBeNull();
+ expect(resolve('mid-subject')).toBeNull();
+ // A changeset that does not exist produces empty git output, not a throw.
+ expect(resolve('never-existed')).toBeNull();
+ } finally {
+ process.chdir(cwd);
+ }
+ });
+});
+
+describe('makeResolveIssuesForUrl (real HTTP handling)', () => {
+ const URL_UNDER_TEST = 'https://github.com/inkeep/agents-private/pull/2767';
+
+ function stubFetch(handler) {
+ const calls = [];
+ globalThis.fetch = async (url, init) => {
+ calls.push({ url, init });
+ return handler(url, init);
+ };
+ return calls;
+ }
+
+ const jsonResponse = (status, body) =>
+ new Response(typeof body === 'string' ? body : JSON.stringify(body), {
+ status,
+ headers: { 'content-type': 'application/json' },
+ });
+
+ test('reports unresolvable without touching the network when no key is configured', async () => {
+ const calls = stubFetch(() => {
+ throw new Error('the resolver must not make a request without a key');
+ });
+ await expect(makeResolveIssuesForUrl(undefined)(URL_UNDER_TEST)).resolves.toEqual({
+ unresolvable: 'no-linear-api-key',
+ });
+ await expect(makeResolveIssuesForUrl('')(URL_UNDER_TEST)).resolves.toEqual({
+ unresolvable: 'no-linear-api-key',
+ });
+ expect(calls).toHaveLength(0);
+ });
+
+ test('sends the raw key as Authorization, with no Bearer prefix', async () => {
+ const calls = stubFetch(() =>
+ jsonResponse(200, { data: { attachmentsForURL: { nodes: [] } } }),
+ );
+ await makeResolveIssuesForUrl('lin_api_secret')(URL_UNDER_TEST);
+ expect(calls).toHaveLength(1);
+ expect(calls[0].init.headers.authorization).toBe('lin_api_secret');
+ // The URL is passed as a GraphQL variable, not interpolated into the query.
+ expect(JSON.parse(calls[0].init.body).variables).toEqual({ url: URL_UNDER_TEST });
+ });
+
+ test('parses issues and their labels out of a well-formed response', async () => {
+ stubFetch(() =>
+ jsonResponse(200, {
+ data: {
+ attachmentsForURL: {
+ nodes: [
+ {
+ issue: {
+ identifier: 'PRD-7490',
+ labels: { nodes: [{ name: 'Bug' }, { name: 'Performance' }] },
+ },
+ },
+ { issue: { identifier: 'PRD-7491', labels: { nodes: [] } } },
+ { issue: null },
+ ],
+ },
+ },
+ }),
+ );
+ await expect(makeResolveIssuesForUrl('key')(URL_UNDER_TEST)).resolves.toEqual({
+ issues: [
+ { identifier: 'PRD-7490', labels: ['Bug', 'Performance'] },
+ { identifier: 'PRD-7491', labels: [] },
+ ],
+ });
+ });
+
+ test('throws on a rate-limit, which arrives as HTTP 400 rather than 429', async () => {
+ stubFetch(() =>
+ jsonResponse(400, {
+ errors: [{ message: 'rate limited', extensions: { code: 'RATELIMITED' } }],
+ }),
+ );
+ await expect(makeResolveIssuesForUrl('key')(URL_UNDER_TEST)).rejects.toThrow(/RATELIMITED/);
+ });
+
+ test('throws on a non-JSON body', async () => {
+ stubFetch(() => new Response('<html>gateway timeout</html>', { status: 504 }));
+ await expect(makeResolveIssuesForUrl('key')(URL_UNDER_TEST)).rejects.toThrow(/non-JSON/);
+ });
+
+ test('throws on a plain HTTP failure that carries no GraphQL errors', async () => {
+ stubFetch(() => jsonResponse(401, { data: null }));
+ await expect(makeResolveIssuesForUrl('key')(URL_UNDER_TEST)).rejects.toThrow(/HTTP 401/);
+ });
+
+ test('reports unresolvable on a 200 whose shape is unrecognized', async () => {
+ stubFetch(() => jsonResponse(200, { data: {} }));
+ await expect(makeResolveIssuesForUrl('key')(URL_UNDER_TEST)).resolves.toEqual({
+ unresolvable: 'malformed-response',
+ });
+ });
+
+ test('a degrading real resolver still leaves the predicate on the standard tier', async () => {
+ // The seams under test, wired together the way main() wires them.
+ stubFetch(() =>
+ jsonResponse(400, { errors: [{ message: 'nope', extensions: { code: 'RATELIMITED' } }] }),
+ );
+ const dir = makeRepoWithChangesets([{ id: 'fix-a', subject: 'fix(ok): a real fix (#2767)' }]);
+ const cwd = process.cwd();
+ try {
+ process.chdir(dir);
+ const r = await evaluateFastTier({
+ candidate: FAST_CANDIDATE,
+ computeDelta: () => deltaOf(),
+ resolveChangesetPrUrl: makeResolveChangesetPrUrl('inkeep/agents-private'),
+ resolveIssuesForUrl: makeResolveIssuesForUrl('key'),
+ });
+ expect(r.qualifies).toBe(false);
+ expect(r.warnings[0]).toMatch(/issues-error .*RATELIMITED/);
+ expect(armedTier(r)).toBe('standard');
+ } finally {
+ process.chdir(cwd);
+ }
+ });
+});
diff --git a/.github/workflows/select-beta-to-promote.yml b/.github/workflows/select-beta-to-promote.yml
index e7d2c52b3..7186e63b7 100644
--- a/.github/workflows/select-beta-to-promote.yml
+++ b/.github/workflows/select-beta-to-promote.yml
@@ -37,15 +37,35 @@ name: Select beta to promote
# trigger is an additive fast-path so an already-eligible beta promotes promptly
# when a Copybara mirror commit lands in-hours.
#
-# Weekend latency: the cron is Mon-Fri, so a beta that crosses 24h over the
-# weekend promotes on the next weekday tick — a Friday-published beta can be up
-# to ~71h online before its Monday-morning promotion. This is the intended "no
-# weekend promotions" behavior; workflow_dispatch (which bypasses the hours gate)
-# is the escape hatch for an urgent out-of-hours promotion.
+# Weekend latency: the in-script gate is Mon-Fri, so a beta that crosses 24h
+# over the weekend promotes on the next weekday tick — a Friday-published beta
+# can be up to ~71h online before its Monday-morning promotion. This is the
+# intended "no weekend promotions" behavior for the standard soak tier;
+# workflow_dispatch (which bypasses the hours gate) is the escape hatch for an
+# urgent out-of-hours promotion.
+#
+# SOAK TIERS. Two are defined; only one is reachable today.
+# standard — 24h online, dispatched inside the window above. Every promotion
+# uses this.
+# fast — 1h online, around the clock, for a cut that is BOTH patch-only
+# AND carries a linked bug fix. Both conjuncts are load-bearing:
+# patch-only alone covers most stable transitions, so on its own it
+# would be a blanket soak reduction rather than a bug lane.
+# This is a different axis from the DMG-smoke fast tier below, which asks
+# whether an under-soaked beta may promote early on a clean smoke. The two are
+# reported as separate step outputs (`soak_tier` and `tier`) and are not wired
+# to each other.
+#
+# The fast SOAK tier is NOT ARMED. `FAST_TIER_ARMED` on the select step is a
+# literal "false", so the script computes and logs the tier it would choose and
+# then always reports soak_tier=standard. SOAK_SECONDS stays 86400 on every path
+# and the dispatched target is always the 24h selection, so which betas promote
+# and when is unchanged. Arming is an owner decision — the "Soak tiers" section
+# of RELEASES.md is the operator-facing record.
#
# DST correctness: GitHub Actions cron is UTC and ignores daylight saving, so a
-# hard-coded "9am ET" would drift an hour twice a year. The cron fires hourly
-# across the UTC band that brackets the ET/PT window year-round; the
+# hard-coded "9am ET" would drift an hour twice a year. The cron therefore does
+# not encode the window at all — it fires around the clock, and the
# AUTHORITATIVE business-hours decision is made in-script against the
# America/New_York clock (see the "Business-hours gate" step), so the window
# stays exact through DST transitions. 4pm PT == 7pm ET == 19:00 ET, so the ET
@@ -68,16 +88,19 @@ name: Select beta to promote
on:
schedule:
- # Hourly at :17 (off the :00 scheduler spike), 13:00-23:59 UTC, Mon-Fri.
- # This UTC band brackets the ET/PT window across DST (9am ET = 13:00 UTC
- # under EDT / 14:00 UTC under EST; 4pm PT = 23:00 UTC under PDT / 00:00 UTC
- # under PST). The in-script ET gate is authoritative and rejects any tick
- # outside 09:00-19:00 ET for the current DST state, so band-edge over-firing
- # is a clean no-op. Ticks land at :17, so up to ~1h can elapse before a
- # newly-soaked beta is noticed, and the final ~40 min before the window
- # closes falls between the last in-window tick and the boundary; both are
- # covered by the next in-window tick or the push fast-path.
- - cron: "17 13-23 * * 1-5"
+ # Every 10 minutes, around the clock. A strict superset of the previous
+ # hourly :17 weekday-band tick, so nothing that used to fire stops firing.
+ # Two reasons for the widening: a 1h fast-tier soak only means ~1h if the
+ # evaluator looks more often than hourly, and the soak-tier decision has to
+ # be observable at every hour of every day, not only inside the window. The
+ # offsets stay off :00 to avoid the scheduler spike.
+ #
+ # Cron here is UTC and DST-blind BY DESIGN — it no longer encodes the window
+ # at all. The AUTHORITATIVE business-hours decision is still made in-script
+ # against the America/New_York clock (see the "Business-hours gate" step), so
+ # the window stays exact through DST transitions and an out-of-window tick is
+ # a clean evaluate-and-no-op.
+ - cron: "7,17,27,37,47,57 * * * *"
# Fast-path: re-evaluate the gate when Copybara mirrors a commit to main so an
# already-eligible beta promotes promptly in-hours. Additive to the schedule;
# the same in-script business-hours + soak gate applies.
@@ -148,8 +171,12 @@ jobs:
echo "open=false" >> "$GITHUB_OUTPUT"
fi
+ # Checkout / setup-node / select deliberately carry NO hours gate. The
+ # soak-tier decision has to be computed and logged on every tick,
+ # including outside the window, and gating the selector on the clock
+ # would make that impossible. Only the dispatch step below is gated, so
+ # an out-of-window tick evaluates and then does nothing.
- name: Checkout (need full tag history for beta resolution)
- if: steps.hours.outputs.open == 'true'
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
@@ -159,25 +186,41 @@ jobs:
persist-credentials: false
- name: Set up Node
- if: steps.hours.outputs.open == 'true'
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
with:
node-version: "24"
- name: Select the beta to promote
id: select
- if: steps.hours.outputs.open == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- SOAK_SECONDS: "86400" # 24h online soak
- # Writes target= to GITHUB_OUTPUT (empty target => no-op). Exits non-zero
- # (fails the job) on any non-404 gh/git infra error — a fail-loud
- # retry-next-tick, never a silent skip. Logic + tests:
- # .github/scripts/select-beta-to-promote.{mjs,test.mjs}.
+ SOAK_SECONDS: "86400" # 24h online soak — the ONLY soak that can set target
+ # THE ARMING SWITCH, and the only edit that turns the fast soak tier
+ # on. While this is "false" the script computes and logs the tier it
+ # would choose but always returns soak_tier=standard, so promotion
+ # keeps using the 24h soak above and this workflow behaves exactly as
+ # it did before the tier existed. Flipping it to "true" requires a
+ # reviewed PR — read the "Soak tiers" section of RELEASES.md first:
+ # arming has prerequisites, and it is an owner decision.
+ FAST_TIER_ARMED: "false"
+ # Optional. Absent (the current state) the bug-link lookup reports
+ # "unresolvable" and the tier degrades to standard, which is the
+ # expected steady state until an operator provisions the secret.
+ LINEAR_API_KEY: ${{ secrets.LINEAR_API_KEY }}
+ # Writes target=, tier=, soak_tier= (plus the predicate inputs) to
+ # GITHUB_OUTPUT; an empty target => no-op. Exits non-zero (fails the job)
+ # on any non-404 gh/git infra error — a fail-loud retry-next-tick, never
+ # a silent skip. The soak-tier predicate is the one deliberate
+ # exception: it degrades to the slower tier rather than failing the tick.
+ # Logic + tests: .github/scripts/select-beta-to-promote.{mjs,test.mjs}.
run: node .github/scripts/select-beta-to-promote.mjs
+ # The hours gate is soak-tier-conditional: a fast-tier cut may dispatch
+ # outside the window, a standard-tier cut may not. With the tier unarmed
+ # the script can never emit soak_tier=fast, so this reduces to the previous
+ # condition — `open == 'true' && target != ''` — on every tick.
- name: Guard against an in-flight promotion, then dispatch
- if: steps.hours.outputs.open == 'true' && steps.select.outputs.target != ''
+ if: (steps.hours.outputs.open == 'true' || steps.select.outputs.soak_tier == 'fast') && steps.select.outputs.target != ''
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TARGET: ${{ steps.select.outputs.target }}