Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
TypeScript automationchromechromiume2e-testingelectronend-to-end-testingfirefoxjavascriptplaywrighttesttest-automationtestingtesting-toolswebwebkit
devops(test-results-db): CLI to compact CI test results into DuckDB (#41673)
Simon Knottb8f840cc3b99
diff --git a/.claude/skills/playwright-test-results/SKILL.md b/.claude/skills/playwright-test-results/SKILL.mdnew file mode 100644index 0000000000..aace3aff88--- /dev/null+++ b/.claude/skills/playwright-test-results/SKILL.md@@ -0,0 +1,135 @@+---+name: playwright-test-results+description: Query Playwright CI test results from the aggregated DuckDB database. Answers questions about flaky tests, failure rates, slow tests, and per-run/SHA/PR results without hunting through GitHub artifacts.+user_invocable: true+---++# Playwright Test Results (DuckDB)++A single DuckDB file holds recent Playwright CI test results, so you can answer+questions about failures, flakiness, and slow tests with plain SQL. It is+refreshed every few hours.++## Get the database++Download the latest snapshot:++```bash+npm ci # first time only, from the repo root+GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts download+```++The snapshot may be missing the newest runs. To top it up locally, run `update`:++```bash+GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts update --lookback-days 3+```++Query it with the `duckdb` CLI (or any DuckDB client):++```bash+duckdb utils/test-results-db/test-results.duckdb "SELECT count(*) FROM test_results"+```++## Schema++Single table `test_results`, one row per test result (**one row per retry**).+The columns are inferred from the parquet the reporter emits+(`tests/config/parquetReporter.ts`), plus two trailing columns this CLI adds:++| Column | Meaning |+| --- | --- |+| `run_id`, `run_attempt` | GitHub Actions run identity |+| `run_started_at` | when the run started |+| `workflow_name` | e.g. `tests 1` / `tests 2` / `tests others` / `MCP` |+| `event` | `push` / `pull_request` |+| `head_sha`, `head_branch`, `pr_number` | what was tested |+| `bot_name` | e.g. `chromium-ubuntu-22.04-node20`, `webkit-macos-15-large` — the CI bot. **OS and arch are encoded here**; there is no separate os column. |+| `project_name` | CI project = browser + suite, e.g. `chromium-page`, `webkit-library`, `playwright-test` |+| `test_title` | title path within the file, joined by ` › ` (`describe › test`) |+| `file`, `line`, `column_number` | source location (file is relative to repo root) |+| `expected_status` | `passed` / `skipped` / ... |+| `status` | actual result: `passed` / `failed` / `timedOut` / `skipped` / `interrupted` |+| `retry` | 0 = first attempt |+| `result_started_at` | when this attempt started |+| `duration_ms` | result duration |+| `error_message` | all errors joined, ANSI-stripped (NULL when none) |+| `tags` | **list** of strings, e.g. `['@slow', '@flaky']` (use list functions / `list_contains`) |+| `annotations` | list of `{type, description}` structs, e.g. `[{'type': 'skip', 'description': 'flaky on CI'}]` (empty list when none) |+| `artifact_id` | the GitHub artifact this row came from (dedupe key) |+| `ingested_at` | debug only — when this row was imported |++Notes:+- **A test is identified by `(project_name, file, test_title)`** — group on that+ tuple. (Playwright's `test_id` hash is deliberately not stored; those three+ columns are its pre-image.)+- **Flakiness is derived**, not stored. The signal that matters most is+ **cross-run**: a test whose *final* verdict (after retries) flips between+ runs — green in some, red in others. A separate **within-run** flake is a+ test a retry rescued inside a single run (`failed`→`passed`).+- **Real failures vs intentional ones:** filter `expected_status = 'passed'`.+ Tests marked `test.fail()` record `status='failed'` *with*+ `expected_status='failed'` and would otherwise dominate any "most failing" list.+- The db is size-capped by **run count**: the oldest whole runs are evicted over+ time, so it holds a recent window, not full history.++## Example queries++Group tests by `(project_name, file, test_title)` and (for failure/flakiness)+scope to `expected_status = 'passed'` so intentional `test.fail()` tests don't+skew the results.++**Flaky across runs** — the test's final verdict flips between runs (this is+what makes a red CI run ambiguous). `least(failed_runs, passed_runs)` ranks+genuinely bimodal tests above both always-broken and one-off failures:++```sql+WITH per_run AS (+ SELECT project_name, file, test_title, run_id, run_attempt,+ arg_max(status, retry) AS final_status,+ any_value(expected_status) AS expected+ FROM test_results+ GROUP BY project_name, file, test_title, run_id, run_attempt)+SELECT project_name, test_title,+ count(*) AS runs,+ count(*) FILTER (WHERE final_status IN ('failed','timedOut')) AS failed_runs,+ count(*) FILTER (WHERE final_status = 'passed') AS passed_runs,+ round(100.0 * count(*) FILTER (WHERE final_status IN ('failed','timedOut'))+ / count(*), 1) AS fail_pct+FROM per_run+WHERE expected = 'passed'+GROUP BY project_name, test_title+HAVING failed_runs > 0 AND passed_runs > 0 AND runs >= 10+ORDER BY least(failed_runs, passed_runs) DESC, failed_runs DESC+LIMIT 20;+```++**Filter by tag** (`tags` is a list, not a string):++```sql+SELECT project_name, test_title, count(*) AS runs+FROM test_results+WHERE list_contains(tags, '@slow')+GROUP BY project_name, test_title+ORDER BY runs DESC+LIMIT 20;+```++## Fetching the full detail++The db stores per-result summaries. For the full step tree / attachments / stdio,+fetch the original blob report for that run, if the run uploaded one. A row+identifies it by `run_id` + `bot_name`: the run's blob artifact is named+`blob-report-<bot_name>`.++```bash+# List the run's blob artifacts and find the one for this bot_name:+gh api /repos/microsoft/playwright/actions/runs/<run_id>/artifacts \+ --jq '.artifacts[] | select(.name | startswith("blob-report")) | {id, name}'++# Download it (name == "blob-report-<bot_name>"):+gh api /repos/microsoft/playwright/actions/artifacts/<artifact_id>/zip > blob.zip+```++Blob and parquet artifacts have a 7-day retention, so this works only for recent+runs; the db itself retains summaries longer (until run-count eviction).diff --git a/.github/workflows/update_test_results_db.yml b/.github/workflows/update_test_results_db.ymlnew file mode 100644index 0000000000..96fecd0244--- /dev/null+++ b/.github/workflows/update_test_results_db.yml@@ -0,0 +1,46 @@+name: "Update test results DB"+on:+ workflow_dispatch:+ schedule:+ - cron: "0 */3 * * *"++concurrency:+ group: test-results-db+ cancel-in-progress: false++jobs:+ update:+ name: Update DuckDB+ runs-on: ubuntu-22.04+ timeout-minutes: 60+ if: github.repository == 'microsoft/playwright'+ permissions:+ actions: read+ contents: read+ steps:+ - uses: actions/checkout@v6+ - uses: actions/setup-node@v6+ with:+ node-version: 26+ - run: npm ci+ - name: Download current database+ env:+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}+ run: node utils/test-results-db/cli.ts download+ - name: Ingest new test results+ id: ingest+ env:+ GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}+ NODE_OPTIONS: --max-old-space-size=8192+ run: node utils/test-results-db/cli.ts update --lookback-days 7 --concurrency 32+ - name: Truncate to run cap+ if: steps.ingest.outputs.imported != '0'+ run: node utils/test-results-db/cli.ts truncate --max-runs 2000+ - name: Upload database+ if: steps.ingest.outputs.imported != '0'+ uses: actions/upload-artifact@v7+ with:+ name: test-results-db+ path: utils/test-results-db/test-results.duckdb+ retention-days: 7+ overwrite: truediff --git a/utils/test-results-db/.gitignore b/utils/test-results-db/.gitignorenew file mode 100644index 0000000000..2570c55120--- /dev/null+++ b/utils/test-results-db/.gitignore@@ -0,0 +1,3 @@+# The maintained database and DuckDB sidecar files (WAL, compaction temp).+*.duckdb+*.duckdb.*diff --git a/utils/test-results-db/cli.ts b/utils/test-results-db/cli.tsnew file mode 100644index 0000000000..d06890cab4--- /dev/null+++ b/utils/test-results-db/cli.ts@@ -0,0 +1,121 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import path from 'path';+import { fileURLToPath } from 'url';+import { parseArgs } from 'util';++import { cmdDownload } from './download.ts';+import { cmdTruncate } from './truncate.ts';+import { cmdUpdate } from './update.ts';++const USAGE = `Usage: node utils/test-results-db/cli.ts <command> [options]++Compacts the per-run parquet CI artifacts into a single queryable DuckDB file.++Commands:+ download Fetch the latest maintained database artifact.+ Starts a fresh database if none exists yet.+ update [options] Ingest parquet artifacts that aren't in the database yet.+ --lookback-days <n> How many days back to scan (default 7).+ --concurrency <n> Parallel downloads per batch (default 16).+ --stop-after-seen <n> Stop after this many consecutive already-ingested+ artifacts (default 100). The list is newest-first, so+ this short-circuits the scan once caught up.+ truncate --max-runs <n> Keep only the newest <n> runs, delete the rest, compact.++Environment:+ GITHUB_TOKEN Required for 'download' and 'update'.+ TRDB_DB_PATH Database file path (default utils/test-results-db/test-results.duckdb).+`;++function defaultDbPath(): string {+ const here = path.dirname(fileURLToPath(import.meta.url));+ return path.join(here, 'test-results.duckdb');+}++const UPDATE_OPTIONS = {+ 'lookback-days': { type: 'string' },+ 'concurrency': { type: 'string' },+ 'stop-after-seen': { type: 'string' },+} as const;++const TRUNCATE_OPTIONS = {+ 'max-runs': { type: 'string' },+} as const;++type Flags = Record<string, string | boolean | undefined>;++function intFlag(flags: Flags, name: string, fallback: number): number {+ const raw = flags[name];+ if (raw === undefined)+ return fallback;+ const value = Number(raw);+ if (!Number.isInteger(value) || value <= 0)+ throw new Error(`--${name} must be a positive integer, got "${raw}"`);+ return value;+}++function requireToken(): string {+ const token = process.env.GITHUB_TOKEN;+ if (!token)+ throw new Error('GITHUB_TOKEN is required for this command.');+ return token;+}++async function main(): Promise<void> {+ const [command, ...rest] = process.argv.slice(2);+ const dbPath = process.env.TRDB_DB_PATH || defaultDbPath();++ switch (command) {+ case 'download': {+ await cmdDownload(dbPath, requireToken());+ break;+ }+ case 'update': {+ const { values } = parseArgs({ args: rest, options: UPDATE_OPTIONS, allowPositionals: false });+ await cmdUpdate(dbPath, requireToken(), {+ lookbackDays: intFlag(values, 'lookback-days', 7),+ concurrency: intFlag(values, 'concurrency', 16),+ stopAfterSeen: intFlag(values, 'stop-after-seen', 100),+ });+ break;+ }+ case 'truncate': {+ const { values } = parseArgs({ args: rest, options: TRUNCATE_OPTIONS, allowPositionals: false });+ if (values['max-runs'] === undefined)+ throw new Error('truncate requires --max-runs <n>');+ await cmdTruncate(dbPath, intFlag(values, 'max-runs', 0));+ break;+ }+ case undefined:+ case 'help':+ case '--help':+ case '-h': {+ process.stdout.write(USAGE);+ break;+ }+ default: {+ process.stderr.write(`Unknown command: ${command}\n\n${USAGE}`);+ process.exitCode = 1;+ }+ }+}++main().catch(error => {+ console.error(error instanceof Error ? error.message : error);+ process.exitCode = 1;+});diff --git a/utils/test-results-db/db.ts b/utils/test-results-db/db.tsnew file mode 100644index 0000000000..0aabfa8131--- /dev/null+++ b/utils/test-results-db/db.ts@@ -0,0 +1,156 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import fs from 'fs';++import { DuckDBInstance } from '@duckdb/node-api';++import type { DuckDBConnection } from '@duckdb/node-api';++const TABLE_NAME = 'test_results';++// The parquet already carries the full per-result schema (see+// tests/config/parquetReporter.ts). We deliberately do not re-declare those+// columns here: the table is created lazily from the first parquet we ingest,+// so it always matches whatever the reporter currently emits, plus these two+// trailing columns we add ourselves:+// - artifact_id: the GitHub artifact this row came from (our dedupe key)+// - ingested_at: when we imported it (debug only)+const INGEST_SELECT =+ `SELECT *, $id AS artifact_id, now() AS ingested_at FROM read_parquet($file)`;++// A row lives in the `test_results` table and originates from a GitHub artifact+// identified by an opaque string id.+export class TestResultsDb {+ private _instance: DuckDBInstance;+ private _conn: DuckDBConnection;+ readonly path: string;++ private constructor(instance: DuckDBInstance, conn: DuckDBConnection, path: string) {+ this._instance = instance;+ this._conn = conn;+ this.path = path;+ }++ static async open(path: string): Promise<TestResultsDb> {+ const instance = await DuckDBInstance.create(path);+ const conn = await instance.connect();+ return new TestResultsDb(instance, conn, path);+ }++ private async _tableExists(): Promise<boolean> {+ const reader = await this._conn.runAndReadAll(+ `SELECT 1 FROM information_schema.tables WHERE table_name = $name`,+ { name: TABLE_NAME });+ return reader.getRows().length > 0;+ }++ // The set of artifact ids already imported. Empty on a fresh (table-less) db.+ async ingestedArtifactIds(): Promise<Set<string>> {+ const ids = new Set<string>();+ if (!await this._tableExists())+ return ids;+ const reader = await this._conn.runAndReadAll(+ `SELECT DISTINCT artifact_id FROM ${TABLE_NAME}`);+ for (const row of reader.getRows())+ ids.add(String(row[0]));+ return ids;+ }++ // Ingest one parquet file, tagging every row with `artifactId`. The table is+ // created (schema inferred) on the first ingest; later ingests are matched by+ // column name, so a reordered/extended parquet schema still lands correctly.+ async ingestParquet(parquetFile: string, artifactId: string): Promise<void> {+ const params = { id: artifactId, file: parquetFile };+ if (!await this._tableExists())+ await this._conn.run(`CREATE TABLE ${TABLE_NAME} AS ${INGEST_SELECT} LIMIT 0`, params);+ await this._conn.run(`INSERT INTO ${TABLE_NAME} BY NAME ${INGEST_SELECT}`, params);+ }++ async runCount(): Promise<number> {+ if (!await this._tableExists())+ return 0;+ const reader = await this._conn.runAndReadAll(+ `SELECT count(DISTINCT (run_id, run_attempt)) FROM ${TABLE_NAME}`);+ return Number(reader.getRows()[0][0]);+ }++ async rowCount(): Promise<number> {+ if (!await this._tableExists())+ return 0;+ const reader = await this._conn.runAndReadAll(`SELECT count(*) FROM ${TABLE_NAME}`);+ return Number(reader.getRows()[0][0]);+ }++ // Keep the newest `maxRuns` runs (a run is a (run_id, run_attempt) pair,+ // ordered by run_started_at), delete the rest, then compact to reclaim disk.+ async truncateToRuns(maxRuns: number): Promise<void> {+ if (!await this._tableExists())+ return;+ await this._conn.run(+ `DELETE FROM ${TABLE_NAME}+ WHERE (run_id, run_attempt) NOT IN (+ SELECT run_id, run_attempt FROM ${TABLE_NAME}+ GROUP BY run_id, run_attempt+ ORDER BY max(run_started_at) DESC NULLS LAST+ LIMIT $n)`,+ { n: maxRuns });+ await this._compact();+ }++ // Copy the live rows into a fresh database file and swap it in — DuckDB's+ // DELETE leaves the space allocated, so this is what actually shrinks the file.+ private async _compact(): Promise<void> {+ const tmpPath = `${this.path}.compact.tmp`;+ for (const p of [tmpPath, `${tmpPath}.wal`]) {+ if (fs.existsSync(p))+ fs.rmSync(p);+ }+ await this._conn.run(`ATTACH '${tmpPath.replace(/'/g, `''`)}' AS compacted`);+ await this._conn.run(`CREATE TABLE compacted.${TABLE_NAME} AS SELECT * FROM ${TABLE_NAME}`);+ await this._conn.run(`DETACH compacted`);+ this.close();++ fs.rmSync(this.path);+ if (fs.existsSync(`${this.path}.wal`))+ fs.rmSync(`${this.path}.wal`);+ fs.renameSync(tmpPath, this.path);++ this._instance = await DuckDBInstance.create(this.path);+ this._conn = await this._instance.connect();+ }++ close(): void {+ this._conn.closeSync();+ this._instance.closeSync();+ }+}++export function fileSize(path: string): number {+ try {+ return fs.statSync(path).size;+ } catch {+ return 0;+ }+}++export function formatBytes(bytes: number): string {+ if (bytes < 1024)+ return `${bytes} B`;+ if (bytes < 1024 * 1024)+ return `${(bytes / 1024).toFixed(1)} KB`;+ return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;+}diff --git a/utils/test-results-db/download.ts b/utils/test-results-db/download.tsnew file mode 100644index 0000000000..f5abdb7255--- /dev/null+++ b/utils/test-results-db/download.ts@@ -0,0 +1,37 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import { TestResultsDb, fileSize, formatBytes } from './db.ts';+import { GitHubClient, extractSingle } from './github.ts';++const DB_ARTIFACT_NAME = 'test-results-db';++// Fetch the latest maintained database artifact. If none exists yet (the cron+// hasn't produced one), start from a fresh empty file.+export async function cmdDownload(dbPath: string, token: string): Promise<void> {+ const github = new GitHubClient(token);+ const artifactId = await github.findLatestArtifact(DB_ARTIFACT_NAME);+ if (!artifactId) {+ const db = await TestResultsDb.open(dbPath);+ db.close();+ console.log(`No "${DB_ARTIFACT_NAME}" artifact yet; started a fresh database at ${dbPath}`);+ return;+ }+ console.log(`Downloading "${DB_ARTIFACT_NAME}" artifact #${artifactId} ...`);+ const zip = await github.downloadArtifactZip(artifactId);+ await extractSingle(zip, '.duckdb', dbPath);+ console.log(`Downloaded database to ${dbPath} (${formatBytes(fileSize(dbPath))})`);+}diff --git a/utils/test-results-db/github.ts b/utils/test-results-db/github.tsnew file mode 100644index 0000000000..97564e8342--- /dev/null+++ b/utils/test-results-db/github.ts@@ -0,0 +1,186 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import fs from 'fs';++import yauzl from 'yauzl';++export type Artifact = {+ id: string;+ name: string;+};++type ListOptions = {+ ingested: Set<string>;+ lookbackDays: number;+ stopAfterSeen: number;+};++type RawArtifact = {+ id: number;+ name: string;+ expired: boolean;+ created_at: string;+};++// Thin GitHub REST client over global fetch. Only the three endpoints this CLI+// needs: list artifacts, list by name, and download an artifact zip.+export class GitHubClient {+ private _base: string;+ private _headers: Record<string, string>;++ constructor(token: string, repo: string = 'microsoft/playwright') {+ if (!token)+ throw new Error('A GitHub token is required (set GITHUB_TOKEN).');+ this._base = `https://api.github.com/repos/${repo}`;+ this._headers = {+ Authorization: `Bearer ${token}`,+ Accept: 'application/vnd.github+json',+ 'X-GitHub-Api-Version': '2022-11-28',+ };+ }++ // Return the not-yet-ingested artifacts matching `prefix`, walking the list+ // from the top and stopping early once we're safely into the already-ingested+ // region.+ //+ // The list is ordered by descending artifact id, and GitHub assigns ids as a+ // monotonic creation-order sequence (verified: no inversions across a 1000-+ // artifact sample). So the newest artifacts are always at the head. An artifact+ // gets its id when its upload *starts* but only appears in the list once the+ // upload *finalizes*, so the single way one can surface below where a prior+ // scan stopped is a still-in-flight upload finalizing late -- a window bounded+ // by one artifact's upload+list latency (seconds, for these KB parquet files).+ // We therefore keep scanning `stopAfterSeen` artifacts past the newest+ // already-ingested one as a cushion; any new (un-ingested) artifact resets the+ // counter. To bury a late finalizer we'd need `stopAfterSeen` newer artifacts+ // ingested above it while it uploads, i.e. an upload outlasting a whole cron+ // interval -- impossible here, so this misses nothing in practice.+ //+ // `lookbackDays` is the absolute backstop for the first run, when nothing is+ // ingested yet and the cushion never triggers.+ async listArtifacts(prefix: string, options: ListOptions): Promise<Artifact[]> {+ const { ingested, lookbackDays, stopAfterSeen } = options;+ const cutoff = Date.now() - lookbackDays * 24 * 60 * 60 * 1000;+ const out: Artifact[] = [];+ let seen = 0;+ for await (const artifact of this._paginateArtifacts('/actions/artifacts?per_page=100')) {+ const createdAt = artifact.created_at ? Date.parse(artifact.created_at) : 0;+ if (createdAt && createdAt < cutoff)+ return out;+ if (artifact.expired || !artifact.name.startsWith(prefix))+ continue;+ const id = String(artifact.id);+ if (ingested.has(id)) {+ if (++seen >= stopAfterSeen)+ return out;+ continue;+ }+ seen = 0;+ out.push({ id, name: artifact.name });+ }+ return out;+ }++ // The newest non-expired artifact with the exact name, or null if none.+ async findLatestArtifact(name: string): Promise<string | null> {+ const query = `/actions/artifacts?name=${encodeURIComponent(name)}&per_page=100`;+ for await (const artifact of this._paginateArtifacts(query)) {+ if (!artifact.expired)+ return String(artifact.id);+ }+ return null;+ }++ async downloadArtifactZip(id: string): Promise<Buffer> {+ // 302 -> blob storage; fetch follows it and strips the Authorization header+ // on the cross-origin redirect, as required by the signed URL.+ const response = await fetch(`${this._base}/actions/artifacts/${id}/zip`, { headers: this._headers });+ if (!response.ok)+ throw new Error(`Failed to download artifact ${id}: ${response.status} ${response.statusText}`);+ return Buffer.from(await response.arrayBuffer());+ }++ private async * _paginateArtifacts(path: string): AsyncGenerator<RawArtifact> {+ let url: string | null = `${this._base}${path}`;+ while (url) {+ const response = await fetch(url, { headers: this._headers });+ if (!response.ok)+ throw new Error(`GitHub API error: ${response.status} ${response.statusText} for ${url}`);+ const body = await response.json() as { artifacts?: RawArtifact[] };+ for (const artifact of body.artifacts ?? [])+ yield artifact;+ url = nextPageUrl(response.headers.get('link'));+ }+ }+}++// Parse the `rel="next"` target out of a GitHub Link header, or null if absent.+function nextPageUrl(link: string | null): string | null {+ if (!link)+ return null;+ for (const part of link.split(',')) {+ const match = part.match(/<([^>]+)>\s*;\s*rel="next"/);+ if (match)+ return match[1];+ }+ return null;+}++// Extract the first zip entry whose name ends with `ext` to `destPath`.+export async function extractSingle(zipBuffer: Buffer, ext: string, destPath: string): Promise<string> {+ return new Promise((resolve, reject) => {+ yauzl.fromBuffer(zipBuffer, { lazyEntries: true }, (error, zipFile) => {+ if (error || !zipFile) {+ reject(error ?? new Error('Failed to open zip'));+ return;+ }+ let found = false;+ zipFile.on('entry', entry => {+ if (/\/$/.test(entry.fileName) || !entry.fileName.endsWith(ext)) {+ zipFile.readEntry();+ return;+ }+ found = true;+ zipFile.openReadStream(entry, (streamError, stream) => {+ if (streamError || !stream) {+ reject(streamError ?? new Error('Failed to read zip entry'));+ return;+ }+ const out = fs.createWriteStream(destPath);+ stream.on('error', reject);+ out.on('error', reject);+ out.on('finish', () => resolve(destPath));+ stream.pipe(out);+ });+ });+ zipFile.on('end', () => {+ if (!found)+ reject(new Error(`No "*${ext}" entry found in artifact zip`));+ });+ zipFile.on('error', reject);+ zipFile.readEntry();+ });+ });+}++// Split `items` into consecutive batches of at most `size`.+export function chunk<T>(items: T[], size: number): T[][] {+ const batches: T[][] = [];+ for (let i = 0; i < items.length; i += size)+ batches.push(items.slice(i, i + size));+ return batches;+}diff --git a/utils/test-results-db/package.json b/utils/test-results-db/package.jsonnew file mode 100644index 0000000000..3dbc1ca591--- /dev/null+++ b/utils/test-results-db/package.json@@ -0,0 +1,3 @@+{+ "type": "module"+}diff --git a/utils/test-results-db/truncate.ts b/utils/test-results-db/truncate.tsnew file mode 100644index 0000000000..b15c073d7f--- /dev/null+++ b/utils/test-results-db/truncate.ts@@ -0,0 +1,32 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import { TestResultsDb, fileSize, formatBytes } from './db.ts';++// Keep only the newest `maxRuns` runs (a run is a `(run_id, run_attempt)` pair),+// delete the rest, and compact to reclaim the freed disk space.+export async function cmdTruncate(dbPath: string, maxRuns: number): Promise<void> {+ const db = await TestResultsDb.open(dbPath);+ try {+ const before = await db.runCount();+ await db.truncateToRuns(maxRuns);+ const after = await db.runCount();+ console.log(`Truncated ${before} -> ${after} runs (cap ${maxRuns})`);+ console.log(` ${await db.rowCount()} rows, size ${formatBytes(fileSize(dbPath))}`);+ } finally {+ db.close();+ }+}diff --git a/utils/test-results-db/update.ts b/utils/test-results-db/update.tsnew file mode 100644index 0000000000..52e732633b--- /dev/null+++ b/utils/test-results-db/update.ts@@ -0,0 +1,81 @@+/**+ * Copyright (c) Microsoft Corporation.+ *+ * Licensed under the Apache License, Version 2.0 (the "License");+ * you may not use this file except in compliance with the License.+ * You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software+ * distributed under the License is distributed on an "AS IS" BASIS,+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ * See the License for the specific language governing permissions and+ * limitations under the License.+ */++import fs from 'fs';+import os from 'os';+import path from 'path';++import { TestResultsDb, fileSize, formatBytes } from './db.ts';+import { GitHubClient, chunk, extractSingle } from './github.ts';++const PARQUET_ARTIFACT_PREFIX = 'parquet-report-';++export type UpdateOptions = {+ lookbackDays: number;+ concurrency: number;+ stopAfterSeen: number;+};++// Ingest the parquet artifacts that aren't in the database yet. Downloads run+// `concurrency`-wide in batches (network-bound, the slow part); each batch is+// then ingested serially on the single connection and its temp files removed,+// so disk usage stays bounded to one batch.+export async function cmdUpdate(dbPath: string, token: string, options: UpdateOptions): Promise<void> {+ const github = new GitHubClient(token);+ const db = await TestResultsDb.open(dbPath);+ try {+ const ingested = await db.ingestedArtifactIds();+ console.log(`Test results database`);+ console.log(` ${await db.rowCount()} rows from ${ingested.size} artifacts`);++ const todo = await github.listArtifacts(PARQUET_ARTIFACT_PREFIX, {+ ingested,+ lookbackDays: options.lookbackDays,+ stopAfterSeen: options.stopAfterSeen,+ });+ console.log(`\nScanning for new artifacts (last ${options.lookbackDays} days)`);+ console.log(` ${todo.length} new artifacts to import`);++ let imported = 0;+ for (const batch of chunk(todo, options.concurrency)) {+ const files = await Promise.all(batch.map(async artifact => {+ const zip = await github.downloadArtifactZip(artifact.id);+ const file = path.join(os.tmpdir(), `trdb-${artifact.id}.parquet`);+ await extractSingle(zip, '.parquet', file);+ return { id: artifact.id, file };+ }));+ for (const { id, file } of files) {+ try {+ await db.ingestParquet(file, id);+ imported++;+ } finally {+ fs.rmSync(file, { force: true });+ }+ }+ console.log(` imported ${imported}/${todo.length}`);+ }++ console.log(`\nSummary`);+ console.log(` imported ${imported} new artifacts`);+ console.log(` ${await db.rowCount()} rows from ${await db.runCount()} runs`);+ console.log(` size ${formatBytes(fileSize(dbPath))}`);++ if (process.env.GITHUB_OUTPUT)+ fs.appendFileSync(process.env.GITHUB_OUTPUT, `imported=${imported}\n`);+ } finally {+ db.close();+ }+}