Understand any codebase instantly. System intelligence for codebases, built for humans and AI.
TypeScript aiai-memoryclaude-codeclicode-analysiscode-mappingcode-memorycodebase-understandingcodexdeveloper-toolsllmopenclawpersistent-memoryprogram-analysissoftware-architecturesystem-intelligencesystem-mapping
feat(cli): --offset paging for inventory and text, deterministic ix text order (HR-02B)
KageBinary699271dcbc4d
diff --git a/docs/llm-format.md b/docs/llm-format.mdindex 3c75d5c5..e8036ef8 100644--- a/docs/llm-format.md+++ b/docs/llm-format.md@@ -122,7 +122,25 @@ record: `shown` is what follows, `total` is what there was before the cut.```callers target=verify_token shown=50 total=212 resolved=48 unresolved=2-diagnostic code=results_truncated message="212 callers; showing 50. Raise --limit to see the rest."+diagnostic code=results_truncated next_offset=50 message="212 callers; showing 50. next: ix callers --offset 50 verify_token"+```++A list that can be paged says where the next page starts: `next_offset=` on+the `results_truncated` diagnostic, and a `next:` command in its message that+repeats the caller's flags with `--offset`. A page past the first carries+`offset=` on its header. A page that starts at or past the end is not an+error: it prints a `diagnostic code=offset_past_end` and exits 0.++The edge commands page over what one `/v1/expand` call returns, and the+server stops at 1000 edges. When it reports a cut, or returns a full 1000,+`total` is a floor: the list stays `results_truncated` on every page, its+message says "At least N" and names the cap, no offset is called past the+end, and the JSON summary carries `capped: true`. Past the cap, narrow the+target (`--kind`, `--path`) instead of paging.++```+callers target=verify_token offset=50 shown=50 total=212 resolved=50+diagnostic code=results_truncated next_offset=100 message="212 callers; showing 51-100. next: ix callers --offset 100 verify_token"````shown < total` is the only signal that a list is partial, and it exists@@ -134,10 +152,22 @@ Two commands cannot know a true total and say so rather than inventing one:- `ix inventory` reports `shown=N truncated=true`. The backend applies thelimit and has no count endpoint, so the CLI only knows the window it asked- for came back full.+ for came back full. Its truncation message says "More than N" rather than+ a total. `/v1/list` has no offset either, so `--offset N` fetches the N rows+ before the page too: a deep page costs O(offset). The server returns at+ most 5000 rows, so no page reaches past row 5000: a page at or beyond it+ stays `truncated=true` (JSON: `capped: true`) and says to narrow with+ `--path`, never `offset_past_end`.- `ix text` reports `shown=N scanned=M`: ripgrep's traversal is bounded by ascan window, and `scanned` is how many matches were ranked, not how many- exist.+ exist. The window holds the best `scanWindow(limit)` matches (code, then+ tests, then prose; then path; then line) over every match ripgrep finds,+ so the output is the same on every run. Pages are cut from that window, so+ keep `--limit` fixed while paging. The page that reaches the end of a full+ window has no `next_offset=`; its message says to narrow with `--path` or+ `--language` instead, and so does a page past a full window (it is not+ `offset_past_end`: more matches exist). `--format json` stays a bare array of matches with no+ paging fields; the next offset is the offset plus its length.## Rows an agent acts ondiff --git a/ix-cli/src/cli/__tests__/honest-truncation.test.ts b/ix-cli/src/cli/__tests__/honest-truncation.test.tsindex fb84af0f..eae80771 100644--- a/ix-cli/src/cli/__tests__/honest-truncation.test.ts+++ b/ix-cli/src/cli/__tests__/honest-truncation.test.ts@@ -1,12 +1,20 @@// Copyright 2026 Ix Infrastructure Inc.import { Command } from "commander";-import { describe, expect, it } from "vitest";+import { afterEach, describe, expect, it, vi } from "vitest";-import { renderEdgeResultsLlm, sliceEdgeResults, sliceRanked } from "../format.js";+const listByKind = vi.hoisted(() => vi.fn());+vi.mock("../../client/factory.js", () => ({ createClient: () => ({ listByKind }) }));+vi.mock("../resolve.js", () => ({ resolveReadSystemId: async () => undefined }));+vi.mock("../bootstrap.js", () => ({ resolveWorkspaceId: () => "ws" }));++import {+ formatTextResults, renderEdgeResultsLlm, renderTextResultsLlm, sliceEdgeResults, sliceRanked, truncationHint,+ type TextResult,+} from "../format.js";import { nextPageCommand } from "../paging.js";-import { renderInventoryLlm } from "../commands/inventory.js";-import { scanWindow, textResultPriority } from "../commands/text.js";+import { LIST_MAX_ROWS, registerInventoryCommand, renderInventoryLlm } from "../commands/inventory.js";+import { compareTextResults, keepTopK, scanWindow, textResultPriority } from "../commands/text.js";function node(name: string, path?: string) {return { id: `${name}-0000-0000-0000-000000000000`, name, kind: "function", ...(path ? { provenance: { source_uri: path } } : {}) };@@ -136,6 +144,16 @@ describe("paging with --offset", () => {expect(lines.some((l) => l.includes("offset_past_end"))).toBe(false);});+ it("--limit 0 past the first page prints the count, not an inverted range", () => {+ const s = sliceEdgeResults(mixed(), 0, 50);+ expect(s).toMatchObject({ offset: 50, shown: 0, total: 212, truncated: true });+ expect(truncationHint(s, "callers")).toBe("212 callers; showing 0. Raise --limit to see them.");+ const lines = renderEdgeResultsLlm(s, "callers", "t", "graph");+ expect(lines[1]).toBe('diagnostic code=results_truncated message="212 callers; showing 0. Raise --limit to see them."');+ expect(lines.join("\n")).not.toContain("51-50");+ expect(lines.join("\n")).not.toContain("next_offset");+ });+it("leaves the header and rows alone without an offset", () => {const lines = renderEdgeResultsLlm(sliceEdgeResults([node("a", "src/a.ts")], 50), "callers", "t", "graph");expect(lines[0]).toBe("callers target=t shown=1 total=1 resolved=1");@@ -180,6 +198,122 @@ describe("renderInventoryLlm", () => {const nodes = [{ name: "Foo", kind: "class", provenance: { source_uri: "src/a.ts" } }];expect(renderInventoryLlm("class", null, nodes)[0]).toBe("inventory kind=class shown=1");});++ it("a middle page names its offset and the next one", () => {+ const nodes = Array.from({ length: 50 }, (_, i) => ({ name: `C${i}`, kind: "class", provenance: { source_uri: "src/a.ts" } }));+ const lines = renderInventoryLlm("class", null, nodes, true, {+ offset: 50, nextOffset: 100, next: "ix inventory --kind class --offset 100",+ });+ expect(lines.slice(0, 2)).toEqual([+ "inventory kind=class offset=50 shown=50 truncated=true",+ 'diagnostic code=results_truncated next_offset=100 message="More than 100 class entities exist. next: ix inventory --kind class --offset 100, or narrow with --path."',+ ]);+ });++ it("keeps the first-page header as it was with offset 0", () => {+ const nodes = [{ name: "Foo", kind: "class", provenance: { source_uri: "src/a.ts" } }];+ expect(renderInventoryLlm("class", null, nodes, true, { offset: 0 })[0]).toBe("inventory kind=class shown=1 truncated=true");+ });++ it("past the end says so", () => {+ const lines = renderInventoryLlm("class", null, [], false, { offset: 500 });+ expect(lines).toEqual([+ "inventory kind=class offset=500 shown=0",+ 'diagnostic code=offset_past_end message="Fewer than 501 class entities exist; --offset 500 is past the end."',+ ]);+ });+});++describe("ix inventory --offset", () => {+ afterEach(() => {+ vi.restoreAllMocks();+ listByKind.mockReset();+ });++ function rows(n: number) {+ return Array.from({ length: n }, (_, i) => ({ name: `C${i + 1}`, kind: "class", provenance: { source_uri: "src/a.ts" } }));+ }++ async function run(argv: string[]): Promise<string[]> {+ const log = vi.spyOn(console, "log").mockImplementation(() => {});+ log.mockClear();+ const program = new Command();+ registerInventoryCommand(program);+ await program.parseAsync(["inventory", "--kind", "class", ...argv], { from: "user" });+ return log.mock.calls.map((c) => String(c[0]));+ }++ it("fetches offset+limit+1 rows and prints the page after the offset", async () => {+ listByKind.mockResolvedValue(rows(5));+ const out = await run(["--offset", "2", "--limit", "2", "--format", "json"]);+ expect(listByKind).toHaveBeenCalledWith("class", expect.objectContaining({ limit: 5 }));+ const json = JSON.parse(out[0]);+ expect(json).toMatchObject({ offset: 2, shown: 2, truncated: true, nextOffset: 4 });+ expect(json.byFile).toEqual([{ path: "src/a.ts", items: ["C3", "C4"] }]);+ });++ it("has no next offset when no row is left after the page", async () => {+ listByKind.mockResolvedValue(rows(4));+ const json = JSON.parse((await run(["--offset", "2", "--limit", "2", "--format", "json"]))[0]);+ expect(json).toMatchObject({ offset: 2, shown: 2, truncated: false });+ expect(json).not.toHaveProperty("nextOffset");+ });++ it("asks for no more than the server's row cap", async () => {+ listByKind.mockResolvedValue(rows(10));+ await run(["--offset", "4990", "--limit", "50", "--format", "json"]);+ expect(listByKind).toHaveBeenCalledWith("class", expect.objectContaining({ limit: LIST_MAX_ROWS }));+ });++ it("an offset at or past the server cap is not past the end", async () => {+ listByKind.mockResolvedValue(rows(LIST_MAX_ROWS));+ for (const offset of ["5000", "6000"]) {+ const out = await run(["--offset", offset, "--limit", "50", "--format", "llm"]);+ expect(out[0]).toBe(`inventory kind=class offset=${offset} shown=0 truncated=true`);+ expect(out[1]).toBe(+ `diagnostic code=results_truncated message="The server returns at most 5000 class entities, so how many exist is unknown; --offset ${offset} is beyond the rows it returns. Narrow with --path (or a more specific --kind) to see the rest."`,+ );+ expect(out.join("\n")).not.toContain("offset_past_end");+ }+ });++ it("the page that reaches the server cap stays truncated with no next page", async () => {+ listByKind.mockResolvedValue(rows(LIST_MAX_ROWS));+ const json = JSON.parse((await run(["--offset", "4990", "--limit", "50", "--format", "json"]))[0]);+ expect(json).toMatchObject({ offset: 4990, shown: 10, truncated: true, capped: true });+ expect(json).not.toHaveProperty("nextOffset");+ const llm = await run(["--offset", "4990", "--limit", "50", "--format", "llm"]);+ expect(llm[1]).toContain("this page ends at row 5000, the last it returns");+ });++ it("a --limit over the cap does not read a full server answer as the whole list", async () => {+ listByKind.mockResolvedValue(rows(LIST_MAX_ROWS));+ const json = JSON.parse((await run(["--limit", "10000", "--format", "json"]))[0]);+ expect(json).toMatchObject({ shown: LIST_MAX_ROWS, truncated: true, capped: true });+ });++ it("a short answer to a request over the cap is the whole list", async () => {+ listByKind.mockResolvedValue(rows(4995));+ const out = await run(["--offset", "6000", "--limit", "50", "--format", "llm"]);+ expect(out[1]).toContain("code=offset_past_end");+ const json = JSON.parse((await run(["--offset", "4990", "--limit", "50", "--format", "json"]))[0]);+ expect(json).toMatchObject({ shown: 5, truncated: false });+ expect(json).not.toHaveProperty("capped");+ });++ it("text mode names the cap on an empty page past it", async () => {+ listByKind.mockResolvedValue(rows(LIST_MAX_ROWS));+ const out = await run(["--offset", "5000"]);+ expect(out[0]).toContain("The server returns at most 5000 class entities");+ });++ it("names the next command in llm output, repeating --limit", async () => {+ listByKind.mockResolvedValue(rows(5));+ const out = await run(["--offset", "2", "--limit", "2", "--format", "llm"]);+ expect(out[0]).toBe("inventory kind=class offset=2 shown=2 truncated=true");+ expect(out[1]).toContain("next_offset=4");+ expect(out[1]).toContain("next: ix inventory --kind class --limit 2 --format llm --offset 4");+ });});describe("text result ranking", () => {@@ -208,4 +342,92 @@ describe("text result ranking", () => {expect(slice.rows.map((r) => r.path)).toEqual(["src/a.ts"]);expect(slice.total).toBe(3);});++ it("orders code before tests before prose, then by path, then by line", () => {+ const rows = [+ text("README.md", 1), text("src/b.ts", 9), text("src/__tests__/a.test.ts", 1),+ text("src/b.ts", 2), text("src/a.ts", 5), text("docs/a.md", 1),+ ];+ expect([...rows].sort(compareTextResults).map((r) => `${r.path}:${r.line_start}`)).toEqual([+ "src/a.ts:5", "src/b.ts:2", "src/b.ts:9", "src/__tests__/a.test.ts:1", "README.md:1", "docs/a.md:1",+ ]);+ });++ it("keepTopK keeps the best k, in order, with ties in arrival order", () => {+ const rows = [text("docs/a.md", 1, "x"), text("src/a.ts", 1, "first"), text("src/a.ts", 1, "second")];+ expect(keepTopK(rows, 2, compareTextResults).map((r) => r.snippet)).toEqual(["first", "second"]);+ expect(keepTopK(rows, 0, compareTextResults)).toEqual([]);+ });++ it("pages of one ranked list are disjoint and add up to its head", () => {+ const all = Array.from({ length: 100 }, (_, i) => text(`src/f${String(i).padStart(3, "0")}.ts`, 1)).sort(compareTextResults);+ const pages = [0, 20, 40].map((offset) => sliceRanked(all, 20, undefined, offset).rows);+ const union = pages.flat();+ expect(new Set(union.map((r) => r.path)).size).toBe(60);+ expect(union).toEqual(all.slice(0, 60));+ });+});++function text(path: string, line: number, snippet = "needle"): TextResult {+ return { path, line_start: line, line_end: line, snippet, engine: "ripgrep", score: 1 };+}++describe("renderTextResultsLlm paging", () => {+ const all = Array.from({ length: 100 }, (_, i) => text(`src/f${String(i).padStart(3, "0")}.ts`, 1));++ it("leaves the first-page header alone", () => {+ const lines = renderTextResultsLlm(sliceRanked(all, 20));+ expect(lines[0]).toBe("text shown=20 scanned=100 truncated=true");+ expect(lines[1]).toBe('diagnostic code=results_truncated next_offset=20 message="100 matches scanned; showing 20. next: --offset 20"');+ });++ it("a middle page names its offset and the next command", () => {+ const slice = { ...sliceRanked(all, 20, undefined, 20), next: "ix text --limit 20 --offset 40 needle" };+ const lines = renderTextResultsLlm(slice, { windowFull: true });+ expect(lines[0]).toBe("text offset=20 shown=20 scanned=100 truncated=true");+ expect(lines[1]).toBe('diagnostic code=results_truncated next_offset=40 message="100 matches scanned; showing 21-40. next: ix text --limit 20 --offset 40 needle"');+ });++ it("the page that reaches the end of a full window says to narrow instead", () => {+ const lines = renderTextResultsLlm(sliceRanked(all, 20, undefined, 80), { windowFull: true });+ expect(lines[0]).toBe("text offset=80 shown=20 scanned=100 truncated=true");+ expect(lines[1]).toBe('diagnostic code=results_truncated message="100 matches scanned; showing 81-100; the scan stops at 100. Narrow with --path or --language, or raise --limit."');+ expect(lines.join("\n")).not.toContain("next_offset");+ });++ it("the last page of a window that held every match has no hint", () => {+ const lines = renderTextResultsLlm(sliceRanked(all, 20, undefined, 80));+ expect(lines[0]).toBe("text offset=80 shown=20 scanned=100");+ expect(lines.some((l) => l.startsWith("diagnostic"))).toBe(false);+ });++ it("past the end of a window that held every match says so", () => {+ const lines = renderTextResultsLlm(sliceRanked(all, 20, undefined, 120));+ expect(lines).toEqual([+ "text offset=120 shown=0 scanned=100",+ 'diagnostic code=offset_past_end message="100 matches scanned; --offset 120 is past the end."',+ ]);+ });++ it("past a full window is not past the end: it stays truncated and says to narrow", () => {+ for (const offset of [100, 120]) {+ const lines = renderTextResultsLlm(sliceRanked(all, 20, undefined, offset), { windowFull: true });+ expect(lines).toEqual([+ `text offset=${offset} shown=0 scanned=100 truncated=true`,+ `diagnostic code=results_truncated message="100 matches scanned; --offset ${offset} is past the scan window, which stops at 100. Narrow with --path or --language, or raise --limit."`,+ ]);+ }+ });++ it("text mode prints the narrow hint, not past-the-end, past a full window", () => {+ const log = vi.spyOn(console, "log").mockImplementation(() => {});+ try {+ formatTextResults(sliceRanked(all, 20, undefined, 100), "text", { windowFull: true });+ const out = log.mock.calls.map((c) => String(c[0])).join("\n");+ expect(out).toContain("is past the scan window, which stops at 100");+ expect(out).not.toContain("is past the end");+ } finally {+ log.mockRestore();+ }+ });});diff --git a/ix-cli/src/cli/__tests__/mcp-structured-output.test.ts b/ix-cli/src/cli/__tests__/mcp-structured-output.test.tsindex a87aae3e..5d310353 100644--- a/ix-cli/src/cli/__tests__/mcp-structured-output.test.ts+++ b/ix-cli/src/cli/__tests__/mcp-structured-output.test.ts@@ -174,7 +174,10 @@ describe("ix mcp structured output", () => {// that number.// The full catalog's descriptions now say when to use each tool and when// not to, which is what a caller of `--tools=all` is choosing between.- expect(total).toBeLessThan(17_000);+ // Raised from 17,000 for the undescribed `offset` on ix_text and+ // ix_inventory (~64 bytes each), which print next_offset= and so have to+ // take it back. Neither is in the core set.+ expect(total).toBeLessThan(17_200);});it("attaches structuredContent only where an outputSchema was declared", async () => {diff --git a/ix-cli/src/cli/__tests__/mcp-tool-params.test.ts b/ix-cli/src/cli/__tests__/mcp-tool-params.test.tsindex f442413c..2078cc23 100644--- a/ix-cli/src/cli/__tests__/mcp-tool-params.test.ts+++ b/ix-cli/src/cli/__tests__/mcp-tool-params.test.ts@@ -76,7 +76,7 @@ describe("resolving an ambiguous symbol over MCP", () => {const tools = (await client.listTools()).tools;const has = (name: string) =>"offset" in (((tools.find((t) => t.name === name)!.inputSchema) as { properties?: Record<string, unknown> }).properties ?? {});- for (const name of ["ix_neighbors", "ix_callers", "ix_callees", "ix_imports", "ix_imported_by"]) {+ for (const name of ["ix_neighbors", "ix_callers", "ix_callees", "ix_imports", "ix_imported_by", "ix_text", "ix_inventory"]) {expect(has(name), name).toBe(true);}for (const name of ["ix_impact", "ix_read", "ix_explain"]) {@@ -84,6 +84,30 @@ describe("resolving an ambiguous symbol over MCP", () => {}});+ it("forwards --offset on ix_text and ix_inventory, which print next_offset= too", async () => {+ const { calls, runIx } = recorder();+ const client = await connect(runIx);+ await client.callTool({ name: "ix_text", arguments: { pattern: "needle", offset: 20 } });+ await client.callTool({ name: "ix_inventory", arguments: { kind: "class", offset: 50 } });+ await client.callTool({ name: "ix_text", arguments: { pattern: "needle" } });+ await client.callTool({ name: "ix_inventory", arguments: { kind: "class" } });+ expect(calls[0]).toContain("--offset=20");+ expect(calls[1]).toContain("--offset=50");+ expect(calls[2].some((a) => a.startsWith("--offset"))).toBe(false);+ expect(calls[3].some((a) => a.startsWith("--offset"))).toBe(false);+ });++ it("rejects a negative offset on ix_text and ix_inventory", async () => {+ const { calls, runIx } = recorder();+ const client = await connect(runIx);+ for (const [name, args] of [["ix_text", { pattern: "needle" }], ["ix_inventory", { kind: "class" }]] as const) {+ const result = await client.callTool({ name, arguments: { ...args, offset: -1 } })+ .then((r) => (r as CallToolResult).isError === true, () => true);+ expect(result, name).toBe(true);+ }+ expect(calls).toEqual([]);+ });+it("sends nothing for the flags the caller left out", async () => {const { calls, runIx } = recorder();const client = await connect(runIx);diff --git a/ix-cli/src/cli/__tests__/text-stream.test.ts b/ix-cli/src/cli/__tests__/text-stream.test.tsindex ae64c7cb..2c574249 100644--- a/ix-cli/src/cli/__tests__/text-stream.test.ts+++ b/ix-cli/src/cli/__tests__/text-stream.test.ts@@ -93,6 +93,92 @@ describe("text ripgrep stream lifecycle", () => {await rejected;expect(log).not.toHaveBeenCalled();});+ function matchAt(file: string, line: number) {+ return JSON.stringify({ type: "match", data: {+ path: { text: file }, line_number: line, lines: { text: `${file}:${line}` },+ } }) + "\n";+ }+ async function runArgs(argv: string[], records: string[]): Promise<string[]> {+ const program = new Command();+ registerTextCommand(program);+ const pending = program.parseAsync(["text", "--root", process.cwd(), ...argv], { from: "user" });+ for (const r of records) child.stdout.write(r);+ close(0);+ await pending;+ return log.mock.calls.map((c: unknown[]) => String(c[0]));+ }+ it("is byte-stable across arrival orders and keeps the best rows, not the first", async () => {+ // --limit 1 scans a window of 5; twelve matches overflow it, and the one+ // code match arrives last, after a full window of prose.+ const records = [+ ...Array.from({ length: 6 }, (_, i) => matchAt(`docs/d${i}.md`, i + 1)),+ ...Array.from({ length: 5 }, (_, i) => matchAt(`tests/t${i}.ts`, 1)),+ matchAt("src/code.ts", 7),+ ];+ const first = await runArgs(["needle", "--limit", "1", "--format", "json"], records);+ log.mockClear();+ child = Object.assign(new EventEmitter(), {+ stdout: new PassThrough(), stderr: new PassThrough(), kill: vi.fn(() => true),+ });+ spawn.mockReturnValue(child);+ const second = await runArgs(["needle", "--limit", "1", "--format", "json"], [...records].reverse());+ expect(second).toEqual(first);+ expect(JSON.parse(first[0]).map((r: { path: string }) => r.path)).toEqual(["src/code.ts"]);+ });+ it("asks ripgrep for one match per file past --limit, to tell a full file from a cut-off one", async () => {+ await runArgs(["needle", "--limit", "5", "--format", "llm"], []);+ const argv = spawn.mock.calls[0][1] as string[];+ expect(argv.slice(0, argv.indexOf("--"))).toEqual(["--json", "--max-count", "6"]);+ });+ it("reports a file with more matches than ripgrep's per-file cap as a truncated scan", async () => {+ // --limit 5 caps each file at 5; the sixth match is ripgrep's probe and is not shown.+ const out = await runArgs(["needle", "--limit", "5", "--format", "llm"],+ Array.from({ length: 6 }, (_, i) => matchAt("src/a.ts", i + 1)));+ expect(out[0]).toBe("text shown=5 scanned=5 truncated=true");+ expect(out[1]).toContain("Narrow with --path or --language");+ expect(out.slice(2).map((l) => l.match(/line=(\d+)/)?.[1])).toEqual(["1", "2", "3", "4", "5"]);+ });+ it.each(["1", "5"])("does not call a file with exactly --limit %s matches truncated", async (limit) => {+ const n = Number(limit);+ const out = await runArgs(["needle", "--limit", limit, "--format", "llm"],+ Array.from({ length: n }, (_, i) => matchAt("src/a.ts", i + 1)));+ expect(out[0]).toBe(`text shown=${n} scanned=${n}`);+ expect(out.some((l) => l.startsWith("diagnostic"))).toBe(false);+ });+ it("pages inside the scan window and repeats --limit in the next command", async () => {+ const records = Array.from({ length: 5 }, (_, i) => matchAt(`src/f${i}.ts`, i + 1));+ const out = await runArgs(["needle", "--limit", "2", "--offset", "2", "--format", "llm"], records);+ expect(out[0]).toBe("text offset=2 shown=2 scanned=5 truncated=true");+ expect(out[1]).toContain("next_offset=4");+ expect(out[1]).toContain("next: ix text --limit 2 --format llm --root ");+ expect(out[1]).toMatch(/--offset 4 needle"$/);+ expect(out.slice(2).map((l) => l.match(/line=(\d+)/)?.[1])).toEqual(["3", "4"]);+ });+ it("ranks a match record whose fields come in another order the same way", async () => {+ const reordered = JSON.stringify({ data: { line_number: 2, lines: { text: "late" }, path: { text: "src/a.ts" } }, type: "match" }) + "\n";+ const out = await runArgs(["needle", "--limit", "5", "--format", "json"], [matchAt("src/b.ts", 1), reordered, matchAt("src/a.ts", 1)]);+ expect(JSON.parse(out[0]).map((r: { path: string; line_start: number }) => `${r.path}:${r.line_start}`))+ .toEqual(["src/a.ts:1", "src/a.ts:2", "src/b.ts:1"]);+ });+ it("counts and names files whose paths are not UTF-8 (ripgrep sends them as bytes)", async () => {+ const bytesMatch = (raw: Buffer, line: number) => JSON.stringify({ type: "match", data: {+ path: { bytes: raw.toString("base64") }, line_number: line, lines: { text: "foo" },+ } }) + "\n";+ const a = Buffer.concat([Buffer.from("a"), Buffer.from([0xff]), Buffer.from(".txt")]);+ const b = Buffer.concat([Buffer.from("b"), Buffer.from([0xfe]), Buffer.from(".txt")]);+ const records = [1, 2, 3].flatMap((n) => [bytesMatch(a, n), bytesMatch(b, n)]).concat(matchAt("c.txt", 1));+ // Seven matches, at most three per file: under the per-file cap of 4, so nothing is cut.+ const out = await runArgs(["needle", "--limit", "4", "--format", "llm"], records);+ expect(out[0]).toBe("text shown=4 scanned=7 truncated=true");+ expect(out[1]).toContain("next_offset=4");+ expect(out.slice(2).map((l) => l.match(/path=(\S+) line=(\d+)/)?.slice(1).join(":")))+ .toEqual(["a�.txt:1", "a�.txt:2", "a�.txt:3", "b�.txt:1"]);+ });+ it("guards a term starting with a dash in the next command", async () => {+ const records = Array.from({ length: 5 }, (_, i) => matchAt(`src/f${i}.ts`, i + 1));+ const out = await runArgs(["--limit", "2", "--format", "llm", "--", "-x"], records);+ expect(out[1]).toMatch(/--offset 2 -- -x"$/);+ });it("preserves the missing-ripgrep error", async () => {const pending = run();const rejected = expect(pending).rejects.toThrow("exit 1");diff --git a/ix-cli/src/cli/commands/inventory.ts b/ix-cli/src/cli/commands/inventory.tsindex e9423ee5..0cde96fd 100644--- a/ix-cli/src/cli/commands/inventory.ts+++ b/ix-cli/src/cli/commands/inventory.ts@@ -9,6 +9,26 @@ import { relativePath, printJson } from "../format.js";import { llmLine } from "../llm.js";import { projectRow } from "../output-shape.js";import { normalizePathSeparators } from "../path-match.js";+import { nextPageCommand, OFFSET_FLAG, parseOffset } from "../paging.js";++/**+ * The most rows one /v1/list call returns. The backend clamps the request's+ * `limit` to this (Ix-memory ListRoutes.scala: `ReadBounds.limit(body.limit,+ * 200, 5000)`), and /v1/list has no offset, so no page can reach past it.+ */+export const LIST_MAX_ROWS = 5000;++/** Where an `ix inventory` page starts, and the command for the next one. */+export interface InventoryPaging {+ offset?: number;+ nextOffset?: number;+ next?: string;+ /**+ * The server stopped at LIST_MAX_ROWS before the page's rows (plus the one+ * past them) were in: how many entities exist is unknown past that row.+ */+ capped?: boolean;+}/*** Render `ix inventory` as llm records: a header line then one `file` row per@@ -21,10 +41,14 @@ import { normalizePathSeparators } from "../path-match.js";* whether the window it asked for came back full. Printing the length of a* capped list as a total is the claim that made `ix callers` report 50 callers* for a symbol with 212.+ *+ * Paged (`--offset`), the header says `offset=` and the truncation diagnostic+ * carries `next_offset=` with the command for the next page.*/export function renderInventoryLlm(- kind: string, scope: string | null, nodes: any[], truncated = false,+ kind: string, scope: string | null, nodes: any[], truncated = false, paging?: InventoryPaging,): string[] {+ const offset = paging?.offset ?? 0;const byFile = new Map<string, string[]>();const ungrouped: Array<{ name: string; kind: string }> = [];for (const n of nodes) {@@ -42,13 +66,21 @@ export function renderInventoryLlm(const lines = [llmLine("inventory", [["kind", kind],["scope", scope ?? undefined],+ ["offset", offset > 0 ? offset : undefined],["shown", nodes.length],["truncated", truncated || undefined],])];if (truncated) {lines.push(llmLine("diagnostic", [["code", "results_truncated"],- ["message", `More than ${nodes.length} ${kind} entities exist. Raise --limit, or narrow with --path.`],+ ["next_offset", paging?.nextOffset],+ ["message", inventoryTruncationMessage(kind, offset, nodes.length, paging)],+ ]));+ }+ if (nodes.length === 0 && offset > 0 && !truncated) {+ lines.push(llmLine("diagnostic", [+ ["code", "offset_past_end"],+ ["message", inventoryPastEndMessage(kind, offset)],]));}for (const [path, names] of byFile) {@@ -60,6 +92,26 @@ export function renderInventoryLlm(return lines;}+function inventoryTruncationMessage(kind: string, offset: number, shown: number, paging?: InventoryPaging): string {+ if (paging?.capped && paging.nextOffset === undefined) return inventoryCapMessage(kind, offset, shown);+ const more = `More than ${offset + shown} ${kind} entities exist.`;+ if (paging?.nextOffset === undefined) return `${more} Raise --limit, or narrow with --path.`;+ return `${more} next: ${paging.next ?? `--offset ${paging.nextOffset}`}, or narrow with --path.`;+}++/** A page at or past the server's row cap: the list goes on, but no offset reaches it. */+function inventoryCapMessage(kind: string, offset: number, shown: number): string {+ const head = `The server returns at most ${LIST_MAX_ROWS} ${kind} entities, so how many exist is unknown`;+ const where = shown > 0+ ? `this page ends at row ${offset + shown}, the last it returns`+ : `--offset ${offset} is beyond the rows it returns`;+ return `${head}; ${where}. Narrow with --path (or a more specific --kind) to see the rest.`;+}++function inventoryPastEndMessage(kind: string, offset: number): string {+ return `Fewer than ${offset + 1} ${kind} entities exist; --offset ${offset} is past the end.`;+}+export function registerInventoryCommand(program: Command): void {program.command("inventory")@@ -67,6 +119,7 @@ export function registerInventoryCommand(program: Command): void {.requiredOption("--kind <kind>", "Entity kind to list (class, method, function, file, module, etc.)").option("--path <path>", "Filter by source file path substring").option("--limit <n>", "Max results", "50")+ .option("--offset <n>", OFFSET_FLAG, "0").option("--format <fmt>", "Output format (text|json|llm)", "text").addHelpText("after", `Examples:@@ -74,12 +127,14 @@ Examples:ix inventory --kind class --path memory-layerix inventory --kind file --path ix-cli/srcix inventory --kind method --limit 100+ ix inventory --kind class --offset 50ix inventory --format json --kind class`).action(async (opts: {- kind: string; path?: string; limit: string; format: string;- }) => {+ kind: string; path?: string; limit: string; offset?: string; format: string;+ }, cmd: Command) => {const client = createClient();const limit = parseInt(opts.limit, 10);+ const offset = parseOffset(opts.offset);// Pass --path as a server-side scope so the LIMIT is applied AFTER path// filtering (otherwise a capped fetch can truncate away the target before@@ -93,9 +148,18 @@ Examples:const pathNeedle = opts.path ? normalizePathSeparators(opts.path) : undefined;// One past the limit, so a full window is distinguishable from an exact// fit. The backend has no count endpoint, so this is the only honest- // answer available: "there is at least one more".- let nodes = await client.listByKind(opts.kind, { limit: limit + 1, workspaceId: systemId ? undefined : resolveWorkspaceId(), scope: pathNeedle, systemId });- const truncated = nodes.length > limit;+ // answer available: "there is at least one more". /v1/list has no+ // offset, so a page past the first fetches the rows before it too and+ // drops them here; its order is stable, so pages line up.+ //+ // The server returns at most LIST_MAX_ROWS rows. A request over that+ // which comes back full may have been cut, so the list is not known to+ // end there: it stays truncated and no offset is called past the end.+ const wanted = offset + limit + 1;+ let nodes = await client.listByKind(opts.kind, { limit: Math.min(wanted, LIST_MAX_ROWS), workspaceId: systemId ? undefined : resolveWorkspaceId(), scope: pathNeedle, systemId });+ const capped = wanted > LIST_MAX_ROWS && nodes.length >= LIST_MAX_ROWS;+ const moreFetched = nodes.length > offset + limit;+ const truncated = moreFetched || capped;if (pathNeedle) {nodes = nodes.filter((n) => {@@ -109,8 +173,13 @@ Examples:});}- nodes = nodes.slice(0, limit);+ nodes = nodes.slice(offset, offset + limit);const scope = opts.path ?? null;+ const paging: InventoryPaging = { offset, ...(capped ? { capped } : {}) };+ if (moreFetched && nodes.length > 0) {+ paging.nextOffset = offset + nodes.length;+ paging.next = nextPageCommand(cmd, [], paging.nextOffset);+ }if (opts.format === "json") {// Group by file to avoid repeating the same path on every entry@@ -135,8 +204,12 @@ Examples:const output: any = {kind: opts.kind,scope: scope ?? undefined,+ offset,shown: nodes.length,truncated,+ ...(paging.nextOffset !== undefined ? { nextOffset: paging.nextOffset } : {}),+ // The server's row cap was reached: the entity count is unknown.+ ...(capped ? { capped: true } : {}),byFile: grouped,};if (ungrouped.length > 0) output.ungrouped = ungrouped;@@ -145,16 +218,24 @@ Examples:}if (opts.format === "llm") {- for (const line of renderInventoryLlm(opts.kind, scope, nodes, truncated)) console.log(line);+ for (const line of renderInventoryLlm(opts.kind, scope, nodes, truncated, paging)) console.log(line);return;}if (nodes.length === 0) {- console.log(`No ${opts.kind} entities found${scope ? ` in ${scope}` : ""}.`);+ console.log(capped+ ? inventoryCapMessage(opts.kind, offset, 0)+ : offset > 0 && !truncated+ ? inventoryPastEndMessage(opts.kind, offset)+ : `No ${opts.kind} entities found${scope ? ` in ${scope}` : ""}.`);return;}- const more = truncated ? " (more exist — raise --limit)" : "";+ const more = truncated+ ? paging.next ? ` (more exist — next: ${paging.next})`+ : capped ? ` (${inventoryCapMessage(opts.kind, offset, nodes.length)})`+ : " (more exist — raise --limit)"+ : "";console.log(`Inventory: ${nodes.length} ${opts.kind} entities${scope ? ` in ${scope}` : ""}${more}`);for (const n of nodes) {const name = n.name || n.attrs?.name || "(unnamed)";diff --git a/ix-cli/src/cli/commands/text.ts b/ix-cli/src/cli/commands/text.tsindex 7dfd4d9a..5df6d560 100644--- a/ix-cli/src/cli/commands/text.ts+++ b/ix-cli/src/cli/commands/text.ts@@ -5,19 +5,62 @@ import { createInterface } from "node:readline";import path from "node:path";import type { Command } from "commander";import { formatTextResults, printJson, sliceRanked, type TextResult } from "../format.js";+import { nextPageCommand, OFFSET_FLAG, parseOffset } from "../paging.js";import { isPathInsideResolvedRoot, resolveWorkspaceRoot } from "../config.js";import { stderr } from "../stderr.js";import { llmError } from "../llm.js";import { currentRunSignal } from "../../client/run-signal.js";-type RunRipgrep = (args: string[], limit: number) => Promise<{ stdout: string }>;+/**+ * Runs ripgrep and returns at most `limit` match lines. `matched`, when given,+ * counts every match record within the per-file `--max-count`, kept or not.+ * `capped` says some file has more matches than `--max-count` let through, so+ * the scan left matches out. The command sorts whatever comes back, so an+ * injected executor's order does not matter.+ */+type RunRipgrep = (args: string[], limit: number) => Promise<{ stdout: string; matched?: number; capped?: boolean }>;++/**+ * Keep the best `k` of `items` under `compare`, in that order.+ *+ * The sort is stable, so items that compare equal keep the order they arrived+ * in — which makes arrival the last tie-break without a sequence field.+ */+export function keepTopK<T>(items: T[], k: number, compare: (a: T, b: T) => number): T[] {+ const sorted = [...items].sort(compare);+ return sorted.length > k ? sorted.slice(0, Math.max(k, 0)) : sorted;+}-async function runRipgrep(args: string[], limit: number): Promise<{ stdout: string }> {+/**+ * The default executor. `root` is the workspace root matches are ranked+ * against: the band regexes have to see workspace-relative paths, or a root+ * under a `tests/` directory would put every match in the test band.+ *+ * ripgrep is asked for one match per file more than `--max-count` says. That+ * extra match is never kept or counted; it only shows the file had more. A+ * file with exactly `--max-count` matches looks the same as a cut-off one+ * otherwise, and calling it cut off told every `--limit 1` search to narrow.+ */+async function runRipgrep(args: string[], limit: number, root: string): Promise<{ stdout: string; matched: number; capped: boolean }> {+ const { argv, perFileCap } = probePerFileCap(args);+ const perFile = new Map<string, number>();+ let capped = false;return new Promise((resolve, reject) => {// Killed when an `ix mcp` tool call that runs it times out (run-signal.ts).- const child = spawn("rg", args, { stdio: ["ignore", "pipe", "pipe"], signal: currentRunSignal() });+ const child = spawn("rg", argv, { stdio: ["ignore", "pipe", "pipe"], signal: currentRunSignal() });const lines = createInterface({ input: child.stdout, crlfDelay: Infinity });- const matches: string[] = [];+ // The best `limit` matches seen so far, not the first `limit` to arrive:+ // ripgrep's walker is parallel, so which matches arrive first changes from+ // run to run. Ranking every match keeps the output byte-stable. The buffer+ // is cut back to `limit` whenever it reaches twice that, so memory stays+ // bounded however many matches there are, and once it has been cut a+ // match that ranks no better than the last row kept is dropped unseen+ // (a later arrival loses a tie, as it would in the stable sort).+ let kept: Array<{ key: RankKey; line: string }> = [];+ let worst: RankKey | undefined;+ const byKey = (a: { key: RankKey }, b: { key: RankKey }) => compareRankKeys(a.key, b.key);+ const relative = new Map<string, { path: string; band: number }>();+ let matched = 0;let errorOutput = "";child.stderr.setEncoding("utf8");@@ -25,20 +68,36 @@ async function runRipgrep(args: string[], limit: number): Promise<{ stdout: strierrorOutput = (errorOutput + chunk).slice(0, 64 * 1024);});lines.on("line", (line: string) => {- if (matches.length >= limit) return;- try {- if (JSON.parse(line).type !== "match") return;- } catch {+ const found = matchRankFields(line);+ if (!found) return;+ const inFile = (perFile.get(found.file) ?? 0) + 1;+ perFile.set(found.file, inFile);+ if (inFile > perFileCap) {+ capped = true;return;}- matches.push(line);+ matched++;+ if (limit <= 0) return;+ let file = relative.get(found.file);+ if (!file) {+ const rel = workspaceRelativePath(found.name, root);+ file = { path: rel, band: textResultPriority(rel) };+ relative.set(found.file, file);+ }+ const key: RankKey = { band: file.band, path: file.path, line: found.line, file: found.file };+ if (worst && compareRankKeys(key, worst) >= 0) return;+ kept.push({ key, line });+ if (kept.length >= limit * 2) {+ kept = keepTopK(kept, limit, byKey);+ worst = kept[kept.length - 1]?.key;+ }});child.on("error", reject);child.on("close", (code, signal) => {lines.close();// Drain through exit so errors after the retained matches are not hidden.if (code === 0 || code === 1) {- resolve({ stdout: matches.join("\n") });+ resolve({ stdout: keepTopK(kept, limit, byKey).map((m) => m.line).join("\n"), matched, capped });} else {reject(Object.assign(new Error("ripgrep failed"), {code,@@ -50,6 +109,118 @@ async function runRipgrep(args: string[], limit: number): Promise<{ stdout: stri});}+/**+ * `args` with its own `--max-count` (the one before `--`; anything after is+ * the caller's term and path) raised by one, and the cap it had. A cap of 0+ * or none at all is left as it is: there is nothing to tell apart.+ */+function probePerFileCap(args: string[]): { argv: string[]; perFileCap: number } {+ const separator = args.indexOf("--");+ const flags = separator >= 0 ? args.slice(0, separator) : args;+ const capIndex = flags.indexOf("--max-count");+ const cap = capIndex >= 0 ? Number(flags[capIndex + 1]) : NaN;+ if (!Number.isSafeInteger(cap) || cap <= 0) {+ return { argv: args, perFileCap: Number.isFinite(cap) ? cap : Infinity };+ }+ const argv = [...args];+ argv[capIndex + 1] = String(cap + 1);+ return { argv, perFileCap: cap };+}++/**+ * What a match ranks by: band, then workspace-relative path, then line. `file`+ * is ripgrep's own name for the file (see `ripgrepPath`) and only breaks ties+ * between two files whose decoded paths are the same string.+ */+interface RankKey {+ band: number;+ path: string;+ line: number;+ file?: string;+}++function compareRankKeys(a: RankKey, b: RankKey): number {+ if (a.band !== b.band) return a.band - b.band;+ if (a.path !== b.path) return a.path < b.path ? -1 : 1;+ if (a.line !== b.line) return a.line - b.line;+ const fileA = a.file ?? "";+ const fileB = b.file ?? "";+ return fileA === fileB ? 0 : fileA < fileB ? -1 : 1;+}++/**+ * A path from a ripgrep `--json` record. ripgrep sends a path that is not+ * valid UTF-8 as `{"bytes": <base64>}` instead of `{"text": ...}`. `id` tells+ * files apart exactly (it keys the per-file `--max-count` count); `name` is+ * the path to show and rank by, with bytes that are not UTF-8 decoded to+ * U+FFFD. Reading only `text` gave every such file the name "", so they all+ * counted as one file and printed as the workspace root.+ */+function ripgrepPath(p: any): { id: string; name: string } {+ if (typeof p?.text === "string") return { id: `t:${p.text}`, name: p.text };+ if (typeof p?.bytes === "string") {+ return { id: `b:${p.bytes}`, name: Buffer.from(p.bytes, "base64").toString("utf8") };+ }+ return { id: "", name: "" };+}++// ripgrep's `--json` match records start with these fields in this order. A+// line that does not fit (a non-UTF-8 path arrives as `bytes`, say) falls back+// to a full parse; nothing is ranked on a guess.+const MATCH_HEAD = /^\{"type":"match","data":\{"path":\{"text":("(?:[^"\\]|\\.)*")\}/;+const LINE_NUMBER = /"line_number":(\d+)/;++/**+ * The path and line of a ripgrep `match` record, or undefined for any other+ * line. Most records are read with two regexes instead of a full JSON.parse:+ * ripgrep is drained to exit and every match is ranked, so on a dense term+ * the parse was most of the cost.+ */+function matchRankFields(line: string): { file: string; name: string; line: number } | undefined {+ const head = MATCH_HEAD.exec(line);+ const number = head ? LINE_NUMBER.exec(line) : null;+ if (head && number) {+ try {+ const { id, name } = ripgrepPath({ text: JSON.parse(head[1]) });+ return { file: id, name, line: Number(number[1]) };+ } catch {+ // fall through to the full parse+ }+ }+ let parsed: any;+ try {+ parsed = JSON.parse(line);+ } catch {+ return undefined;+ }+ if (parsed?.type !== "match") return undefined;+ const { id, name } = ripgrepPath(parsed.data?.path);+ return { file: id, name, line: parsed.data?.line_number ?? 0 };+}++/** A path ripgrep printed, relative to `root` with forward slashes. */+function workspaceRelativePath(filePath: string, root: string): string {+ const absoluteFilePath = path.isAbsolute(filePath)+ ? filePath+ : path.resolve(process.cwd(), filePath);+ return path.relative(root, absoluteFilePath).split(path.sep).join("/");+}++/** One ripgrep `match` record as a result, its path relative to `root`. */+function textResultFromMatch(data: any, root: string): TextResult {+ const filePath = ripgrepPath(data?.path).name;+ const lineNum = data?.line_number ?? 0;+ return {+ path: workspaceRelativePath(filePath, root),+ line_start: lineNum,+ line_end: lineNum,+ snippet: data?.lines?.text ?? "",+ engine: "ripgrep",+ score: 1.0,+ language: inferLanguage(filePath),+ };+}+export function resolveTextSearchPath(root: string, searchPath: string): string {return path.resolve(root, searchPath);}@@ -63,6 +234,11 @@ export function resolveTextSearchPath(root: string, searchPath: string): string* Ranking the twenty rows it already cut cannot fix that; the scan has to see* more than it prints. Bounded so a term matching everything is still one* cheap pass.+ *+ * The window is the best `scanWindow(limit)` matches under+ * `compareTextResults`, taken over every match ripgrep finds — not the first+ * that many to arrive, which would differ between runs. `--offset` pages are+ * cut from this one window, so they only line up while `--limit` stays fixed.*/const SCAN_WINDOW_FACTOR = 5;const SCAN_WINDOW_MAX = 500;@@ -84,9 +260,9 @@ const GENERATED_FILE = /(^|\/)(package-lock\.json|[^/]*\.lock|[^/]*\.sum)$/i;* definition, or edited with any confidence. A test at least shows the symbol* in use.*- * Only these three bands, and the sort is stable, so within a band ripgrep's- * order survives. The alternative — scoring by line content — guesses at what- * the caller meant from a single grep term.+ * Only these three bands; within a band rows go by path and line (see+ * `compareTextResults`). The alternative — scoring by line content — guesses+ * at what the caller meant from a single grep term.*/export function textResultPriority(filePath: string): number {const normalized = filePath.split(path.sep).join("/");@@ -95,18 +271,36 @@ export function textResultPriority(filePath: string): number {return 0;}-export function registerTextCommand(program: Command, executeRipgrep: RunRipgrep = runRipgrep): void {+/**+ * The total order `ix text` ranks by: band (`textResultPriority`), then path,+ * then line. Paths compare by code unit, not locale, so the order is the same+ * on every machine. Callers sort stably, so equal rows keep arrival order.+ */+export function compareTextResults(a: TextResult, b: TextResult): number {+ return compareRankKeys(+ { band: textResultPriority(a.path), path: a.path, line: a.line_start },+ { band: textResultPriority(b.path), path: b.path, line: b.line_start },+ );+}++export function registerTextCommand(program: Command, executeRipgrep?: RunRipgrep): void {program.command("text <term>").description("Fast lexical/text search across the codebase (uses ripgrep)").option("--limit <n>", "Max results", "20")+ .option("--offset <n>", OFFSET_FLAG, "0").option("--path <dir>", "Restrict search to a workspace-relative directory", ".").option("--language <lang>", "Filter by language (python, typescript, scala, etc.)").option("--format <fmt>", "Output format (text|json|llm)", "text").option("--root <dir>", "Workspace root directory")- .addHelpText("after", "\nExamples:\n ix text verify_token --language python\n ix text \"class.*Service\" --limit 10 --format json\n ix text TODO --path src/")- .action(async (term: string, opts: { limit: string; path: string; format: string; language?: string; root?: string }) => {+ .addHelpText("after", "\nExamples:\n ix text verify_token --language python\n ix text \"class.*Service\" --limit 10 --format json\n ix text TODO --path src/\n ix text TODO --offset 20")+ .action(async (+ term: string,+ opts: { limit: string; offset?: string; path: string; format: string; language?: string; root?: string },+ cmd: Command,+ ) => {const limit = parseInt(opts.limit, 10);+ const offset = parseOffset(opts.offset);const root = path.resolve(resolveWorkspaceRoot(opts.root));const searchPath = resolveTextSearchPath(root, opts.path);if (!isPathInsideResolvedRoot(root, searchPath)) {@@ -147,39 +341,27 @@ export function registerTextCommand(program: Command, executeRipgrep: RunRipgrep// other tool until the timeout and leaves the child process behind.rgArgs.push("--", term, searchPath);- const { stdout } = await executeRipgrep(rgArgs, window);+ const run: RunRipgrep = executeRipgrep ?? ((args, k) => runRipgrep(args, k, root));+ const { stdout, matched, capped } = await run(rgArgs, window);const results: TextResult[] = [];for (const line of stdout.split("\n")) {if (!line.trim()) continue;try {const parsed = JSON.parse(line);- if (parsed.type === "match") {- const data = parsed.data;- const filePath = data.path?.text ?? "";- const absoluteFilePath = path.isAbsolute(filePath)- ? filePath- : path.resolve(process.cwd(), filePath);- const lineNum = data.line_number ?? 0;- results.push({- path: path.relative(root, absoluteFilePath).split(path.sep).join("/"),- line_start: lineNum,- line_end: lineNum,- snippet: data.lines?.text ?? "",- engine: "ripgrep",- score: 1.0,- language: inferLanguage(filePath),- });- }+ if (parsed.type === "match") results.push(textResultFromMatch(parsed.data, root));} catch {// skip non-JSON lines}}- formatTextResults(- sliceRanked(results, limit, (r) => textResultPriority(r.path)),- opts.format,- );+ // Sorted here as well as in the executor, so the order never depends+ // on which executor ran or how it delivered the rows. The window is+ // then ranked, and every page is a plain cut of it.+ results.sort(compareTextResults);+ const slice = sliceRanked(results.slice(0, window), limit, undefined, offset);+ if (slice.nextOffset !== undefined) slice.next = nextPageCommand(cmd, [term], slice.nextOffset);+ formatTextResults(slice, opts.format, { windowFull: (matched ?? results.length) > window || capped === true });} catch (err: any) {if (err.code === "ENOENT") {stderr("Error: ripgrep (rg) is not installed. Install it: https://github.com/BurntSushi/ripgrep#installation");@@ -187,7 +369,7 @@ export function registerTextCommand(program: Command, executeRipgrep: RunRipgrep}// rg exit 1 = no matches; render as empty result setif (err.code === 1 || err.status === 1) {- formatTextResults(sliceRanked([] as TextResult[], limit), opts.format);+ formatTextResults(sliceRanked([] as TextResult[], limit, undefined, offset), opts.format);return;}// rg exit 2 = usage/regex error (e.g. literal "\n" in regex,diff --git a/ix-cli/src/cli/format.ts b/ix-cli/src/cli/format.tsindex 879dc885..80e5917c 100644--- a/ix-cli/src/cli/format.ts+++ b/ix-cli/src/cli/format.ts@@ -198,11 +198,20 @@ export function expandCapped(result: { nodes?: unknown[]; edges?: unknown[]; tru|| (result.nodes?.length ?? 0) >= EXPAND_MAX_EDGES;}+/**+ * Which rows a page printed: `51-100` past the first page, the bare count on+ * it. A page that printed nothing is `0` — with `--limit 0 --offset 50` the+ * range form would read `51-50`.+ */+function pageRange(slice: Slice<unknown>): string {+ const offset = slice.offset ?? 0;+ return offset > 0 && slice.shown > 0 ? `${offset + 1}-${offset + slice.shown}` : `${slice.shown}`;+}+/** What was left out, and the one flag that gets it. */export function truncationHint(slice: Slice<unknown>, relation: string): string {- const offset = slice.offset ?? 0;if (slice.capped) return cappedHint(slice, relation);- const range = offset > 0 ? `${offset + 1}-${offset + slice.shown}` : `${slice.shown}`;+ const range = pageRange(slice);if (slice.nextOffset === undefined) return `${slice.total} ${relation}; showing ${range}. Raise --limit to see them.`;const next = slice.next ?? `--offset ${slice.nextOffset}`;return `${slice.total} ${relation}; showing ${range}. next: ${next}`;@@ -216,11 +225,10 @@ function cappedHint(slice: Slice<unknown>, relation: string): string {const offset = slice.offset ?? 0;const head = `At least ${slice.total} ${relation} (the server returns at most ${EXPAND_MAX_EDGES} edges)`;if (slice.nextOffset !== undefined) {- const range = offset > 0 ? `${offset + 1}-${offset + slice.shown}` : `${slice.shown}`;- return `${head}; showing ${range}. next: ${slice.next ?? `--offset ${slice.nextOffset}`}`;+ return `${head}; showing ${pageRange(slice)}. next: ${slice.next ?? `--offset ${slice.nextOffset}`}`;}const where = slice.shown > 0- ? `showing ${offset > 0 ? `${offset + 1}-${offset + slice.shown}` : slice.shown}, the end of what it returned`+ ? `showing ${pageRange(slice)}, the end of what it returned`: offset > 0 ? `--offset ${offset} is beyond the rows it returned` : "showing 0";return `${head}; ${where}. Narrow the target with --kind or --path to see the rest.`;}@@ -661,22 +669,76 @@ export interface TextResult {symbol_hint?: string;}+/** How `ix text` paged: whether the scan window filled before ripgrep ran out of matches. */+export interface TextPaging {+ /**+ * More matches existed than the scan window kept. Pages are cut from that+ * window, so the page that reaches its end has no next page even though the+ * search does — the caller has to narrow the search instead.+ */+ windowFull?: boolean;+}++/**+ * Whether a text page starts past the last match. Only when the scan window+ * held every match: a full window is not the end of the search, so an offset+ * beyond it is not past the end of anything the caller can know.+ */+function isTextPastEnd(slice: Slice<TextResult>, paging?: TextPaging): boolean {+ return !paging?.windowFull && isPastEnd(slice);+}++/**+ * What a text page left out, and how to get it: the next page while the scan+ * window has rows left, a narrower search once the page reached the window's+ * end. Undefined when nothing was left out.+ */+function textTruncationMessage(slice: Slice<TextResult>, paging?: TextPaging): string | undefined {+ if (isTextPastEnd(slice, paging)) return undefined;+ const range = pageRange(slice);+ if (slice.nextOffset !== undefined) {+ return `${slice.total} matches scanned; showing ${range}. next: ${slice.next ?? `--offset ${slice.nextOffset}`}`;+ }+ if (paging?.windowFull && slice.shown > 0) {+ return `${slice.total} matches scanned; showing ${range}; the scan stops at ${slice.total}. Narrow with --path or --language, or raise --limit.`;+ }+ if (paging?.windowFull && (slice.offset ?? 0) >= slice.total) {+ return `${slice.total} matches scanned; --offset ${slice.offset ?? 0} is past the scan window, which stops at ${slice.total}. Narrow with --path or --language, or raise --limit.`;+ }+ if (slice.truncated) {+ return `${slice.total} matches scanned; showing ${range}. Narrow with --path or --language, or raise --limit.`;+ }+ return undefined;+}+/*** Render lexical search hits as llm `match` records (one per line).** No `lang=`: the extension on `path` already says it, and on a 20-row answer* the field was ~80 tokens of nothing. `--format json` keeps `language`.+ *+ * `scanned` is the size of the ranked scan window, not the number of matches+ * in the repo; pages (`--offset`) are cut from that window.*/-export function renderTextResultsLlm(slice: Slice<TextResult>): string[] {+export function renderTextResultsLlm(slice: Slice<TextResult>, paging?: TextPaging): string[] {+ const message = textTruncationMessage(slice, paging);const lines = [llmLine("text", [+ ["offset", slice.offset ? slice.offset : undefined],["shown", slice.shown],["scanned", slice.total],- ["truncated", slice.truncated || undefined],+ ["truncated", slice.truncated || message !== undefined || undefined],])];- if (slice.truncated) {+ if (message !== undefined) {lines.push(llmLine("diagnostic", [["code", "results_truncated"],- ["message", `${slice.total} matches scanned; showing the ${slice.shown} highest-ranked. Narrow with --path or --language, or raise --limit.`],+ ["next_offset", slice.nextOffset],+ ["message", message],+ ]));+ }+ if (slice.rows.length === 0 && isTextPastEnd(slice, paging)) {+ lines.push(llmLine("diagnostic", [+ ["code", "offset_past_end"],+ ["message", pastEndMessage(slice, "matches scanned")],]));}for (const r of slice.rows) {@@ -690,9 +752,12 @@ export function renderTextResultsLlm(slice: Slice<TextResult>): string[] {return lines;}-export function formatTextResults(slice: Slice<TextResult>, format: string): void {+export function formatTextResults(slice: Slice<TextResult>, format: string, paging?: TextPaging): void {const results = slice.rows;if (format === "json") {+ // A raw array on purpose: the plugin runners parse it as one. Paging state+ // stays out of it; a JSON caller's next offset is its offset plus the+ // array's length.const compact = results.map(r => ({...r,path: relativePath(r.path) ?? r.path,@@ -701,11 +766,13 @@ export function formatTextResults(slice: Slice<TextResult>, format: string): voireturn;}if (format === "llm") {- for (const line of renderTextResultsLlm(slice)) console.log(line);+ for (const line of renderTextResultsLlm(slice, paging)) console.log(line);return;}if (results.length === 0) {- console.log("No text matches found.");+ console.log(isTextPastEnd(slice, paging)+ ? pastEndMessage(slice, "matches scanned")+ : textTruncationMessage(slice, paging) ?? "No text matches found.");return;}for (const r of results) {@@ -715,6 +782,8 @@ export function formatTextResults(slice: Slice<TextResult>, format: string): voi` ${lang}${chalk.dim(r.path)}${chalk.cyan(":" + r.line_start)} ${sym}${r.snippet.trim()}`);}+ const message = textTruncationMessage(slice, paging);+ if (message !== undefined) console.log(chalk.dim(` ${message}`));}export interface LocateResult {diff --git a/ix-cli/src/mcp/server.ts b/ix-cli/src/mcp/server.tsindex 5732d695..a47926cc 100644--- a/ix-cli/src/mcp/server.ts+++ b/ix-cli/src/mcp/server.ts@@ -435,11 +435,14 @@ export function createIxMcpServer(options: CreateServerOptions = {}): McpServerlimit: z.number().int().min(1).max(100).default(20),path: z.string().min(1).optional(),language: z.string().min(1).optional(),+ // Takes back next_offset= (see EDGE_TOOL_INPUT); no description, same budget.+ offset: z.number().int().min(0).max(100000).optional(),},async (input) => {const options = [`--limit=${numberArg(input, "limit")}`];pushOption(options, "--path", input.path);pushOption(options, "--language", input.language);+ if (typeof input.offset === "number") options.push(`--offset=${numberArg(input, "offset")}`);return runFormatted(runIx, "ix_text", ix("text", [stringArg(input, "pattern")], options));},);@@ -610,10 +613,13 @@ export function createIxMcpServer(options: CreateServerOptions = {}): McpServerkind: z.string().min(1).default("file"),path: z.string().min(1).optional(),limit: z.number().int().min(1).max(500).default(50),+ // Takes back next_offset= (see EDGE_TOOL_INPUT); no description, same budget.+ offset: z.number().int().min(0).max(100000).optional(),},async (input) => {const options = [`--kind=${stringArg(input, "kind")}`, `--limit=${numberArg(input, "limit")}`];pushOption(options, "--path", input.path);+ if (typeof input.offset === "number") options.push(`--offset=${numberArg(input, "offset")}`);return runFormatted(runIx, "ix_inventory", ix("inventory", [], options));},);diff --git a/skills/ix/references/commands.md b/skills/ix/references/commands.mdindex 33bd131c..cda004a6 100644--- a/skills/ix/references/commands.md+++ b/skills/ix/references/commands.md@@ -189,8 +189,12 @@ ix inventory --kind function --path auth.py- Always use `--limit` to cap result sets. Not on `ix diff --summary` or`ix diff --full`, though: each already fixes the volume, so `--limit`alongside either is refused rather than silently ignored.-- Page `callers`, `callees`, `imports`, `imported-by` and `contains` with- `--offset`, following the `next:` hint (or `next_offset=` in llm output).+- Page `callers`, `callees`, `imports`, `imported-by`, `contains`,+ `inventory` and `text` with `--offset`, following the `next:` hint (or+ `next_offset=` in llm output). Keep `--limit` the same while paging. `text`+ pages stop at its scan window; past that, narrow with `--path` or+ `--language`. The server returns at most 1000 edges and 5000 `inventory`+ rows; past those caps the output stays truncated and says to narrow.- Use `--path` or `--language` to restrict text searches.- Reuse exact entity IDs from previous JSON results when chaining commands.- Decompose large questions into multiple targeted calls.diff --git a/skills/ix/references/flags.md b/skills/ix/references/flags.mdindex fef27c00..28db9096 100644--- a/skills/ix/references/flags.md+++ b/skills/ix/references/flags.md@@ -510,6 +510,7 @@ List entities by kind with optional path scoping.| `--kind` | `<kind>` | — | Entity kind to list (class, method, function, file, module, etc.) || `--path` | `<path>` | — | Filter by source file path substring || `--limit` | `<n>` | `50` | Max results |+| `--offset` | `<n>` | `0` | Skip the first N results (use the next: hint to page) || `--format` | `text\|json\|llm` | `text` | Output format — see [output-formats.md](output-formats.md) || `--pretty` | — | off | Indent JSON output; the default only when stdout is a terminal || `--quiet` | — | off | Drop headers, section titles and advisory hints |@@ -784,6 +785,7 @@ Fast lexical/text search across the codebase (uses ripgrep).| Flag | Value | Default | Effect ||---|---|---|---|| `--limit` | `<n>` | `20` | Max results |+| `--offset` | `<n>` | `0` | Skip the first N results (use the next: hint to page) || `--path` | `<dir>` | `.` | Restrict search to a workspace-relative directory || `--language` | `<lang>` | — | Filter by language (python, typescript, scala, etc.) || `--format` | `text\|json\|llm` | `text` | Output format — see [output-formats.md](output-formats.md) |