Reverse engineer anything with agents, from app behavior down to native binaries.
TypeScript agent-skillsai-agentsbinary-analysisclaude-codeclicodexcordisdecompilerdeveloper-toolsdshdsh-pluginghidrahopperllmmcpmodel-context-protocolreareverse-engineeringstatic-analysistouhou
fix(tests): separate Copilot model IDs from wire models (#1647)
Ysopwnableb09dd333ab75
diff --git a/docs/installation.md b/docs/installation.mdindex 7a6359a9b..4a59ad89b 100644--- a/docs/installation.md+++ b/docs/installation.md@@ -153,6 +153,13 @@ listed after the table because its connector is not one of these files:For GitHub Copilot CLI 1.0.95, `gpt-4.1` model metadata can block ordinarychat with REA's full input-schema profile before sending a model request(`compaction_static_context_blocked`; [#1554](https://github.com/morluto/rea/issues/1554)).+For the native offline OpenAI-compatible BYOK adapter, full-profile chat and+analysis also passed with a distinct `COPILOT_PROVIDER_MODEL_ID` such as+`rea-gpt-4.1`, `COPILOT_PROVIDER_WIRE_MODEL=gpt-4.1`, and that distinct ID selected+through `copilot --model rea-gpt-4.1`. The local fixture requested+`COPILOT_PROVIDER_MAX_PROMPT_TOKENS=1000000`; configure the limit supported by+your backend. This keeps every full schema and the actual wire model intact.+It does not establish the effective client budget or fix the built-in-ID path.The existing compact profile passed native chat and analysis with the completetool inventory on Linux using a loopback model fixture. After setup, add`"REA_MCP_INPUT_SCHEMA_PROFILE": "compact"` to the `env` object of thediff --git a/docs/testing.md b/docs/testing.mdindex 3bcc882bd..3c05df096 100644--- a/docs/testing.md+++ b/docs/testing.md@@ -160,8 +160,25 @@ Use `-- chat` for ordinarychat or `REA_VERIFY_RUNTIME_ROOT` for a production-only installed package.The native offline BYOK adapter uses a loopback OpenAI completions/SSE fixtureand `gpt-5.4` model metadata; model inference and token usage are synthetic.-Set `REA_VERIFY_COPILOT_MODEL` to examine another model configuration. In the-verified client, `gpt-4.1` blocks the full input-schema profile before HTTP with+Set `REA_VERIFY_COPILOT_MODEL` to examine another client model ID and+`REA_VERIFY_COPILOT_WIRE_MODEL` for its backend model name (defaults to the client+ID). Every actual HTTP request's model is checked and recorded separately.+Both `HOME` and `USERPROFILE` point to the disposable account; native `skill add`+still selects the installed skill directory explicitly.++The same native client passed full-profile chat and analysis with the actual+wire model `gpt-4.1` under a distinct BYOK model ID:++```bash+REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- chat+REA_VERIFY_COPILOT_MODEL=rea-gpt-4.1 REA_VERIFY_COPILOT_WIRE_MODEL=gpt-4.1 npm run verify:copilot-client -- call+```++These runs retain full canonical input/output schemas, all tools, installed-skill+loading and complete Evidence readback. They use the fixture's requested+one-million-token BYOK prompt setting; effective capacity and live API acceptance+remain unknown. With the built-in `gpt-4.1` model ID, the verified client still+blocks the full input-schema profile before HTTP with`compaction_static_context_blocked`, even when requesting a larger BYOK promptcapacity. The effective capacity is unknown. The default verifier preservesthe complete catalog and full schemas. To verify the existing compact profile:diff --git a/scripts/verify-copilot-client.mjs b/scripts/verify-copilot-client.mjsindex e3980d269..98ed62efb 100644--- a/scripts/verify-copilot-client.mjs+++ b/scripts/verify-copilot-client.mjs@@ -22,6 +22,7 @@ const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;const command = process.env.REA_VERIFY_COPILOT_COMMAND ?? "copilot";const mode = process.argv[2] ?? "call";const model = process.env.REA_VERIFY_COPILOT_MODEL ?? "gpt-5.4";+const wireModel = process.env.REA_VERIFY_COPILOT_WIRE_MODEL ?? model;const inputSchemaProfile =process.env.REA_VERIFY_COPILOT_SCHEMA_PROFILE ?? "full";assert(@@ -50,6 +51,7 @@ await exec("git", ["init", "--quiet", workspace], { timeout: 30_000 });const environment = {PATH: process.env.PATH,NODE_OPTIONS: process.env.NODE_OPTIONS,+ HOME: account,USERPROFILE: account,COPILOT_HOME: profile,COPILOT_OFFLINE: "true",@@ -60,7 +62,7 @@ const environment = {COPILOT_PROVIDER_BEARER_TOKEN: "",COPILOT_PROVIDER_HEADERS: "",COPILOT_PROVIDER_MODEL_ID: model,- COPILOT_PROVIDER_WIRE_MODEL: model,+ COPILOT_PROVIDER_WIRE_MODEL: wireModel,COPILOT_PROVIDER_MAX_PROMPT_TOKENS: "1000000",COPILOT_PROVIDER_MAX_OUTPUT_TOKENS: "4096",REA_PROCESS_RUN_ID: verifier.run_id,@@ -118,10 +120,17 @@ const extractEvidence = (value) => {}return undefined;};+const wireModels = [];const fixture = await createOpenAiModelFixture({directory: lab,prefix: "copilot",onRequest: async ({ body, tools, requestIndex }) => {+ assert.equal(+ body.model,+ wireModel,+ "Copilot must send the selected wire model",+ );+ wireModels.push(body.model);// Auxiliary requests without tools cannot execute the analyst's workflow.if (tools.length === 0)return { role: "assistant", content: "REA fixture title" };@@ -424,6 +433,7 @@ try {clientVersion,mode,model,+ wireModel,inputSchemaProfile,passed: failure === undefined,artifacts: lab,@@ -440,7 +450,7 @@ try {capacityBoundary:inputSchemaProfile === "compact"? "Explicit compact profile: complete inventory and canonical server validation; advertised schemas may be reduced. This does not fix the full-profile gpt-4.1 context boundary."- : "gpt-4.1 model configuration blocks the full catalog before HTTP even with a ten-million-token requested prompt override; gpt-5.4 is the verified full-profile local fixture configuration",+ : "Complete canonical input and output schemas. Client model ID and wire model are recorded separately; effective token capacity and live provider compatibility remain unknown.",catalogSize,schemasChecked,spillReadRequested,@@ -448,6 +458,7 @@ try {skillLoaded,artifactDigest,finalSeen,+ wireModels,requests: fixture.requests,probes: fixture.probes,fixtureFailure: fixture.failure,