Skip to content

Business operating system for Claude Code — 57 skills, 21 agents, smart daemon. Unified inbox (WhatsApp/Email/Slack/Telegram), autonomous PR merge, full-AWS monitoring, revenue (Stripe+RevenueCat), e-commerce (Shopify), marketing (Klaviyo/Meta/GA4), voice (Bland/ElevenLabs), APM (Datadog/NewRelic/OTEL), YOLO mode.

Remove CRS, make CLIProxyAPI the only multi-account path (rebased, supersedes #819) (#830)

Aurora Capital723a2268a779
diff --git a/.cursor/approval-policies/daemon-policy.md b/.cursor/approval-policies/daemon-policy.md
index 49a16e49..1fa18b42 100644
--- a/.cursor/approval-policies/daemon-policy.md
+++ b/.cursor/approval-policies/daemon-policy.md
@@ -15,4 +15,4 @@
## Reviewer routing
- Request platform maintainers
-- Flag PRs that touch rotator or CRS-related scripts for extra scrutiny
+- Flag PRs that touch rotator or account-auth scripts for extra scrutiny
diff --git a/claude-ops/CHANGELOG.md b/claude-ops/CHANGELOG.md
index 2d4d2eb7..a9d5c386 100644
--- a/claude-ops/CHANGELOG.md
+++ b/claude-ops/CHANGELOG.md
@@ -1,6 +1,26 @@
## Unreleased
- fix(ops-mac): the application firewall is opt-in. `/ops:mac fix` no longer offers to enable it and no longer lists it as a recommendation; the audit reports its state and stops. Turning the firewall on breaks local listeners (dev servers, MCP proxies, VNC, tunnels) and the user rarely connects the breakage to a `fix` run they asked for other reasons. `socketfilterfw` now runs only when the user asks for the firewall by name in that turn.
+- **Removed the CRS relay backend.** claude-relay-service handed every session a
+ static `cr_` bearer token pinned into `settings.json`, and the plugin had to keep
+ base URL and token in lockstep across respawns, settings overlays and daemons.
+ CLIProxyAPI is now the only supported path for multi-account rotation and OAuth
+ seat management: it holds one OAuth seat file per account, and `rotate.mjs`
+ already writes those files during a rotation.
+ Deleted: the `crs-*` daemons in `scripts/account-rotation/`, all of
+ `scripts/crsproxy-reauth/`, the three `install-crs-*.sh` installers, the five
+ `crs-*` systemd units, the `com.claude-ops.crs-*` plists, and
+ `docs/runbooks/crs-full-tuning-plan.md`.
+ Renamed, same behaviour: `crs-pool-config.mjs` → `rotation-config.mjs`,
+ `crs-refresh-lock.mjs` → `refresh-lock.mjs`, `crs-reconciler-state.mjs` →
+ `reconciler-state.mjs`.
+ Migration on an existing machine: route mode `crs-oauth` is read as `oauth`, a
+ `crs` config block is read as the `rotation` block, `CRS_REFRESH_*` env names
+ still work, and reconciler state files fall back to their old names. Background
+ respawn strips a leftover relay base URL, `cr_` token and `--settings` overlay
+ rather than re-applying them. `claude-stack doctor` reports a leftover pair as
+ `legacy_relay_env`; `claude-stack route --mode oauth` clears it.
+ `ops-accounts crs` and `crs-tick` now print where rotation lives and exit 2.
- fix(whatsapp): `apply-patches.py` Fix Q no longer appends a duplicate `/api/app_state_status` route. The sentinel gated on the comment text, which Fix Y later rewrote to `Fix Q/Y`, so on any tree that had taken Fix Y the check missed and a second `http.HandleFunc` for the same path was added. Go panics at startup on a duplicate pattern, so the bridge died on the next `go build`. The sentinel is now the handler registration itself, which is stable across variants.
- docs(whatsapp): stop assuming the MCP server is named `whatsapp`. Multi-account installs run one bridge and one server per account (`whatsapp-<label>`) and have no plain `mcp__whatsapp__*`, so every hardcoded reference failed with an unknown tool. Adds CLAUDE.md Rule 8 (resolve the name at runtime, pick the account deliberately, never send from the wrong number, Rule 6 applies to every variant) and points ops-inbox, ops-comms, comms-scanner, and `scripts/whatsapp/ENDPOINTS.md` at it. Also warns that `allowed-tools` entries are exact strings and that host send-gate hook matchers pinned to the literal `mcp__whatsapp__send_message` silently stop firing when an account is renamed or added.
- Security: replace direct/unattended Claude browser reauthentication with short-lived HMAC-approved, single-use staged enrollment and atomic rollback activation; legacy reauth wrappers now fail closed.
diff --git a/claude-ops/README.md b/claude-ops/README.md
index 2e8917be..1680c3ac 100644
--- a/claude-ops/README.md
+++ b/claude-ops/README.md
@@ -29,7 +29,7 @@ Purely additive — no v1 behaviour changes by default. Full migration guide: [`
| Direct main-push warning | `git push *` on `main`/`master`/`prod` | `permissionDecision: ask` to confirm | (always-on) | — | [safety-hooks.md](docs/safety-hooks.md) |
| Task\* tracking nudge | every Nth non-Task tool call | One-line `additionalContext` reminder | (transparent) | `task_reminder_enabled` (default `true`) | [CHANGELOG](CHANGELOG.md#4-universal-task-tracking-nudge) |
| Recap marquee | every 30s | Multi-session digest in tmux `status-right` / `statusLine` | [`/ops:recap`](skills/ops-recap/SKILL.md) | `recap_marquee_enabled` (default `true`) | [recap.md](docs/recap.md) |
-| Multi-account Claude Max rotator | quota approaching cap | launchd daemon swaps `Claude Code-credentials` keychain entry to next account | [`/ops:rotate`](skills/ops-rotate/SKILL.md), [`/ops:rotate-setup`](skills/ops-rotate-setup/SKILL.md) | `account_rotation_enabled` (default `false` — opt-in) | [CHANGELOG](CHANGELOG.md#6-multi-account-claude-max-rotator) |
+| Multi-account Claude Max rotator | quota approaching cap | launchd daemon swaps `Claude Code-credentials` keychain entry to next account, and writes CLIProxyAPI seat files for pooled setups | [`/ops:rotate`](skills/ops-rotate/SKILL.md), [`/ops:rotate-setup`](skills/ops-rotate-setup/SKILL.md) | `account_rotation_enabled` (default `false` — opt-in) | [CHANGELOG](CHANGELOG.md#6-multi-account-claude-max-rotator) |
### v2 quick start — deploy auto-fix in 60 seconds
diff --git a/claude-ops/bin/ops-accounts b/claude-ops/bin/ops-accounts
index 1f8efee0..87d83f1b 100755
--- a/claude-ops/bin/ops-accounts
+++ b/claude-ops/bin/ops-accounts
@@ -31,7 +31,9 @@ CLI="$ROOT/scripts/ops-accounts/cli.mjs"
# Shell-owned commands (dual-backend + gateway + seats). Node CLI only covers
# multi-provider status/reauth/switch/refresh — do not exec it for these.
cmd="${1:-status}"
-SHELL_CMDS="util setup crs crs-tick seats gateway grok-proxy help -h --help"
+# crs/crs-tick are retired; they stay listed so the shell case can print the
+# migration message instead of the Node CLI reporting an unknown command.
+SHELL_CMDS="util setup seats gateway grok-proxy crs crs-tick help -h --help"
if [[ -f "$CLI" ]]; then
case " $SHELL_CMDS " in
*" $cmd "*) ;;
@@ -59,7 +61,6 @@ plugin_root() {
ROOT="$(plugin_root)"
ROT="${ROOT}/scripts/account-rotation"
GROK_PROXY_URL="${GROK_PROXY_URL:-}"
-CRS_GROK_BASE_URL="${CRS_GROK_BASE_URL:-}"
print_claude() {
echo "=== Claude ==="
@@ -91,11 +92,6 @@ PY
else
echo "grok-cli-auth-proxy not configured or unreachable (set GROK_PROXY_URL)"
fi
- if [[ -n "$CRS_GROK_BASE_URL" ]] && curl -fsS -m 2 -o /dev/null -w "crs_grok_hop HTTP %{http_code}\n" "${CRS_GROK_BASE_URL%/}/models" 2>/dev/null; then
- :
- else
- echo "crs_grok_hop: not reachable (optional thin hop)"
- fi
}
# --- CLIProxyAPI backend (optional local multi-provider proxy) ---
# Convention: single static binary + config dir; override via env if relocated.
@@ -295,19 +291,10 @@ PY
echo "Quick reauth Claude: ops-accounts reauth claude <email>"
echo "Quick reauth Grok: ops-accounts reauth grok <email>"
;;
- crs)
- if [[ -x "$ROT/crs-priority-daemon.sh" ]]; then
- bash "$ROT/crs-priority-daemon.sh" --status 2>&1 | head -40
- else
- echo "CRS priority script missing (optional)"
- fi
- ;;
- crs-tick)
- if [[ -x "$ROT/crs-priority-daemon.sh" ]]; then
- bash "$ROT/crs-priority-daemon.sh" "$@"
- else
- echo "CRS priority script missing"; exit 1
- fi
+ crs|crs-tick)
+ echo "The relay backend has been removed. Multi-account rotation and OAuth seat" >&2
+ echo "management run through CLIProxyAPI: use 'ops-accounts util' or /ops:ops-fleet." >&2
+ exit 2
;;
seats)
sub="${1:-status}"
@@ -365,7 +352,7 @@ PY
fi
nohup node "$GW" >>"${TMPDIR:-/tmp}/ops-accounts-gateway.log" 2>&1 &
echo "started pid=$! log=${TMPDIR:-/tmp}/ops-accounts-gateway.log url=http://${GW_HOST}:${GW_PORT}"
- echo "migrate: CRS_BASE_URL → OPS_ACCOUNTS_GATEWAY_URL=http://${GW_HOST}:${GW_PORT}"
+ echo "set OPS_ACCOUNTS_GATEWAY_URL=http://${GW_HOST}:${GW_PORT} for clients"
;;
path)
echo "$GW"
@@ -392,7 +379,6 @@ ops-accounts — multi-provider seats (canonical; /ops:rotate is alias)
refresh [all|claude|grok]
reauth claude <email> | reauth grok <email>
setup [claude|grok]
- crs | crs-tick
seats [status|list|set|toggle|import-claude-config|tick]
gateway status|start|path|self-test|config
grok-proxy status # endpoint configured by GROK_PROXY_URL
diff --git a/claude-ops/bin/ops-bg b/claude-ops/bin/ops-bg
index 59ef2675..0d61239e 100755
--- a/claude-ops/bin/ops-bg
+++ b/claude-ops/bin/ops-bg
@@ -86,23 +86,12 @@ cmd_dispatch() {
exit 2
fi
local prompt="$*"
- # CRS routing: every bg session ops-bg spawns inherits the sanctioned overlay
- # (ANTHROPIC_BASE_URL + CLAUDE_CODE_OAUTH_TOKEN). Preflight refuses to spawn if
- # the overlay is half-set or the key is 401-rejected. Opt out with CRS_OVERLAY="".
- local CRS_OVERLAY="${CRS_OVERLAY-$HOME/.claude/crs-session-settings.json}"
- local -a crs_args=()
- if [ -n "$CRS_OVERLAY" ]; then
- if [ -x "$HOME/.claude/scripts/crs-overlay-preflight.sh" ]; then
- if ! bash "$HOME/.claude/scripts/crs-overlay-preflight.sh" "$CRS_OVERLAY" >&2; then
- echo "ops-bg: CRS overlay preflight failed — refusing to spawn" >&2
- exit 1
- fi
- fi
- crs_args=(--settings "$CRS_OVERLAY")
- fi
- local -a args=(--bg "${crs_args[@]}" --model "$model" --effort "$effort" --add-dir "$add_dir")
+ # Background sessions run on Claude Code's own OAuth credential. A retired
+ # relay backend used to inject a --settings overlay here that pinned every
+ # session to a base URL and a static relay token; nothing is injected now.
+ local -a args=(--bg --model "$model" --effort "$effort" --add-dir "$add_dir")
if [ -n "$name" ]; then
- args=(--bg "${crs_args[@]}" --name "$name" --model "$model" --effort "$effort" --add-dir "$add_dir")
+ args=(--bg --name "$name" --model "$model" --effort "$effort" --add-dir "$add_dir")
fi
if [ -n "$agent" ]; then
args+=(--agent "$agent")
diff --git a/claude-ops/docs/ops/HOST-VS-PLUGIN-ROTATE-GAP.md b/claude-ops/docs/ops/HOST-VS-PLUGIN-ROTATE-GAP.md
index c2572d11..0231f3e0 100644
--- a/claude-ops/docs/ops/HOST-VS-PLUGIN-ROTATE-GAP.md
+++ b/claude-ops/docs/ops/HOST-VS-PLUGIN-ROTATE-GAP.md
@@ -11,19 +11,19 @@ Public-repo clean: no host emails, IPs, or personal account lists.
|--------|------------------------|------------------------|
| `rotate.mjs` | ~8981 lines | ~5275 lines (after this pass: soft captcha hooks) |
| Captcha helper stack | Present (full) | **Ported this pass** |
-| CRS reconcilers / priority | Present (ops + host forks) | Present (portable subset) |
| Magic-link autoloop + reauth env | Host thicker; plugin has portable `reauth-env.mjs` | Present (PR #723 era) |
| Production systemd/user units | Still point at **host** path | Templates only (`%h` / `$CLAUDE_PLUGIN_ROOT`) |
-**CRS is optional.** Standalone rotate-magic (keychain + `rotate.mjs` /
-`rotate-magic.mjs`) is enough for one or a few accounts. CRS is multi-account
-load balancing / rate-limit spreading for a relay pool.
+Standalone rotate-magic (keychain + `rotate.mjs` / `rotate-magic.mjs`) is enough
+for one or a few accounts. To pool several accounts at once, run CLIProxyAPI;
+`rotate.mjs` writes its seat files directly. The relay backend this document
+used to compare against has been removed from the plugin.
## Line-count delta (key modules)
| File | Host | Plugin (this branch) | Notes |
|------|------|----------------------|--------|
-| `rotate.mjs` | 8981 | ~5275 | Host has magic-link-auto depth, OAuth harden, CRS sync, vault safety, captcha call sites throughout |
+| `rotate.mjs` | 8981 | ~5275 | Host has magic-link-auto depth, OAuth harden, vault safety, captcha call sites throughout |
| `captcha-helper.mjs` | 1023 | 1023 | Ported; residualAfterWait + multi-provider |
| `visual-captcha-solver.mjs` | 496 | ~531 | Ported; portable desktop-act resolve + self-contained Gemini check |
| `bright-data-cascade.mjs` | 296 | 296 | Ported; env-only zones |
@@ -47,7 +47,7 @@ Minimal set operators still lean on from host for unattended reauth:
5. `virtual-display.mjs` — headed X seat helpers
6. `magic-link-cleanup.mjs` — gog trash after reauth
7. `proxy-helper.mjs` / `oauth-proxy-fetch.mjs` — residential egress helpers
-8. `crs-operator.mjs`, `crs-token-refresher.mjs`, host-only CRS ops scripts
+8. Host-only relay operator scripts (retired along with the relay backend)
9. One-off captcha debug drivers (`solve-*.mjs`, `click-turnstile*.mjs`, `.tmp-*`) — **not** for port
10. `refresh-tokens.mjs` — unit `claude-token-refresh.service` still references host
@@ -59,7 +59,7 @@ Also host-only (ops noise / personal debug): dozens of `click-*`, `trace-*`,
- `reauth-env.mjs`, `setup-account.mjs`, `bulk-setup-token.mjs`
- `provider-env.mjs` / `provider-router.mjs` (multi-provider direction)
- `vault-linux.mjs`, credit digest / kapture claim helpers
-- `crs-reconciler-state.mjs`, shell wrappers for reconcilers
+- `reconciler-state.mjs`, shell wrappers for reconcilers
- Public `config.example.json`, launchd templates, install agents
- `rotate-captcha-soft.mjs`, `ensure-rotate-captcha-hooks.mjs`, `rotate-magic.mjs` ensure-on-start
@@ -73,12 +73,7 @@ Do **not** repoint production units in this PR. Observed on a typical box
| `claude-rotation-daemon.service` | `%h/.claude/scripts/account-rotation/daemon.mjs` |
| `claude-account-rotation.service` | host `daemon.mjs` + host WorkingDirectory |
| `claude-token-refresh.service` | host `refresh-tokens.mjs` |
-| `crs-magic-link-autoloop.service` | host `magic-link-autoloop.sh` |
-| `crs-priority.service` | host `crs-priority-daemon.sh` |
-| `crs-token-feed.service` | host `crs-token-feed.sh` |
-| `crs-bedrock-guard.service` | host `crs-bedrock-guard.mjs` |
-| `crs-egress-failover.service` | host `crs-egress-failover.sh` |
-| Drop-ins `20-crs-config.conf` | `CRS_CONFIG=%h/.claude/scripts/account-rotation/config.json` |
+| Retired relay units (`crs-*.service`, `20-crs-config.conf` drop-ins) | host relay scripts; the plugin no longer ships or installs any of them |
Safe later end-state (not this PR): repoint to
`$CLAUDE_PLUGIN_ROOT/scripts/account-rotation/…` after cascade parity, then
@@ -96,7 +91,7 @@ After this pass, cascade **modules** exist; full host `rotate.mjs` still has:
6. Residential Chrome / PAC / EFG proxy args for reauth browser
7. `secrets-bootstrap` before solver use
8. `virtual-display` ensure for headed seats
-9. CRS push helpers (`--sync-crs-all` style)
+9. Relay push helpers (retired; CLIProxyAPI seat files are written inline)
10. Stricter OAuth callback / credential race handling
## Minimal port set (standalone plugin rotate-magic)
@@ -115,7 +110,7 @@ After this pass, cascade **modules** exist; full host `rotate.mjs` still has:
- (keeps after-magic-link verify in `finishMagicLinkLogin`)
- `rotate-magic.mjs` runs ensure before spawning `rotate.mjs`
- Thin `rotate-magic.mjs` entry → `rotate.mjs --magic-link`
-- CRS optional UX in `/ops:rotate-setup` and `/ops:rotate` skills
+- Pooling UX in `/ops:rotate-setup` and `/ops:ops-fleet` skills
- Units: `__tests__/captcha-cascade.test.mjs`, `__tests__/ensure-rotate-captcha-hooks.test.mjs`
**Apply hooks on a checkout**
@@ -134,7 +129,7 @@ node scripts/account-rotation/__tests__/run-captcha-unit-tests.mjs
- `rotation-safety` / vault harden merge
- Repoint host systemd → plugin (operator change, not automatic)
-## How to run standalone (no CRS)
+## How to run standalone (single seat)
```bash
export CLAUDE_PLUGIN_ROOT=/path/to/claude-ops
diff --git a/claude-ops/docs/ops/OPS-ACCOUNTS-GATEWAY.md b/claude-ops/docs/ops/OPS-ACCOUNTS-GATEWAY.md
index 3644a4dc..f86037da 100644
--- a/claude-ops/docs/ops/OPS-ACCOUNTS-GATEWAY.md
+++ b/claude-ops/docs/ops/OPS-ACCOUNTS-GATEWAY.md
@@ -2,9 +2,14 @@
**Status:** skeleton shipped (`scripts/account-rotation/ops-accounts-gateway.mjs`).
-**Goal:** Optional thin OpenAI-compat gateway so fleets that today set
-`base_url=http://127.0.0.1:3005` (CRS) can point at a plugin-owned process
-**without** installing weishaw/claude-relay-service.
+**Goal:** an optional thin OpenAI-compat gateway, owned by this plugin, for
+fleets that want one local endpoint in front of their seats.
+
+For pooling several accounts, the supported backend is
+[CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI). `rotate.mjs`
+writes its seat files directly, `/ops:ops-fleet` reads the pool, and
+`/ops:ops-rotate-setup` enrolls a seat. This gateway is a smaller, separate
+thing: a local compat shim, not an account pool.
## What it does (skeleton)
@@ -24,6 +29,7 @@
- Grafana / Prometheus stack by default
- Require Docker
- Re-implement Grok seat table inside Claude account rows
+- Hold long-lived bearer tokens on behalf of a session
## CLI
@@ -36,14 +42,7 @@ Env: `OPS_ACCOUNTS_GATEWAY_HOST` (127.0.0.1), `OPS_ACCOUNTS_GATEWAY_PORT` (3005)
`OPS_ACCOUNTS_GATEWAY_KEY`, `OPS_ACCOUNTS_STATE_PATH`, `GROK_PROXY_URL`,
`CLAUDE_UPSTREAM_URL`.
-## Migration
-
-```
-CRS_BASE_URL → OPS_ACCOUNTS_GATEWAY_URL
-# harnesses keep OpenAI-compat client shape
-```
-
-External CRS remains **advanced-only** for operators who want the full CRS admin UI.
+Clients set `OPS_ACCOUNTS_GATEWAY_URL` and keep their OpenAI-compat shape.
## Build order
@@ -51,9 +50,4 @@ External CRS remains **advanced-only** for operators who want the full CRS admin
2. Plugin Grok proxy (**done**)
3. Gateway skeleton (**this**) — health, auth, grok hop, Claude seat pick + 501
4. Claude vault OAuth hop (next) — wire seat → access token → Anthropic/OpenAI upstream
-5. Harness matrix update (`docs/ops` CLI path matrix) — `CRS_BASE_URL` → `OPS_ACCOUNTS_GATEWAY_URL`
-
-## License
-
-If any line is copied from CRS (MIT), keep attribution. Prefer rewrite of the
-proxy surface over vendoring the monorepo.
+5. Harness matrix update (`docs/ops` CLI path matrix)
diff --git a/claude-ops/docs/ops/OPS-ACCOUNTS-VISION.md b/claude-ops/docs/ops/OPS-ACCOUNTS-VISION.md
index 7d9cc3b5..ba20005d 100644
--- a/claude-ops/docs/ops/OPS-ACCOUNTS-VISION.md
+++ b/claude-ops/docs/ops/OPS-ACCOUNTS-VISION.md
@@ -21,7 +21,7 @@ Ideal box: **public claude-ops plugin only** — no host-only `grok-rotate`, `co
## Anthropic reference stack (what “done” looks like)
-Claude already has (in plugin + optional CRS):
+Claude already has:
| Layer | Role |
|-------|------|
@@ -31,7 +31,7 @@ Claude already has (in plugin + optional CRS):
| Reauth | magic-link-autoloop + captcha cascade (residential egress) |
| Util | 5h / 7d utilization queries |
| Switch | keychain swap / force-rotate |
-| Optional LB | CRS pool priority, 429 cooldown, token feed |
+| Pooling | CLIProxyAPI seat files, written by `rotate.mjs` |
**Every other provider gets the same layers**, with provider-native OAuth and util APIs.
@@ -39,19 +39,19 @@ Claude already has (in plugin + optional CRS):
| Provider | OAuth / capture | Reauth | Util | Switch / LB notes |
|----------|-----------------|--------|------|-------------------|
-| **Claude (Anthropic)** | magic-link + captcha | magic-link-autoloop | 5h/7d | keychain daemon; CRS optional |
-| **Grok (xAI SuperGrok)** | device-code + Google (dcli) | residential cascade (EFG SOCKS → Bright Data residential → ISP → mobile) | weekly / 429 | `grok-cli-auth-proxy` RR; CRS is thin relay only |
+| **Claude (Anthropic)** | magic-link + captcha | magic-link-autoloop | 5h/7d | keychain daemon; CLIProxyAPI for pooling |
+| **Grok (xAI SuperGrok)** | device-code + Google (dcli) | residential cascade (EFG SOCKS → Bright Data residential → ISP → mobile) | weekly / 429 | `grok-cli-auth-proxy` round-robin |
| **OpenAI / Codex** | OAuth / API key | codex OAuth bridge | usage API | `codex-rotate` absorbed |
| **CLIProxyAPI (local multi-provider pool)** | per-provider OAuth login flows (browser/device-code) | proxy-internal token refresh | management API (pool-level) | single static binary + flat-file auth-dir; chosen over sub2api (no Postgres/Redis to maintain). Optional local backend: `providers/cliproxy.mjs` adapter reports auth-dir inventory; `bin/ops-accounts` prints per-provider counts + `util <provider>` views. Env overrides: `CLIPROXYAPI_HOME`, `CLIPROXYAPI_AUTH_DIR`, `CLIPROXYAPI_BASE_URL`. |
| **Factory** | provider OAuth / tokens | native reauth | quota-feed-factory patterns | full adapter |
| **Cursor** | Cursor account OAuth | browser/device OAuth | plan limits if available | full adapter |
| **Extensible** | adapter interface | same contract | best-effort | `provider-env` / `provider-router` |
-### Grok + CRS (do not mis-sell)
+### Grok (do not mis-sell)
-Grok Build can use `CRS_GROK_BASE_URL` for its CRS-compatible endpoint. CRS **does not** own a SuperGrok account pool. It forwards to the configured **`grok-cli-auth-proxy`** (`GROK_PROXY_URL`), which holds OAuth seats and round-robins. ops-accounts status must show:
+No relay owns a SuperGrok account pool. Requests go to **`grok-cli-auth-proxy`** (`GROK_PROXY_URL`), which holds the OAuth seats and round-robins between them. ops-accounts status must show:
-1. CRS hop present/absent
+1. Proxy reachable or not
2. Proxy RR seat health + exhaust cooldowns
3. Slot ↔ `auth.json` sync
@@ -75,61 +75,48 @@ No secrets in skill output. Provider-scoped vault service names. Env-templated p
|-------|------|----------------|
| **0** | Skill merge + contract — `/ops:accounts` owns verbs; rotate/setup are aliases; thin router over existing engines | One entrypoint; multi-provider status |
| **1** | Claude plugin parity (captcha cascade, standalone reauth) | Host not required for Claude magic-link |
-| **2** | Claude CRS optional | Detect; never required for single seat |
+| **2** | Claude pooling via CLIProxyAPI | Detect; never required for a single seat |
| **3** | Grok adapter complete (plugin reauth + residential cascade + slot sync + proxy status) | Multi-seat healthy under weekly cap without host-only ops |
| **4** | OpenAI/Codex + Factory adapters | same verbs |
| **5** | Cursor adapter | same verbs |
| **6** | Companions required co-install | done (#726) |
| **7** | Cutover: units → `$CLAUDE_PLUGIN_ROOT`; retire host forks | public plugin only |
-| **8** | Optional bundled multi-provider LB | license-safe cherry-pick if still needed |
+| **8** | Optional local OpenAI-compat gateway | `ops-accounts gateway`, no external service required |
-## CRS cherry-pick (so people do not need CRS)
+## Pooling: CLIProxyAPI only
-### Split CRS into three jobs
+An earlier design ran a self-hosted relay (claude-relay-service) in front of the
+accounts. That is removed. It handed every session a static `cr_` bearer token
+pinned into `settings.json`, and the plugin had to keep base URL and token in
+lockstep across respawns, overlays and daemons. One half-applied pair meant a
+401 loop, and the token itself was long-lived credential material sitting in a
+settings file.
-1. **API gateway** — OpenAI-compat `:3005`, `cr_` keys, harness routing
-2. **Claude seat LB** — schedulable pool, 429/weekly park, token feed
-3. **Grok thin hop** — forward to host SuperGrok OAuth proxy (not a CRS pool)
+CLIProxyAPI replaces it, and is the only supported multi-account path:
-ops-accounts absorbs (1)+(2) as optional **ops-accounts-gateway** + **policy daemons**,
-and treats (3) as the **Grok provider backend** (port `grok-cli-auth-proxy` into
-the plugin). External CRS remains advanced-only.
+- one OAuth seat file per account in its auth dir, refreshed by the proxy
+- `rotate.mjs` writes those seat files during a rotation, so there is no separate
+ token feed to keep in sync
+- `providers/cliproxy.mjs` reports the inventory; `/ops:ops-fleet` shows the pool
+- `CLIPROXYAPI_HOME`, `CLIPROXYAPI_AUTH_DIR`, `CLIPROXYAPI_BASE_URL` locate it
-### Already portable in this repo (rename, do not re-clone CRS)
+What this plugin still owns:
-- `crs-priority-daemon.mjs` — pool priority / schedulable policy
-- `crs-429-cooldown.mjs` — rate-limit cooldown
-- `refresh-tokens.mjs` — identity-verified proactive vault refresh
-- `crs-token-feed.mjs` — identity-verified vault → pool publication
-- `crs-pool-config.mjs`, `crs-health-watch.mjs`, `crs-bedrock-guard.mjs` — optional
-
-These currently *call* CRS admin APIs. Dual-mode target:
-
-- `backend=crs` — talk to external CRS (compat)
-- `backend=local` — file/SQLite seat state, no Docker
-
-### Build sequence
-
-| Step | Deliverable |
-|------|-------------|
-| A | Local seat-state backend for policy daemons (no CRS) |
-| B | Plugin-owned Grok OAuth proxy (today’s `:31845` logic) |
-| C | Optional OpenAI-compat gateway replacing `:3005` for most harnesses |
-| D | Docs: migrate `CRS_BASE_URL` → gateway; CRS optional advanced |
-
-### License
-
-CRS lineage on this box is **MIT**. Cherry-pick with attribution; prefer
-reimplemented policy + small gateway over vendoring the full CRS monorepo
-(Redis/Grafana/admin SPA out of default install).
+| Job | Where |
+|-----|-------|
+| Seat policy / schedulable state | `seat-state.mjs`, `seat-policy-tick.mjs` |
+| Proactive token refresh | `refresh-tokens.mjs` |
+| Grok OAuth seats | `grok-cli-auth-proxy` |
+| Optional local OpenAI-compat endpoint | `ops-accounts-gateway.mjs` |
### Non-goals
- Shipping Redis/Prometheus/Grafana as required deps
-- Multi-tenant CRS admin SPA
-- Claiming Grok seats live inside a Claude-style CRS account table
+- A multi-tenant admin SPA
+- Injecting a long-lived bearer token into `settings.json` for any provider
+- Claiming Grok seats live inside a Claude-style account table
## Risks
@@ -143,9 +130,9 @@ reimplemented policy + small gateway over vendoring the full CRS monorepo
## Non-goals (immediate)
-- Full CRS fork for every vendor
+- A relay fork for every vendor
- Deleting host trees before parity proof
-- Hard-requiring CRS
+- Hard-requiring CLIProxyAPI for a single seat
## Related
diff --git a/claude-ops/docs/safety-hooks.md b/claude-ops/docs/safety-hooks.md
index a9840418..508766ef 100644
--- a/claude-ops/docs/safety-hooks.md
+++ b/claude-ops/docs/safety-hooks.md
@@ -119,7 +119,7 @@ bash claude-ops/tests/test-safety-hooks.sh
## bedrock-billing-guard (PreToolUse) — metered-spend protection
`hooks/bedrock-billing-guard.mjs` hard-stops any Claude session that is **measured
-to be billing AWS Bedrock** (metered per-token spend, not the Max/CRS account
+to be billing AWS Bedrock** (metered per-token spend, not the Max OAuth account
pool). It is the agent-facing complement to the rotation daemon's
`scripts/account-rotation/bedrock-watchdog.mjs`, which detects + force-swaps such
sessions off Bedrock within ~45s.
@@ -139,7 +139,7 @@ network/env measurement.
| `warn` | Never block — inject a one-line non-blocking warning into the agent's context on each tool call. |
| `off` | No-op. For users who intentionally run **Bedrock unrestricted**. |
-When OAuth headroom exists, the watchdog respawns the session onto CRS/OAuth and
+When OAuth headroom exists, the watchdog respawns the session onto OAuth and
clears the flag, so the block lifts on its own. The guard is **fail-open**: any
error, missing input, or unexpected state allows the tool (a money guard must
never wedge an innocent session).
diff --git a/claude-ops/hooks/bedrock-billing-guard.mjs b/claude-ops/hooks/bedrock-billing-guard.mjs
index 68470fd2..b5a2f738 100644
--- a/claude-ops/hooks/bedrock-billing-guard.mjs
+++ b/claude-ops/hooks/bedrock-billing-guard.mjs
@@ -108,8 +108,8 @@ try {
if (MODE === 'warn') {
const warn =
`⚠️💸 BEDROCK BILLING: this session${sid8 ? ` (${sid8})` : ''} is making metered AWS Bedrock ` +
- `model calls (real $$$, not the Max/CRS pool).${detail} The rotation watchdog should swap you ` +
- `to CRS/OAuth shortly. (bedrock-guard mode=warn — not blocking.)`;
+ `model calls (real $$$, not the Max OAuth pool).${detail} The rotation watchdog should swap you ` +
+ `back to OAuth shortly. (bedrock-guard mode=warn — not blocking.)`;
try {
process.stdout.write(
JSON.stringify({
@@ -124,9 +124,9 @@ if (MODE === 'warn') {
const msg =
`🛑🔥 METERED AWS BEDROCK BILLING DETECTED on THIS session${sid8 ? ` (${sid8})` : ''}.\n` +
`Every model call you make is spending REAL $$$ on AWS Bedrock per-token billing — ` +
- `NOT the free Max/CRS account pool. This is the opposite of intended routing.${detail}\n` +
+ `NOT the Max OAuth account pool. This is the opposite of intended routing.${detail}\n` +
`\nALL tool calls are BLOCKED until you acknowledge. The rotation watchdog will normally ` +
- `swap you to CRS/OAuth within ~45s and this clears automatically. If it does NOT clear, ` +
+ `swap you back to OAuth within ~45s and this clears automatically. If it does NOT clear, ` +
`there is no OAuth headroom and Bedrock is the only option.\n` +
`\nTo acknowledge and continue ANYWAY (knowingly on metered Bedrock), run exactly:\n` +
` echo BEDROCK-ACK\n` +
diff --git a/claude-ops/hooks/bedrock-fallback-guard.mjs b/claude-ops/hooks/bedrock-fallback-guard.mjs
index 5daabbf1..2457ed06 100644
--- a/claude-ops/hooks/bedrock-fallback-guard.mjs
+++ b/claude-ops/hooks/bedrock-fallback-guard.mjs
@@ -18,19 +18,19 @@ try {
if (settings.mixedProvider) {
block(
- 'Claude routing is unsafe: settings.json has both Bedrock and CRS/OAuth enabled. Run `claude-stack route --mode crs-oauth` or explicitly confirm Bedrock with `claude-stack route --mode bedrock-confirmed --confirm-metered-bedrock --reason "<reason>"`.',
+ 'Claude routing is unsafe: settings.json has both Bedrock and an OAuth/base-URL override enabled. Run `claude-stack route --mode oauth` or explicitly confirm Bedrock with `claude-stack route --mode bedrock-confirmed --confirm-metered-bedrock --reason "<reason>"`.',
);
}
if (settings.bedrock && (route.mode !== 'bedrock-confirmed' || !status.bedrockConfirmationActive)) {
block(
- 'Bedrock fallback blocked. Bedrock is metered AWS usage, and OAuth/CRS is unavailable or not selected. Confirm only for this session with `claude-stack route --mode bedrock-confirmed --reason "<why OAuth is unavailable>" --ttl-minutes 60 --confirm-metered-bedrock`, or restore OAuth with `claude-stack route --mode crs-oauth`.',
+ 'Bedrock fallback blocked. Bedrock is metered AWS usage, and OAuth is unavailable or not selected. Confirm only for this session with `claude-stack route --mode bedrock-confirmed --reason "<why OAuth is unavailable>" --ttl-minutes 60 --confirm-metered-bedrock`, or restore OAuth with `claude-stack route --mode oauth`.',
);
}
if (route.mode === 'fail-closed') {
block(
- `Claude routing is fail-closed: ${route.reason || 'CRS OAuth is unavailable and Bedrock has not been explicitly confirmed'}. Run \`claude-stack doctor --json\` for current CRS/account status.`,
+ `Claude routing is fail-closed: ${route.reason || 'OAuth is unavailable and Bedrock has not been explicitly confirmed'}. Run \`claude-stack doctor --json\` for current routing and account status.`,
);
}
diff --git a/claude-ops/plugin-dependencies.json b/claude-ops/plugin-dependencies.json
index 9b580f5f..80abe560 100644
--- a/claude-ops/plugin-dependencies.json
+++ b/claude-ops/plugin-dependencies.json
@@ -17,7 +17,7 @@
"userConfigGate": "desktop_act_co_install",
"prefsKey": "plugins.desktop_act",
"detect": { "find": "*/desktop-act/*plugin.json" },
- "essentialFor": ["/ops:desktop", "unattended captcha cascade (rotate-magic / magic-link-autoloop)"],
+ "essentialFor": ["/ops:desktop", "unattended captcha cascade (rotate-magic)"],
"reason": "MCP companion for /ops:desktop and unattended captcha cascade"
},
{
diff --git a/claude-ops/scripts/account-rotation/CAPTCHA-CASCADE.md b/claude-ops/scripts/account-rotation/CAPTCHA-CASCADE.md
index f8d2a025..428e3320 100644
--- a/claude-ops/scripts/account-rotation/CAPTCHA-CASCADE.md
+++ b/claude-ops/scripts/account-rotation/CAPTCHA-CASCADE.md
@@ -5,13 +5,13 @@ Contract for **unattended** re-auth owned by `magic-link-autoloop` →
Agents must not spawn parallel `rotate.mjs` drivers against the same fleet
(they thrash the global `.rotating` lock and the reauth browser profile).
-**CRS is not required.** Captcha cascade runs in standalone rotate-magic.
+Captcha cascade runs in standalone rotate-magic. No relay or proxy backend is involved.
## Ownership
| Component | Role |
|-----------|------|
-| `magic-link-autoloop` | Serial, one account per tick; opt-in (often CRS-flagged, but not required) |
+| `magic-link-autoloop` | Serial, one account per tick; opt-in |
| `rotate.mjs` / `rotate-magic.mjs` | Browser OAuth / magic-link / setup |
| `captcha-helper.mjs` | Token solvers + residential wait (`residualAfterWait`) |
| `visual-captcha-solver.mjs` | Vision tiles + desktop-act + VNC layers |
@@ -33,7 +33,7 @@ If captcha modules fail to load, the env keys below are harmless no-ops.
## Order (when captcha helpers are present)
-1. Residential / browser wait (`CRS_CAPTCHA_BROWSER_WAIT_MS`)
+1. Residential / browser wait (`CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS`)
2. **Large interactive** walls (visible hCaptcha challenge / CF frame): prefer
autonomous cascade first (paid token inject rarely clears pick/drag).
Override with `CLAUDE_ROT_LARGE_HC_TOKEN_FIRST=1` only when debugging.
@@ -56,10 +56,10 @@ If captcha modules fail to load, the env keys below are harmless no-ops.
| `CLAUDE_ROT_DESKTOP_ACT` | `1` | desktop-act `act` layer |
| `CLAUDE_ROT_VNC_AGENT` | `1` | Last-resort VNC computer-use |
| `CLAUDE_ROT_CAPTCHA_MAX_ATTEMPTS` | `4` | Per-page solver budget |
-| `CRS_CAPTCHA_BROWSER_WAIT_MS` | `8000` | Residential wait before paid solvers |
+| `CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS` | `8000` | Residential wait before paid solvers |
| `CLAUDE_ROT_SKIP_TOKEN_SOLVERS` | `0` | Debug only; autoloop clears sticky `1` |
-| `CRS_MAGIC_LINK_DISPATCH` | `setup` | `setup` or `magic-link` |
-| `CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS` | `1200000` | Child wall-clock (20m) |
+| `CLAUDE_REAUTH_DISPATCH` | `setup` | `setup` or `magic-link` |
+| `CLAUDE_REAUTH_TIMEOUT_MS` | `1200000` | Child wall-clock (20m) |
| `DESKTOP_ACT_CLI` | auto | Path to desktop-act CLI if not on `PATH` |
| `DESKTOP_ACT_HOME` / `DESKTOP_ACT_VENV` | — | Optional install roots |
| `CLAUDE_PLUGIN_ROOT` | installer | Plugin root (never hardcode) |
@@ -69,11 +69,11 @@ If captcha modules fail to load, the env keys below are harmless no-ops.
```bash
export CLAUDE_PLUGIN_ROOT=/path/to/claude-ops # or rely on installer
-export CRS_ENABLE_MAGIC_LINK=1
+export CLAUDE_ROTATION_ENABLE_MAGIC_LINK=1
export CLAUDE_DESKTOP_DISPLAY=:1
export CLAUDE_ROT_HEADED=1
# optional: forks with magic-link + captcha cascade in rotate.mjs
-# export CRS_MAGIC_LINK_DISPATCH=magic-link
+# export CLAUDE_REAUTH_DISPATCH=magic-link
bash "$CLAUDE_PLUGIN_ROOT/scripts/account-rotation/magic-link-autoloop.sh"
# or one-shot:
diff --git a/claude-ops/scripts/account-rotation/README.md b/claude-ops/scripts/account-rotation/README.md
index 6913f759..05848ffb 100644
--- a/claude-ops/scripts/account-rotation/README.md
+++ b/claude-ops/scripts/account-rotation/README.md
@@ -54,7 +54,6 @@ Each account entry:
| `dashlaneTokenPath` | no | If you store the token in Dashlane: `dl://<vault-name>/password`. |
| `extraUsageEnabled` | no | Set to `true` ONLY if the account has paid overage on. Triggers safety margin. |
| `capacityMultiplier` | no | Override per-account threshold (default 1.0 = standard Max 20x quota). |
-| `crsAccountName` | no | CRS admin account name this vault entry maps to (for crs-token-feed / crs-priority). |
## Signed staged Claude enrollment
@@ -200,27 +199,28 @@ the coordination key. Trust files must remain outside all mutable auth roots.
autoloop no longer dispatch authentication. They return a staged-enrollment
handoff instead; no environment flag restores the old write path.
-## CRS pool (optional)
+## Pooling several accounts
-CRS = multi-account **load balancing / rate-limit spreading** via a relay pool.
-Useful mainly when many accounts share one API endpoint and you hit 429s.
-Skip CRS for single/few-account keychain rotation.
+Keychain rotation swaps one account at a time. To spread load across several
+accounts at once, run [CLIProxyAPI](https://github.com/router-for-me/CLIProxyAPI)
+locally. It holds one OAuth seat file per account and answers on a local port.
-If you run [claude-relay-service](https://github.com/anthropics/claude-relay-service) (CRS) alongside the rotator:
+`rotate.mjs` already writes CLIProxyAPI seat files as part of a rotation, so the
+proxy and the rotator stay in sync without a separate feed. Point clients at the
+proxy with `CLIPROXYAPI_BASE_URL`, inspect the pool with `/ops:ops-fleet`, and
+set up seats with `/ops:ops-rotate-setup`.
-1. Add `crsAccountName` on each account (must match the account `name` in the CRS admin UI), or supply `crs.nameByVaultKey` in `config.json`.
-2. Set `crs.enabled`, `crs.baseUrl`, and install the priority daemon: `scripts/install-crs-priority-agent.sh`.
-3. On Linux/EC2, `crs-token-feed.mjs` propagates vault tokens into CRS (systemd timer when installed).
-4. On macOS clients that reach a **remote** CRS via SSH, install the tunnel (requires `CRS_TUNNEL_SSH_HOST`):
+CLIProxyAPI is the only supported multi-account path. The earlier relay backend
+(claude-relay-service) has been removed: it required a static bearer token
+injected into `settings.json` for every session, which is exactly the credential
+handling this plugin should not do. If a machine still has a
+`~/.claude/crs-session-settings.json` overlay or a `cr_` token in
+`settings.json`, run `claude-stack route --mode oauth` once to clear it;
+`claude-stack doctor` reports it as `legacy_relay_env` until you do.
- ```bash
- CRS_TUNNEL_SSH_HOST=your-remote-host bash scripts/install-crs-fra-tunnel.sh
- ```
-
- Point Claude Code at `http://127.0.0.1:3005/api` (or your `CRS_TUNNEL_LOCAL_PORT`).
-
-See `config.example.json` → `crs` block for all tunables (`policy`, `fileVaultPath`, `containerName`, thresholds).
-Configure via `/ops:rotate-setup` (detects CRS; never required).
+See `config.example.json` → `rotation` block for the rotator's own tunables. A
+config that still has a `crs` block keeps working: it is read as the legacy name
+for the same block.
## Keychain layout
@@ -236,7 +236,7 @@ Override the keychain account name via `CLAUDE_ROTATOR_KEYCHAIN_ACCOUNT` if you
| File | Purpose |
| --------------------------- | ------------------------------------------------------------------------- |
| `rotate.mjs` | Main rotation logic. CLI: `--status`, `--utilization`, `--to`, `--setup`. |
-| `rotate-magic.mjs` | Thin entry → `rotate.mjs --magic-link` (standalone reauth, no CRS). |
+| `rotate-magic.mjs` | Thin entry → `rotate.mjs --magic-link` (standalone reauth). |
| `captcha-cascade.mjs` | Post-verify captcha orchestration (token + visual + desktop-act + VNC). |
| `captcha-helper.mjs` | Pluggable token captcha solvers + residential wait. |
| `visual-captcha-solver.mjs` | Vision tile clicks, desktop-act, VNC layers. |
diff --git a/claude-ops/scripts/account-rotation/__tests__/claude-settings-mode-crs.test.mjs b/claude-ops/scripts/account-rotation/__tests__/claude-settings-mode-crs.test.mjs
deleted file mode 100644
index ca7f5baa..00000000
--- a/claude-ops/scripts/account-rotation/__tests__/claude-settings-mode-crs.test.mjs
+++ /dev/null
@@ -1,119 +0,0 @@
-/**
- * claude-settings-mode CRS-awareness tests (node --test).
- *
- * Run: node --test claude-ops/scripts/account-rotation/__tests__/claude-settings-mode-crs.test.mjs
- *
- * Guards the money-leak fix: a CRS-routed box (cr_ token + ANTHROPIC_BASE_URL)
- * must never be flipped onto metered Bedrock, and an OAuth-mode flip must keep
- * the CRS relay pair intact (a cr_ token with no base URL 401s the relay).
- *
- * These functions read/write ~/.claude/settings.json via $HOME, so each test
- * points HOME at a fresh temp dir. persistBedrockClaudeSettings on a CRS box
- * throws BEFORE any `aws` exec, so no AWS mocking is needed for that path.
- */
-
-import { test } from 'node:test';
-import assert from 'node:assert/strict';
-import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs';
-import { tmpdir } from 'node:os';
-import { join } from 'node:path';
-
-import { persistBedrockClaudeSettings, clearHardcodedModelsForOAuthClaudeSettings } from '../claude-settings-mode.mjs';
-
-const CRS_BASE = 'http://127.0.0.1:3005/api';
-const CRS_TOKEN = 'cr_deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef';
-
-function withTempHome(initialEnv) {
- const home = mkdtempSync(join(tmpdir(), 'crs-settings-test-'));
- mkdirSync(join(home, '.claude'), { recursive: true });
- writeFileSync(join(home, '.claude', 'settings.json'), JSON.stringify({ env: initialEnv }, null, 2));
- const prevHome = process.env.HOME;
- process.env.HOME = home;
- return {
- readEnv() {
- return JSON.parse(readFileSync(join(home, '.claude', 'settings.json'), 'utf8')).env || {};
- },
- cleanup() {
- process.env.HOME = prevHome;
- rmSync(home, { recursive: true, force: true });
- },
- };
-}
-
-test('clearHardcodedModelsForOAuth: preserves CRS relay pair across Bedrock→OAuth flip', () => {
- const ctx = withTempHome({
- CLAUDE_CODE_USE_BEDROCK: '1',
- AWS_REGION: 'us-east-1',
- ANTHROPIC_MODEL: 'us.anthropic.claude-sonnet-4-6',
- ANTHROPIC_BASE_URL: CRS_BASE,
- CLAUDE_CODE_OAUTH_TOKEN: CRS_TOKEN,
- });
- try {
- clearHardcodedModelsForOAuthClaudeSettings();
- const env = ctx.readEnv();
- assert.equal(env.ANTHROPIC_BASE_URL, CRS_BASE, 'CRS base URL restored');
- assert.equal(env.CLAUDE_CODE_OAUTH_TOKEN, CRS_TOKEN, 'CRS token retained');
- assert.equal('CLAUDE_CODE_USE_BEDROCK' in env, false, 'Bedrock flag cleared');
- assert.equal('ANTHROPIC_MODEL' in env, false, 'hardcoded model cleared');
- assert.equal('AWS_REGION' in env, false, 'AWS region cleared');
- } finally {
- ctx.cleanup();
- }
-});
-
-test('clearHardcodedModelsForOAuth: non-CRS box still strips ANTHROPIC_BASE_URL (regression)', () => {
- const ctx = withTempHome({
- CLAUDE_CODE_USE_BEDROCK: '1',
- ANTHROPIC_BASE_URL: 'https://some-proxy.example/api',
- CLAUDE_CODE_OAUTH_TOKEN: 'sk-ant-oat-not-a-relay-token',
- ANTHROPIC_MODEL: 'us.anthropic.claude-sonnet-4-6',
- });
- try {
- clearHardcodedModelsForOAuthClaudeSettings();
- const env = ctx.readEnv();
- assert.equal('ANTHROPIC_BASE_URL' in env, false, 'non-cr_ base URL stripped (raw OAuth)');
- assert.equal('CLAUDE_CODE_USE_BEDROCK' in env, false);
- } finally {
- ctx.cleanup();
- }
-});
-
-test('persistBedrockClaudeSettings: refuses to flip a CRS-routed box onto Bedrock', () => {
- const ctx = withTempHome({
- ANTHROPIC_BASE_URL: CRS_BASE,
- CLAUDE_CODE_OAUTH_TOKEN: CRS_TOKEN,
- });
- try {
- assert.throws(() => persistBedrockClaudeSettings('us-east-1'), /CRS-routed/, 'throws on CRS box');
- const env = ctx.readEnv();
- // settings must be untouched — no Bedrock vars written, CRS pair intact.
- assert.equal('CLAUDE_CODE_USE_BEDROCK' in env, false, 'no Bedrock flag written');
- assert.equal(env.ANTHROPIC_BASE_URL, CRS_BASE, 'CRS base URL untouched');
- assert.equal(env.CLAUDE_CODE_OAUTH_TOKEN, CRS_TOKEN, 'CRS token untouched');
- } finally {
- ctx.cleanup();
- }
-});
-
-test('persistBedrockClaudeSettings: CLAUDE_ALLOW_BEDROCK_OVER_CRS=1 overrides the refusal', () => {
- const ctx = withTempHome({
- ANTHROPIC_BASE_URL: CRS_BASE,
- CLAUDE_CODE_OAUTH_TOKEN: CRS_TOKEN,
- });
- const prev = process.env.CLAUDE_ALLOW_BEDROCK_OVER_CRS;
- const prevSkip = process.env.BEDROCK_SKIP_RESOLVE;
- process.env.CLAUDE_ALLOW_BEDROCK_OVER_CRS = '1';
- process.env.BEDROCK_SKIP_RESOLVE = '1'; // avoid live `aws bedrock` calls in CI
- try {
- persistBedrockClaudeSettings('us-east-1');
- const env = ctx.readEnv();
- assert.equal(env.CLAUDE_CODE_USE_BEDROCK, '1', 'override allows Bedrock flip');
- assert.equal('ANTHROPIC_BASE_URL' in env, false, 'base URL dropped when overridden onto Bedrock');
- } finally {
- if (prev === undefined) delete process.env.CLAUDE_ALLOW_BEDROCK_OVER_CRS;
- else process.env.CLAUDE_ALLOW_BEDROCK_OVER_CRS = prev;
- if (prevSkip === undefined) delete process.env.BEDROCK_SKIP_RESOLVE;
- else process.env.BEDROCK_SKIP_RESOLVE = prevSkip;
- ctx.cleanup();
- }
-});
diff --git a/claude-ops/scripts/account-rotation/__tests__/coordination-provisioning.test.mjs b/claude-ops/scripts/account-rotation/__tests__/coordination-provisioning.test.mjs
index c755f817..e946ad95 100755
--- a/claude-ops/scripts/account-rotation/__tests__/coordination-provisioning.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/coordination-provisioning.test.mjs
@@ -66,7 +66,6 @@ function fixture(legacy = false) {
'macos-plist': join(root, 'generated', 'rotator.plist'),
'runtime-inventory': join(trustRoot, 'account-inventory.json'),
'authoritative-consumer-inventory': join(trustRoot, 'consumer-inventory.json'),
- 'linux-token-feed-unit': join(root, 'generated', 'crs-token-feed.service'),
'linux-refresh-unit': join(root, 'generated', 'claude-token-refresh.service'),
};
const manifestSource = join(trustRoot, 'reviewed-manifest.json');
diff --git a/claude-ops/scripts/account-rotation/__tests__/model-args.test.mjs b/claude-ops/scripts/account-rotation/__tests__/model-args.test.mjs
index e6b86d7a..ca8330b0 100644
--- a/claude-ops/scripts/account-rotation/__tests__/model-args.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/model-args.test.mjs
@@ -89,7 +89,7 @@ function runRespawnCase(mode) {
rmSync(respawned, { force: true });
writeFileSync(
fakeRespawn,
- `#!/bin/sh\nRESPAWN_ARGS="$*" node - <<'NODE'\nconst fs=require('fs');\nconst cp=require('child_process');\nconst path=require('path');\nconst state=JSON.parse(fs.readFileSync(process.env.STATE_PATH,'utf8'));\nfs.writeFileSync(process.env.CAPTURE_PATH, JSON.stringify({ args: process.env.RESPAWN_ARGS.split(' '), state }));\nconst mode=process.env.RESPAWN_TEST_MODE;\nif (mode !== 'spawn-failure' && mode !== 'no-session') {\n let pid=999999;\n if (mode !== 'stale-pid') {\n const child=cp.spawn(process.execPath,['-e','setTimeout(()=>{},10000)'],{detached:true,stdio:'ignore',env:process.env});\n child.unref();\n pid=child.pid;\n fs.writeFileSync(process.env.CHILD_PID_PATH,String(pid));\n }\n const expectedRoute=JSON.parse(process.env.CLAUDE_OPS_EXPECTED_ROUTE);\n const effectiveRoute=mode === 'wrong-route'\n ? {...expectedRoute,mode:'crs'}\n : mode === 'wrong-endpoint'\n ? {...expectedRoute,baseUrl:'https://wrong-relay.example/api'}\n : mode === 'mismatched-credential'\n ? {...expectedRoute,credentialFingerprint:'0'.repeat(64)}\n : expectedRoute;\n fs.mkdirSync(process.env.SESSIONS_DIR,{recursive:true});\n fs.writeFileSync(path.join(process.env.SESSIONS_DIR,pid+'.json'),JSON.stringify({kind:'bg',jobId:process.env.SESSION_ID,pid,status:'waiting',effectiveRoute}));\n}\nNODE\nexit ${exitCode}\n`,
+ `#!/bin/sh\nRESPAWN_ARGS="$*" node - <<'NODE'\nconst fs=require('fs');\nconst cp=require('child_process');\nconst path=require('path');\nconst state=JSON.parse(fs.readFileSync(process.env.STATE_PATH,'utf8'));\nfs.writeFileSync(process.env.CAPTURE_PATH, JSON.stringify({ args: process.env.RESPAWN_ARGS.split(' '), state }));\nconst mode=process.env.RESPAWN_TEST_MODE;\nif (mode !== 'spawn-failure' && mode !== 'no-session') {\n let pid=999999;\n if (mode !== 'stale-pid') {\n const child=cp.spawn(process.execPath,['-e','setTimeout(()=>{},10000)'],{detached:true,stdio:'ignore',env:process.env});\n child.unref();\n pid=child.pid;\n fs.writeFileSync(process.env.CHILD_PID_PATH,String(pid));\n }\n const expectedRoute=JSON.parse(process.env.CLAUDE_OPS_EXPECTED_ROUTE);\n const effectiveRoute=mode === 'wrong-route'\n ? {...expectedRoute,mode:'relay'}\n : mode === 'wrong-endpoint'\n ? {...expectedRoute,baseUrl:'https://wrong-relay.example/api'}\n : mode === 'mismatched-credential'\n ? {...expectedRoute,credentialFingerprint:'0'.repeat(64)}\n : expectedRoute;\n fs.mkdirSync(process.env.SESSIONS_DIR,{recursive:true});\n fs.writeFileSync(path.join(process.env.SESSIONS_DIR,pid+'.json'),JSON.stringify({kind:'bg',jobId:process.env.SESSION_ID,pid,status:'waiting',effectiveRoute}));\n}\nNODE\nexit ${exitCode}\n`,
{ mode: 0o700 },
);
const result = spawnSync(
@@ -169,21 +169,21 @@ const directRoute = {
credentialFingerprint: routeCredentialFingerprint(directToken),
settings: null,
};
-const crsRoute = {
- mode: 'crs',
+const relayRoute = {
+ mode: 'relay',
baseUrl: 'http://127.0.0.1:3005/api',
- credentialSource: 'crs-relay',
+ credentialSource: 'relay',
credentialFingerprint: routeCredentialFingerprint('cr_test-relay'),
- settings: '/tmp/crs-session-settings.json',
+ settings: '/tmp/relay-session-settings.json',
};
-assert.equal(effectiveRouteMatches({ ...crsRoute, baseUrl: 'http://127.0.0.1:3002/api' }, crsRoute), false);
+assert.equal(effectiveRouteMatches({ ...relayRoute, baseUrl: 'http://127.0.0.1:3002/api' }, relayRoute), false);
assert.equal(
- effectiveRouteMatches({ ...crsRoute, credentialFingerprint: routeCredentialFingerprint(directToken) }, crsRoute),
+ effectiveRouteMatches({ ...relayRoute, credentialFingerprint: routeCredentialFingerprint(directToken) }, relayRoute),
false,
);
assert.equal(effectiveRouteMatches({ ...directRoute, credentialSource: 'keychain' }, directRoute), false);
assert.equal(effectiveRouteMatches({ ...directRoute, credentialFingerprint: null }, directRoute), false);
-assert.equal(effectiveRouteMatches({ ...crsRoute }, crsRoute), true);
+assert.equal(effectiveRouteMatches({ ...relayRoute }, relayRoute), true);
assert.equal(effectiveRouteMatches({ ...directRoute }, directRoute), true);
const success = runRespawnCase('success');
diff --git a/claude-ops/scripts/account-rotation/__tests__/ops-accounts-backend.test.mjs b/claude-ops/scripts/account-rotation/__tests__/ops-accounts-backend.test.mjs
index 27fe8501..9fa904e1 100644
--- a/claude-ops/scripts/account-rotation/__tests__/ops-accounts-backend.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/ops-accounts-backend.test.mjs
@@ -1,5 +1,5 @@
/**
- * Unit tests: OPS_ACCOUNTS_BACKEND resolution + seat-state merge (no CRS).
+ * Unit tests: OPS_ACCOUNTS_BACKEND resolution + seat-state merge.
*/
import { resolveAccountsBackend, dualWriteEnabled, mergeSeatsIntoState } from '../ops-accounts-backend.mjs';
@@ -10,7 +10,11 @@ function assert(cond, msg) {
assert(resolveAccountsBackend({ env: {} }) === 'auto', 'default auto');
assert(resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'local' } }) === 'local', 'local');
assert(resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'seat-state' } }) === 'local', 'seat-state');
-assert(resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'CRS' } }) === 'crs', 'crs case');
+assert(resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'CLIPROXY' } }) === 'cliproxy', 'cliproxy case');
+assert(
+ resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'crs' } }) === 'local',
+ 'retired relay value falls back to local',
+);
assert(resolveAccountsBackend({ env: {}, cfgBackend: 'file' }) === 'local', 'cfg file');
assert(resolveAccountsBackend({ env: { OPS_ACCOUNTS_BACKEND: 'auto' } }) === 'auto', 'explicit auto');
diff --git a/claude-ops/scripts/account-rotation/__tests__/production-recovery.test.mjs b/claude-ops/scripts/account-rotation/__tests__/production-recovery.test.mjs
index 208b5b30..a0211b98 100644
--- a/claude-ops/scripts/account-rotation/__tests__/production-recovery.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/production-recovery.test.mjs
@@ -1,259 +1,21 @@
#!/usr/bin/env node
+/**
+ * Retry-After parsing used by every 429/503 recovery path.
+ *
+ * This file previously also drove three relay-pool daemons (cooldown, priority,
+ * 401 refresher). Those daemons are gone: multi-account rotation and OAuth seat
+ * management run through CLIProxyAPI, which owns its own backoff.
+ */
import assert from 'node:assert/strict';
-import { spawn } from 'node:child_process';
-import { mkdtempSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs';
-import { createServer } from 'node:http';
-import { tmpdir } from 'node:os';
-import { join } from 'node:path';
import { retryAfterDelayMs } from '../retry-after.mjs';
-const cooldownScript = new URL('../crs-429-cooldown.mjs', import.meta.url);
-const priorityScript = new URL('../crs-priority-daemon.mjs', import.meta.url);
-const retired401Script = new URL('../crs-401-refresher.mjs', import.meta.url);
-
-function run(script, env, args = []) {
- return new Promise((resolve, reject) => {
- const child = spawn(process.execPath, [script.pathname, ...args], { env: { ...process.env, ...env } });
- let stdout = '';
- let stderr = '';
- child.stdout.on('data', (chunk) => (stdout += chunk));
- child.stderr.on('data', (chunk) => (stderr += chunk));
- child.once('error', reject);
- child.once('close', (status) => resolve({ status, stdout, stderr }));
- });
-}
-
-function startFixture(initialAccounts, toggleStatuses = []) {
- let accounts = structuredClone(initialAccounts);
- const toggles = [];
- const queuedToggleStatuses = [...toggleStatuses];
- const server = createServer(async (req, res) => {
- let body = '';
- for await (const chunk of req) body += chunk;
- const json = (status, value) => {
- const text = JSON.stringify(value);
- res.writeHead(status, { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(text) });
- res.end(text);
- };
- if (req.url === '/web/auth/login' && req.method === 'POST') return json(200, { token: 'fixture-admin' });
- if (req.url === '/admin/claude-accounts' && req.method === 'GET') return json(200, accounts);
- const toggle = req.url?.match(/^\/admin\/claude-accounts\/([^/]+)\/toggle-schedulable$/);
- if (toggle && req.method === 'PUT') {
- const update = JSON.parse(body || '{}');
- const status = queuedToggleStatuses.shift() ?? 200;
- toggles.push({ id: toggle[1], update, status });
- if (status >= 200 && status < 300) {
- accounts = accounts.map((account) =>
- account.id === toggle[1]
- ? {
- ...account,
- schedulable: update.schedulable,
- rateLimitEndAt: update.rateLimitEndAt ?? account.rateLimitEndAt,
- rateLimitReason: update.rateLimitReason ?? account.rateLimitReason,
- }
- : account,
- );
- }
- return json(status, { success: status >= 200 && status < 300 });
- }
- if (req.url?.endsWith('/reset-status') && req.method === 'POST') return json(200, { success: true });
- if (req.url?.startsWith('/admin/claude-accounts/') && req.method === 'PUT') return json(200, { success: true });
- return json(404, { error: 'not found' });
- });
- return new Promise((resolve, reject) => {
- server.once('error', reject);
- server.listen(0, '127.0.0.1', () => {
- resolve({
- base: `http://127.0.0.1:${server.address().port}`,
- close: () => new Promise((done) => server.close(done)),
- toggles,
- });
- });
- });
-}
-
+// Delta-seconds form, plus the 2s safety margin.
assert.equal(retryAfterDelayMs('7', 0), 9_000);
+// HTTP-date form.
assert.equal(retryAfterDelayMs(new Date(12_000).toUTCString(), 0), 14_000);
+// Unparseable value falls back to a fixed minute.
assert.equal(retryAfterDelayMs('invalid-date', 0), 60_000);
+// No header at all means "caller decides".
assert.equal(retryAfterDelayMs(null, 0), null);
-function fixtureEnv(root, base, config) {
- const configPath = join(root, 'config.json');
- writeFileSync(configPath, JSON.stringify(config), { mode: 0o600 });
- return {
- HOME: root,
- CLAUDE_ROTATOR_CONFIG: configPath,
- CLAUDE_PLUGIN_DATA_DIR: join(root, 'data'),
- CRS_BASE: base,
- CRS_ADMIN_PASSWORD: 'fixture-password',
- OPS_ACCOUNTS_BACKEND: 'crs',
- OPS_ACCOUNTS_DUAL_WRITE: '0',
- };
-}
-
-{
- const root = mkdtempSync(join(tmpdir(), 'retired-401-'));
- const configPath = join(root, 'config.json');
- writeFileSync(configPath, JSON.stringify({ accounts: [], crs: {} }), { mode: 0o600 });
- const result = await run(retired401Script, { HOME: root, CLAUDE_ROTATOR_CONFIG: configPath });
- assert.notEqual(result.status, 0);
- assert.match(`${result.stdout}${result.stderr}`, /RETIRED_USE_VERIFIED_TOKEN_FEED/);
-}
-
-{
- const root = mkdtempSync(join(tmpdir(), 'cooldown-recovery-'));
- mkdirSync(join(root, 'state'), { recursive: true });
- const resetAt = new Date(Date.now() + 60_000).toISOString();
- const fixture = await startFixture([
- {
- id: 'limited',
- platform: 'claude',
- isActive: true,
- schedulable: true,
- rateLimitStatus: { isRateLimited: true, minutesRemaining: 1, rateLimitEndAt: resetAt },
- },
- ]);
- const statePath = join(root, 'state', 'crs-429-state.json');
- const env = {
- ...fixtureEnv(root, fixture.base, { accounts: [], crs: { cooldownEnabled: true } }),
- CRS_COOLDOWN_ENABLED: '1',
- CRS_COOLDOWN_STATE_PATH: statePath,
- };
- try {
- const held = await run(cooldownScript, env);
- assert.equal(held.status, 0, held.stderr);
- assert.equal(fixture.toggles.at(-1)?.update.schedulable, false);
- const state = JSON.parse(readFileSync(statePath, 'utf8'));
- assert.equal(state.limited.cooldownUntil > Date.now(), true);
-
- state.limited.cooldownUntil = Date.now() - 1;
- writeFileSync(statePath, JSON.stringify(state), { mode: 0o600 });
- const released = await startFixture([
- {
- id: 'limited',
- platform: 'claude',
- isActive: true,
- schedulable: false,
- rateLimitStatus: { isRateLimited: false },
- },
- ]);
- try {
- const releaseEnv = { ...env, CRS_BASE: released.base };
- const result = await run(cooldownScript, releaseEnv);
- assert.equal(result.status, 0, result.stderr);
- assert.equal(released.toggles.at(-1)?.update.schedulable, true);
- assert.equal(JSON.parse(readFileSync(statePath, 'utf8')).limited, undefined);
- } finally {
- await released.close();
- }
- } finally {
- await fixture.close();
- }
-}
-
-{
- const root = mkdtempSync(join(tmpdir(), 'cooldown-toggle-failure-'));
- mkdirSync(join(root, 'state'), { recursive: true });
- const statePath = join(root, 'state', 'crs-429-state.json');
- const resetAt = new Date(Date.now() + 60_000).toISOString();
- const holdFixture = await startFixture(
- [
- {
- id: 'limited',
- platform: 'claude',
- isActive: true,
- schedulable: true,
- rateLimitStatus: { isRateLimited: true, minutesRemaining: 1, rateLimitEndAt: resetAt },
- },
- ],
- [503],
- );
- const env = {
- ...fixtureEnv(root, holdFixture.base, { accounts: [], crs: { cooldownEnabled: true } }),
- CRS_COOLDOWN_ENABLED: '1',
- CRS_COOLDOWN_STATE_PATH: statePath,
- };
- try {
- const result = await run(cooldownScript, env);
- assert.equal(result.status, 0, result.stderr);
- const held = JSON.parse(readFileSync(statePath, 'utf8')).limited;
- assert.equal(held.retryAction, 'hold');
- assert.equal(held.lastToggleStatus, 503);
- } finally {
- await holdFixture.close();
- }
-
- const expired = JSON.parse(readFileSync(statePath, 'utf8'));
- expired.limited.cooldownUntil = Date.now() - 1;
- writeFileSync(statePath, JSON.stringify(expired), { mode: 0o600 });
- const releaseFixture = await startFixture(
- [
- {
- id: 'limited',
- platform: 'claude',
- isActive: true,
- schedulable: false,
- rateLimitStatus: { isRateLimited: false },
- },
- ],
- [502],
- );
- try {
- const result = await run(cooldownScript, { ...env, CRS_BASE: releaseFixture.base });
- assert.equal(result.status, 0, result.stderr);
- const retained = JSON.parse(readFileSync(statePath, 'utf8')).limited;
- assert.equal(retained.retryAction, 'release');
- assert.equal(retained.lastToggleStatus, 502);
- } finally {
- await releaseFixture.close();
- }
-}
-
-{
- const root = mkdtempSync(join(tmpdir(), 'priority-auth-recovery-'));
- const fixture = await startFixture([
- {
- id: 'invalid-1',
- platform: 'claude',
- isActive: true,
- schedulable: true,
- status: 'auth_repair',
- tokenExpiresAt: Date.now() - 1,
- lastError: 'HTTP 401 authentication_error invalid token',
- },
- {
- id: 'invalid-2',
- platform: 'claude',
- isActive: true,
- schedulable: true,
- tokenExpiresAt: Date.now() - 1,
- errorMessage: '403 revoked credential',
- },
- ]);
- const env = {
- ...fixtureEnv(root, fixture.base, { accounts: [], crs: { enabled: true, policy: 'conservative', floor: 2 } }),
- CRS_FLOOR: '2',
- CRS_LIVE_USAGE_TTL_MS: '1',
- };
- try {
- const result = await run(priorityScript, env);
- assert.equal(result.status, 0, result.stderr);
- assert.equal(fixture.toggles.length, 2);
- assert.equal(
- fixture.toggles.every((entry) => entry.update.schedulable === false),
- true,
- );
- assert.match(result.stdout, /all active Claude accounts unavailable; leaving pool honestly unschedulable/);
- assert.match(result.stdout, /schedulable=0/);
- } finally {
- await fixture.close();
- }
-}
-
-console.log('Production 401/429 recovery tests: PASS');
+console.log('Retry-After recovery tests: PASS');
diff --git a/claude-ops/scripts/account-rotation/__tests__/reauth-env.test.mjs b/claude-ops/scripts/account-rotation/__tests__/reauth-env.test.mjs
index 6fc8f4b3..4dcb37f5 100644
--- a/claude-ops/scripts/account-rotation/__tests__/reauth-env.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/reauth-env.test.mjs
@@ -37,23 +37,23 @@ const env = buildReauthChildEnv({
PATH: '/usr/bin',
CLAUDE_DESKTOP_DISPLAY: ':5',
},
- crs: {},
+ rotation: {},
});
assert(env.CLAUDE_ROT_HEADED === '1', 'headed default');
assert(env.CLAUDE_DESKTOP_DISPLAY === ':5', 'seat');
assert(env.DESKTOP_ACT_DISPLAY === ':5', 'desktop-act seat');
assert(env.CLAUDE_ROT_STEAL_LOCK === '0', 'no steal');
assert(env.CLAUDE_ROT_SKIP_TOKEN_SOLVERS === '0', 'solvers on');
-assert(env.CRS_MAGIC_LINK_DISPATCH === 'setup', 'default dispatch');
+assert(env.CLAUDE_REAUTH_DISPATCH === 'setup', 'default dispatch');
const magic = buildReauthChildEnv({
- env: { CRS_MAGIC_LINK_DISPATCH: 'magic-link', HOME: '/tmp/x', PATH: '/bin' },
+ env: { CLAUDE_REAUTH_DISPATCH: 'magic-link', HOME: '/tmp/x', PATH: '/bin' },
});
-assert(magic.CRS_MAGIC_LINK_DISPATCH === 'magic-link', 'dispatch override');
+assert(magic.CLAUDE_REAUTH_DISPATCH === 'magic-link', 'dispatch override');
// timeout
assert(resolveReauthTimeoutMs({}) === 20 * 60_000, 'default 20m');
-assert(resolveReauthTimeoutMs({ CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS: '90000' }) === 90000, 'env timeout');
+assert(resolveReauthTimeoutMs({ CLAUDE_REAUTH_TIMEOUT_MS: '90000' }) === 90000, 'env timeout');
// success markers
assert(reauthOutputLooksSuccessful('{"ok": true}'), 'json ok');
diff --git a/claude-ops/scripts/account-rotation/__tests__/refresh-lock.test.mjs b/claude-ops/scripts/account-rotation/__tests__/refresh-lock.test.mjs
index f49c27cb..d7cec29c 100644
--- a/claude-ops/scripts/account-rotation/__tests__/refresh-lock.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/refresh-lock.test.mjs
@@ -5,9 +5,9 @@ import { tmpdir } from 'node:os';
import { join } from 'node:path';
const lockDir = mkdtempSync(join(tmpdir(), 'refresh-lock-owner-'));
-process.env.CRS_REFRESH_LOCK_DIR = lockDir;
-process.env.CRS_REFRESH_LOCK_TTL_SEC = '0.15';
-const { acquireRefreshLock } = await import(`../crs-refresh-lock.mjs?test=${Date.now()}`);
+process.env.CLAUDE_REFRESH_LOCK_DIR = lockDir;
+process.env.CLAUDE_REFRESH_LOCK_TTL_SEC = '0.15';
+const { acquireRefreshLock } = await import(`../refresh-lock.mjs?test=${Date.now()}`);
const release = acquireRefreshLock('[email protected]');
assert.ok(release);
diff --git a/claude-ops/scripts/account-rotation/__tests__/refresh-pacing-worker.mjs b/claude-ops/scripts/account-rotation/__tests__/refresh-pacing-worker.mjs
index 60123640..cf56e9bf 100644
--- a/claude-ops/scripts/account-rotation/__tests__/refresh-pacing-worker.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/refresh-pacing-worker.mjs
@@ -1,5 +1,5 @@
#!/usr/bin/env node
-import { claimRefreshPace } from '../crs-refresh-lock.mjs';
+import { claimRefreshPace } from '../refresh-lock.mjs';
const key = process.argv[2];
const now = Number(process.argv[3]);
diff --git a/claude-ops/scripts/account-rotation/__tests__/refresh-pacing.test.mjs b/claude-ops/scripts/account-rotation/__tests__/refresh-pacing.test.mjs
index 76ee564e..4d1e924b 100644
--- a/claude-ops/scripts/account-rotation/__tests__/refresh-pacing.test.mjs
+++ b/claude-ops/scripts/account-rotation/__tests__/refresh-pacing.test.mjs
@@ -10,10 +10,10 @@ const worker = new URL('./refresh-pacing-worker.mjs', import.meta.url);
const now = 1_000_000;
const env = {
...process.env,
- CRS_REFRESH_LOCK_DIR: lockDir,
- CRS_REFRESH_MIN_PACE_MS: '40',
- CRS_REFRESH_JITTER_MS: '0',
- CRS_REFRESH_TEST_ALLOW_ZERO_PACE: '1',
+ CLAUDE_REFRESH_LOCK_DIR: lockDir,
+ CLAUDE_REFRESH_MIN_PACE_MS: '40',
+ CLAUDE_REFRESH_JITTER_MS: '0',
+ CLAUDE_REFRESH_TEST_ALLOW_ZERO_PACE: '1',
};
function runWorker() {
@@ -46,10 +46,10 @@ const production = await new Promise((resolve, reject) => {
const child = spawn(process.execPath, [worker.pathname, '[email protected]', String(now)], {
env: {
...process.env,
- CRS_REFRESH_LOCK_DIR: productionLockDir,
- CRS_REFRESH_MIN_PACE_MS: '0',
- CRS_REFRESH_JITTER_MS: '0',
- CRS_REFRESH_TEST_ALLOW_ZERO_PACE: '0',
+ CLAUDE_REFRESH_LOCK_DIR: productionLockDir,
+ CLAUDE_REFRESH_MIN_PACE_MS: '0',
+ CLAUDE_REFRESH_JITTER_MS: '0',
+ CLAUDE_REFRESH_TEST_ALLOW_ZERO_PACE: '0',
},
});
let stderr = '';
diff --git a/claude-ops/scripts/account-rotation/bg-respawn.mjs b/claude-ops/scripts/account-rotation/bg-respawn.mjs
index 172d1ee6..bbc56e42 100644
--- a/claude-ops/scripts/account-rotation/bg-respawn.mjs
+++ b/claude-ops/scripts/account-rotation/bg-respawn.mjs
@@ -1,4 +1,3 @@
-import { loadClaudeHarnessEnv } from './claude-harness-env.mjs';
import { normalizeClaudeModelArgs } from './model-args.mjs';
// ── Background-session respawn after rotation ────────────────────────────────
// Daemon-hosted `claude --bg` sessions have no TTY, so the /login-injection
@@ -40,7 +39,7 @@ import { join, dirname } from 'path';
import { homedir } from 'os';
import { fileURLToPath } from 'url';
import { getTokenForSession, extractAccessToken, readLeases, recordSessionLease } from './session-router.mjs';
-import { applyCrsSessionSettings, readRouteState } from './route-state.mjs';
+import { isLegacyRelayBase, isLegacyRelayToken, readRouteState } from './route-state.mjs';
const __dirname = dirname(fileURLToPath(import.meta.url));
const CONFIG_PATH = join(__dirname, 'config.json');
@@ -59,13 +58,10 @@ const DEFERRED_MARKER = (id) => `/tmp/claude-respawn-deferred-${id}`;
const RESPAWN_THROTTLE_MS = 10 * 60_000; // never respawn the same session twice in 10 min
const BUSY_FORCE_AFTER_MS = 90 * 60_000; // busy this long after rotation → respawn anyway
-// Dashboard marker: present ⇔ at least one session was last respawned to DIRECT
-// because CRS was unhealthy. Lets fleet-status explain a "direct" session as
-// "relay-down" rather than a silent/unexplained desync. Cleared whenever a
-// session successfully routes via a healthy relay.
-const CRS_RELAY_DOWN_MARKER = '/tmp/claude-crs-relay-down';
-const HEALTH_WATCH_STATE = join(__dirname, 'crs-health-watch.state.json');
-const HEALTH_STATE_FRESH_MS = 90_000; // trust the health-watch verdict if <90s old (it ticks every 60s)
+// A retired relay backend once left this marker to explain a "direct" session.
+// Respawn is always direct now, so the only thing left to do with it is clear it.
+const LEGACY_RELAY_DOWN_MARKER = '/tmp/claude-crs-relay-down';
+const LEGACY_SESSION_SETTINGS_SUFFIX = 'crs-session-settings.json';
// TTY hygiene helper: on abnormal paths or exit, attempt to restore sane mode so
// background errors or claude subprocs do not leave ^C ^[ etc. in a raw-mode TUI.
@@ -76,30 +72,14 @@ function resetTty() {
}
process.once('exit', resetTty);
-// Cheap CRS health read: prefer the recent crs-health-watch verdict (no network),
-// fall back to a 3s /health curl when the state file is missing/stale. Returns
-// boolean healthy. Keeps respawn fast and avoids hammering the relay per-session.
-function crsHealthyCheap() {
+/** Drop the leftovers a retired relay backend may have left on this machine. */
+function clearLegacyRelayArtifacts(log) {
try {
- const st = JSON.parse(readFileSync(HEALTH_WATCH_STATE, 'utf8'));
- const mtime = statSync(HEALTH_WATCH_STATE).mtimeMs;
- if (Date.now() - mtime < HEALTH_STATE_FRESH_MS) {
- // mode 'crs' with down=0 ⇒ relay solid; mode 'fail-closed' ⇒ relay down.
- if (st.mode === 'fail-closed') return false;
- if (st.mode === 'crs' && (st.down || 0) === 0) return true;
+ if (existsSync(LEGACY_RELAY_DOWN_MARKER)) {
+ unlinkSync(LEGACY_RELAY_DOWN_MARKER);
+ log(`[bg-respawn] cleared stale relay marker ${LEGACY_RELAY_DOWN_MARKER}`);
}
} catch {}
- // Stale/absent state → authoritative live probe.
- try {
- const hc = execFileSync(
- 'curl',
- ['-sf', '-o', '/dev/null', '-w', '%{http_code}', '--max-time', '3', 'http://127.0.0.1:3005/health'],
- { encoding: 'utf8', timeout: 5000 },
- ).trim();
- return hc === '200';
- } catch {
- return false;
- }
}
// Graceful rotation-over stagger: respawn bg sessions ONE AT A TIME with this gap
@@ -198,17 +178,18 @@ function writeJsonAtomic(path, value) {
renameSync(tmp, path);
}
-function respawnStateCoherent(stateFile, expectedCrsActive) {
+function respawnStateCoherent(stateFile) {
try {
const persisted = JSON.parse(readFileSync(stateFile, 'utf8'));
- if (!Array.isArray(persisted.respawnFlags)) return expectedCrsActive === false;
+ if (!Array.isArray(persisted.respawnFlags)) return true;
const normalized = normalizeClaudeModelArgs(persisted.respawnFlags);
if (!normalized.every((flag, index) => flag === persisted.respawnFlags[index])) return false;
- const hasCrsSettings = persisted.respawnFlags.some(
+ // A relay session-settings overlay must not survive into a respawn.
+ return !persisted.respawnFlags.some(
(flag, index) =>
- flag === '--settings' && String(persisted.respawnFlags[index + 1] || '').endsWith('crs-session-settings.json'),
+ flag === '--settings' &&
+ String(persisted.respawnFlags[index + 1] || '').endsWith(LEGACY_SESSION_SETTINGS_SUFFIX),
);
- return hasCrsSettings === expectedCrsActive;
} catch {
return false;
}
@@ -291,17 +272,11 @@ export function effectiveRouteFromProcess(pid, expectedSettings) {
if (baseUrl && apiBase && baseUrl !== apiBase) return null;
const effectiveBase = baseUrl || apiBase || null;
const tokens = [env.ANTHROPIC_API_KEY, env.ANTHROPIC_AUTH_TOKEN, env.CLAUDE_CODE_OAUTH_TOKEN].filter(Boolean);
- if (effectiveBase) {
- if (!tokens.length || tokens.some((token) => token !== tokens[0] || !String(token).startsWith('cr_'))) return null;
- return {
- mode: 'crs',
- baseUrl: effectiveBase,
- credentialSource: 'crs-relay',
- credentialFingerprint: routeCredentialFingerprint(tokens[0]),
- settings: expectedSettings,
- };
- }
- if (tokens.some((token) => String(token).startsWith('cr_'))) return null;
+ void expectedSettings;
+ // Any base-URL override, or any relay-shaped token, means the child is not on
+ // the direct OAuth route this module now guarantees.
+ if (effectiveBase) return null;
+ if (tokens.some((token) => isLegacyRelayToken(token))) return null;
const directToken = tokens[0] || activeKeychainAccessToken();
if (!directToken || tokens.some((token) => token !== directToken)) return null;
return {
@@ -366,23 +341,22 @@ export function listLiveBgSessions() {
export function doRespawn(session, log, opts = {}) {
const route = readRouteState();
+ clearLegacyRelayArtifacts(log);
if (route.mode === 'fail-closed') {
- // OPT-IN (2026-06-22, CRS-connection auto-recovery): a connection-wedged
- // session must be respawnable DURING a confirmed CRS outage (fail-closed is
- // exactly that state). In fail-closed, settings.json already has the CRS env
- // stripped, and the FINAL ATOMIC NORMALIZATION below strips any leaked CRS
- // base/token — so the respawn boots cleanly on direct keychain OAuth (the
- // intended "fall back to local OAuth"). Without the opt-in, keep refusing so
- // no existing caller (rotate.mjs post-rotation, daemon deferred sweep) changes
- // behavior. crs-health-watch's rotate-magic keeps the keychain account fresh.
+ // OPT-IN: a connection-wedged session must still be respawnable while the
+ // route is fail-closed. The env scrub below strips any leaked base URL or
+ // relay token, so the respawn boots cleanly on direct keychain OAuth — the
+ // intended "fall back to local OAuth". Without the opt-in, keep refusing so
+ // no existing caller (rotate.mjs post-rotation, daemon deferred sweep)
+ // changes behavior.
if (!opts.allowFailClosedDirect) {
log(
- `[bg-respawn] route=fail-closed; refusing to respawn session ${session.id} without healthy CRS OAuth or confirmed Bedrock`,
+ `[bg-respawn] route=fail-closed; refusing to respawn session ${session.id} without healthy OAuth or confirmed Bedrock`,
);
return false;
}
log(
- `[bg-respawn] route=fail-closed + allowFailClosedDirect — respawning session ${session.id} onto direct keychain OAuth (CRS-connection recovery)`,
+ `[bg-respawn] route=fail-closed + allowFailClosedDirect — respawning session ${session.id} onto direct keychain OAuth`,
);
}
@@ -452,252 +426,70 @@ export function doRespawn(session, log, opts = {}) {
log(`[bg-respawn] Failed to retrieve session token: ${err.message}`);
}
- // CRS relay allowlist: override base URL + auth for specific sessions (scoped partial cutover).
- // Rollback: delete crs-allowlist.json + respawn listed sessions (removes CRS env, falls back to keychain).
- //
- // ATOMIC INVARIANT (2026-06-14, root-caused 401 loop): ANTHROPIC_BASE_URL and the cr_
- // relay token MUST be set together or not at all. If base→CRS but the Bearer token is a
- // real OAuth token (non-cr_ prefix), CRS rejects every request with
- // {"error":"Invalid API key","message":"Invalid API key format"} → HTTP 401
- // which Claude Code surfaces as "API Error: 401 Invalid API key format" and retries
- // forever. The earlier bug: the cr_ key lived in a job tmp dir that got cleaned up, so the
- // readFileSync threw, the catch logged "non-fatal", and the session respawned with CRS base
- // URL + real OAuth token → infinite loop. Defenses below:
- // 1. Default key path is now a STABLE location (not job tmp).
- // 2. On ANY failure to obtain a cr_-prefixed key, we DELETE ANTHROPIC_BASE_URL from
- // childEnv so the session falls back to direct Anthropic — never CRS-with-wrong-token.
- const CRS_ALLOWLIST_PATH = join(__dirname, 'crs-allowlist.json');
- let crsActive = false; // true ⇔ a VALID base+cr_ pair was applied (the only state in which CRS routing is allowed)
- let crsBaseUrl = 'http://127.0.0.1:3005/api';
- let crsRelayKey = '';
- try {
- if (existsSync(CRS_ALLOWLIST_PATH)) {
- const al = JSON.parse(readFileSync(CRS_ALLOWLIST_PATH, 'utf8'));
- crsBaseUrl = al.baseUrl || crsBaseUrl;
- // CARVE-OUT (2026-06-15): explicit per-session exclusions that MUST stay
- // off CRS even under the global default — orchestrator 0d298397, app-store
- // sonnet build e35cdd60, driver 0b465ce3. The global default lives in
- // settings.json env, so an excluded session would otherwise inherit
- // base=CRS+cr_ at startup; we strip the inherited CRS base here and leave
- // crsActive=false so it falls back to keychain/direct (the final atomic
- // normalization below then drops any leaked cr_ token too). Reversible:
- // remove the id from al.exclude.
- const excluded = Array.isArray(al.exclude) && al.exclude.includes(String(session.id));
- if (excluded) {
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- log(
- `[bg-respawn] CRS-allowlist: session ${session.id} is an explicit carve-out (al.exclude) — forcing OFF CRS, stripping inherited CRS base variables (keychain/direct)`,
- );
- }
- // GLOBAL CRS (2026-06-14, Sam directive "by default all sessions route there, new AND existing"):
- // al.global===true routes EVERY respawned session via CRS, not just the listed cohort.
- // Downstream health-gate + cr_-key validity + atomic invariant still apply, so a global
- // session falls back to direct auth if CRS is unhealthy or the key is bad. Reversible: set global:false.
- if (
- !excluded &&
- (al.global === true || (Array.isArray(al.sessions) && al.sessions.includes(String(session.id))))
- ) {
- const crKeyPath = join(homedir(), '.claude', 'crs-keys', 'claude-cli.env');
- let crKey = '';
- try {
- crKey = loadClaudeHarnessEnv({ path: crKeyPath }).CRS_API_KEY;
- } catch {
- crKey = '';
- }
- // HEALTH GATE (2026-06-14, refined 2026-06-15): never route a session
- // onto a DEAD relay. A valid cr_ key is necessary but not sufficient —
- // if CRS is unhealthy, injecting the CRS base wedges the session on an
- // unreachable upstream. crsHealthyCheap() prefers the recent
- // crs-health-watch verdict (no network) and falls back to a 3s /health
- // curl. When down, fall through to strip-base direct-auth + drop the
- // dashboard relay-down marker so the "direct" session is explained.
- const crsHealthy = crsHealthyCheap();
- if (crKey.startsWith('cr_') && crsHealthy) {
- // Both together + relay alive — the only safe state.
- childEnv.ANTHROPIC_BASE_URL = crsBaseUrl;
- childEnv.ANTHROPIC_API_BASE = crsBaseUrl;
- childEnv.ANTHROPIC_AUTH_TOKEN = crKey;
- childEnv.CLAUDE_CODE_OAUTH_TOKEN = crKey;
- // Keep API_KEY=cr_ so Claude hits CRS (unsetting broke routing).
- childEnv.ANTHROPIC_API_KEY = crKey;
- crsRelayKey = crKey;
- crsActive = true;
- try {
- unlinkSync(CRS_RELAY_DOWN_MARKER);
- } catch {} // relay healthy → clear stale marker
- log(`[bg-respawn] CRS-allowlist: routing session ${session.id} via relay ${childEnv.ANTHROPIC_BASE_URL}`);
- } else if (crKey.startsWith('cr_') && !crsHealthy) {
- // Valid key but relay DOWN → fail safe to direct auth (do not wedge).
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- try {
- writeFileSync(
- CRS_RELAY_DOWN_MARKER,
- JSON.stringify({ ts: Date.now(), reason: 'relay-down', lastSession: String(session.id) }),
- // mode keeps the marker owner-only; the path under /tmp is predictable.
- { mode: 0o600 },
- );
- } catch {}
- log(
- `[bg-respawn] CRS-allowlist: session ${session.id} listed but CRS unhealthy — stripping ANTHROPIC_BASE_URL, falling back to direct auth (marker: ${CRS_RELAY_DOWN_MARKER})`,
- );
- } else {
- // Key missing/malformed → MUST NOT leave base URL pointing at CRS with a non-cr_ token.
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- log(
- `[bg-respawn] CRS-allowlist: session ${session.id} listed but cr_ key invalid/missing at ${crKeyPath} — stripping ANTHROPIC_BASE_URL, falling back to direct auth`,
- );
- }
- }
+ // Every respawned session authenticates directly with its own OAuth token.
+ // A retired relay backend used to override the base URL and inject a `cr_`
+ // bearer token here; both are now stripped rather than set, so a leftover
+ // pair inherited from settings.json or a parent env can never re-point a
+ // child at a relay this plugin no longer manages.
+ const scrubbed = [];
+ for (const key of ['ANTHROPIC_BASE_URL', 'ANTHROPIC_API_BASE']) {
+ if (isLegacyRelayBase(childEnv[key])) {
+ delete childEnv[key];
+ scrubbed.push(key);
}
- } catch (err) {
- // Allowlist parse failure must also be fail-safe: never route to CRS half-configured.
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- log(`[bg-respawn] CRS allowlist read failed (fail-safe: stripped ANTHROPIC_BASE_URL): ${err.message}`);
}
-
- // ── FINAL ATOMIC NORMALIZATION (2026-06-14, both-directions hardening) ────
- // The earlier guard only covered the allowlist-injection path. Two real
- // regressions slipped past it for spare-worker / --settings respawns:
- // (a) lone cr_ key (no CRS base) → sent to direct api.anthropic.com →
- // "401 Invalid API key format" (Anthropic rejects cr_).
- // (b) base=CRS (leaked via --settings crs-session-settings.json or an
- // inherited daemon env) + a real keychain OAuth token (which childEnv
- // injection sets, OVERRIDING the settings-file cr_ token) → the relay
- // rejects the non-cr_ token: "🔒 Invalid API key format from 172.18.0.1".
- // Invariant enforced here, unconditionally, as the LAST word on env:
- // base==CRS XNOR token==cr_ (both, or neither — never one).
- // crsActive is the ONLY sanctioned "both" path; anything else is forced to
- // direct: strip the CRS base AND replace any cr_ token with the keychain one.
- {
- const crsBasePattern =
- /127\.0\.0\.1:(3000|3002|3005|8091|18091)|100\.87\.53\.96:8091|:(3000|3002|3005|8091|18091)\/api/;
- const baseIsCrs =
- crsBasePattern.test(String(childEnv.ANTHROPIC_BASE_URL || '')) ||
- crsBasePattern.test(String(childEnv.ANTHROPIC_API_BASE || ''));
- const tokIsCr = [
- childEnv.CLAUDE_CODE_OAUTH_TOKEN,
- childEnv.ANTHROPIC_AUTH_TOKEN,
- childEnv.ANTHROPIC_API_KEY,
- ].some((token) => String(token || '').startsWith('cr_'));
- if (crsActive) {
- const crKey = String(childEnv.ANTHROPIC_API_KEY || '').startsWith('cr_')
- ? childEnv.ANTHROPIC_API_KEY
- : String(childEnv.ANTHROPIC_AUTH_TOKEN || '').startsWith('cr_')
- ? childEnv.ANTHROPIC_AUTH_TOKEN
- : childEnv.CLAUDE_CODE_OAUTH_TOKEN;
- if (String(crKey || '').startsWith('cr_')) {
- childEnv.ANTHROPIC_API_KEY = crKey;
- childEnv.ANTHROPIC_AUTH_TOKEN = crKey;
- childEnv.CLAUDE_CODE_OAUTH_TOKEN = crKey;
- }
- }
- if (!crsActive && (baseIsCrs || tokIsCr)) {
- if (baseIsCrs) {
- delete childEnv.ANTHROPIC_BASE_URL; // → falls back to default direct api.anthropic.com
- delete childEnv.ANTHROPIC_API_BASE;
- log(
- `[bg-respawn] ATOMIC: session ${session.id} not CRS-active but base→CRS leaked — stripped CRS base variables (direct)`,
- );
- }
- if (tokIsCr) {
- // Never let a lone cr_ relay key reach the direct API. Prefer the keychain token.
- let kc = null;
- try {
- const config = readConfig();
- const tokenJson = getTokenForSession(session.id, config);
- kc = tokenJson ? extractAccessToken(tokenJson) : null;
- } catch {}
- if (kc) {
- selectedDirectToken = kc;
- directCredentialSource = 'session-oauth';
- delete childEnv.ANTHROPIC_API_KEY;
- delete childEnv.ANTHROPIC_AUTH_TOKEN;
- childEnv.CLAUDE_CODE_OAUTH_TOKEN = kc;
- log(`[bg-respawn] ATOMIC: session ${session.id} had lone cr_ key — swapped to keychain OAuth (direct)`);
- } else {
- delete childEnv.ANTHROPIC_API_KEY; // fall through to keychain credentials file
- delete childEnv.ANTHROPIC_AUTH_TOKEN;
- delete childEnv.CLAUDE_CODE_OAUTH_TOKEN;
- log(
- `[bg-respawn] ATOMIC: session ${session.id} had lone cr_ key, no per-session token — deleted (keychain file fallback)`,
- );
- }
- }
+ for (const key of ['ANTHROPIC_API_KEY', 'ANTHROPIC_AUTH_TOKEN', 'CLAUDE_CODE_OAUTH_TOKEN']) {
+ if (isLegacyRelayToken(childEnv[key])) {
+ delete childEnv[key];
+ scrubbed.push(key);
}
}
+ if (scrubbed.length) {
+ log(`[bg-respawn] session ${session.id}: dropped leftover relay env (${scrubbed.join(', ')})`);
+ }
- if (!crsActive) {
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- delete childEnv.ANTHROPIC_API_KEY;
- delete childEnv.ANTHROPIC_AUTH_TOKEN;
- const effectiveEnvToken = String(childEnv.CLAUDE_CODE_OAUTH_TOKEN || '');
- if (effectiveEnvToken && !effectiveEnvToken.startsWith('cr_')) {
- selectedDirectToken = effectiveEnvToken;
- directCredentialSource = 'session-oauth';
- } else if (!selectedDirectToken) {
- selectedDirectToken = activeKeychainAccessToken();
- directCredentialSource = 'keychain';
- }
- if (!selectedDirectToken) {
- preserveDeferredMarker(session.id);
- log(`[bg-respawn] cannot fingerprint intended direct credential for ${session.id}; deferring retry`);
- return false;
- }
+ delete childEnv.ANTHROPIC_BASE_URL;
+ delete childEnv.ANTHROPIC_API_BASE;
+ delete childEnv.ANTHROPIC_API_KEY;
+ delete childEnv.ANTHROPIC_AUTH_TOKEN;
+ const effectiveEnvToken = String(childEnv.CLAUDE_CODE_OAUTH_TOKEN || '');
+ if (effectiveEnvToken) {
+ selectedDirectToken = effectiveEnvToken;
+ directCredentialSource = 'session-oauth';
+ } else if (!selectedDirectToken) {
+ selectedDirectToken = activeKeychainAccessToken();
+ directCredentialSource = 'keychain';
}
- const expectedRoute = crsActive
- ? {
- mode: 'crs',
- baseUrl: crsBaseUrl,
- credentialSource: 'crs-relay',
- credentialFingerprint: routeCredentialFingerprint(crsRelayKey),
- settings: join(homedir(), '.claude', 'crs-session-settings.json'),
- }
- : {
- mode: 'direct',
- baseUrl: null,
- credentialSource: directCredentialSource,
- credentialFingerprint: routeCredentialFingerprint(selectedDirectToken),
- settings: null,
- };
+ if (!selectedDirectToken) {
+ preserveDeferredMarker(session.id);
+ log(`[bg-respawn] cannot fingerprint intended direct credential for ${session.id}; deferring retry`);
+ return false;
+ }
+
+ const expectedRoute = {
+ mode: 'direct',
+ baseUrl: null,
+ credentialSource: directCredentialSource,
+ credentialFingerprint: routeCredentialFingerprint(selectedDirectToken),
+ settings: null,
+ };
childEnv.CLAUDE_OPS_EXPECTED_ROUTE = JSON.stringify(expectedRoute);
- // ── RESPAWN-FLAG RECONCILIATION (durable: base+token can't separate) ─────
- // `--settings crs-session-settings.json` injects ANTHROPIC_BASE_URL=CRS at
- // process startup, which childEnv stripping cannot undo. Make the allowlist
- // the SOLE authority for CRS routing: add the CRS settings flag iff crsActive,
- // remove it otherwise. Persisted to state.json so `claude respawn` re-reads it.
+ // A relay session-settings overlay passed via --settings survives a respawn
+ // because Claude Code re-reads it at startup. Drop it from the persisted
+ // flags so an upgraded machine stops re-applying it.
try {
const originalFlags = Array.isArray(state.respawnFlags) ? state.respawnFlags : null;
const flags = originalFlags ? normalizeClaudeModelArgs(originalFlags) : null;
if (flags) {
- const isCrsSettingsVal = (v) => typeof v === 'string' && v.endsWith('crs-session-settings.json');
- const hasCrsSettings = flags.some((f, i) => f === '--settings' && isCrsSettingsVal(flags[i + 1]));
+ const isLegacySettingsVal = (v) => typeof v === 'string' && v.endsWith(LEGACY_SESSION_SETTINGS_SUFFIX);
let mutated = flags.some((flag, index) => flag !== originalFlags[index]);
- if (!crsActive && hasCrsSettings) {
- for (let i = flags.length - 1; i >= 0; i--) {
- if (flags[i] === '--settings' && isCrsSettingsVal(flags[i + 1])) {
- flags.splice(i, 2);
- mutated = true;
- }
+ for (let i = flags.length - 1; i >= 0; i--) {
+ if (flags[i] === '--settings' && isLegacySettingsVal(flags[i + 1])) {
+ flags.splice(i, 2);
+ mutated = true;
+ log(`[bg-respawn] session ${session.id}: removed --settings relay overlay from respawnFlags`);
}
- log(
- `[bg-respawn] RECONCILE: session ${session.id} not CRS-active — removed --settings crs-session-settings.json from respawnFlags`,
- );
- } else if (crsActive && !hasCrsSettings) {
- const settingsPath = join(homedir(), '.claude', 'crs-session-settings.json');
- applyCrsSessionSettings({ baseUrl: crsBaseUrl, key: crsRelayKey });
- flags.push('--settings', settingsPath);
- mutated = true;
- log(
- `[bg-respawn] RECONCILE: session ${session.id} CRS-active — added --settings ${settingsPath} to respawnFlags`,
- );
- } else if (crsActive) {
- applyCrsSessionSettings({ baseUrl: crsBaseUrl, key: crsRelayKey });
}
if (mutated) {
state.respawnFlags = flags;
@@ -739,7 +531,7 @@ export function doRespawn(session, log, opts = {}) {
appendFileSync(LOGP, `[${new Date().toISOString()}] ${session.id}: ${respawnOut.slice(0, 2000)}\n`);
} catch {}
}
- if (!respawnStateCoherent(stateFile, crsActive)) {
+ if (!respawnStateCoherent(stateFile)) {
preserveDeferredMarker(session.id);
log(`[bg-respawn] respawn state verification failed for ${session.id}; deferring retry`);
return false;
diff --git a/claude-ops/scripts/account-rotation/bright-data-cascade.mjs b/claude-ops/scripts/account-rotation/bright-data-cascade.mjs
index 26c06646..b0adbe2c 100644
--- a/claude-ops/scripts/account-rotation/bright-data-cascade.mjs
+++ b/claude-ops/scripts/account-rotation/bright-data-cascade.mjs
@@ -1,5 +1,5 @@
// bright-data-cascade.mjs
-// Bright Data FALLBACK helpers for CRS captcha/oauth path.
+// Bright Data FALLBACK helpers for the captcha/oauth path.
//
// Order of intent (callers enforce solver gate separately):
// 1. Local residential browser (EFG SOCKS + PAC) — not this module
diff --git a/claude-ops/scripts/account-rotation/captcha-cascade.mjs b/claude-ops/scripts/account-rotation/captcha-cascade.mjs
index 144a5d7a..af339be7 100644
--- a/claude-ops/scripts/account-rotation/captcha-cascade.mjs
+++ b/claude-ops/scripts/account-rotation/captcha-cascade.mjs
@@ -309,7 +309,7 @@ export async function maybeSolvePostVerifyVisualChallenge(page, reason = '', opt
const injected = await maybeSolvePageCaptcha(page, reason || visual.kind || 'post-verify', {
timeoutMs: Number(process.env.CLAUDE_ROT_CAPTCHA_TIMEOUT_MS || 180_000),
attempt: used + 1,
- browserWaitMs: Number(process.env.CRS_CAPTCHA_BROWSER_WAIT_MS || 12000),
+ browserWaitMs: Number(process.env.CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS || 12000),
log,
});
diff --git a/claude-ops/scripts/account-rotation/captcha-helper.mjs b/claude-ops/scripts/account-rotation/captcha-helper.mjs
index d1cc8d63..04716ddf 100644
--- a/claude-ops/scripts/account-rotation/captcha-helper.mjs
+++ b/claude-ops/scripts/account-rotation/captcha-helper.mjs
@@ -12,7 +12,7 @@
// proxy alignment (token still from 2captcha). Web Unlocker is
// page-fetch only — see bright-data-cascade.mjs (not token inject).
//
-// Solvers remain FALLBACK only (CRS_CAPTCHA_SOLVER_MODE=fallback): residential
+// Solvers remain FALLBACK only (CLAUDE_ROT_CAPTCHA_SOLVER_MODE=fallback): residential
// browser wait first (residualAfterWait), then this provider chain.
//
// Secrets hydrated by secrets-bootstrap.mjs from env / optional secrets bootstrap.
@@ -119,10 +119,10 @@ export function providerOrder() {
import net from 'node:net';
const { createConnection } = net;
-/** True when local EFG SOCKS (or CRS_EFG_SOCKS_*) accepts connections. */
+/** True when local EFG SOCKS (CLAUDE_ROT_EFG_SOCKS_*) accepts connections. */
export async function residentialEgressHealthy() {
- const host = process.env.CRS_EFG_SOCKS_HOST || '127.0.0.1';
- const port = Number(process.env.CRS_EFG_SOCKS_PORT || 1089);
+ const host = process.env.CLAUDE_ROT_EFG_SOCKS_HOST || '127.0.0.1';
+ const port = Number(process.env.CLAUDE_ROT_EFG_SOCKS_PORT || 1089);
return await new Promise((resolve) => {
const s = createConnection({ host, port });
const t = setTimeout(() => {
@@ -146,18 +146,18 @@ export async function residentialEgressHealthy() {
}
/**
- * Solvers are fallback-only by default (CRS_CAPTCHA_SOLVER_MODE=fallback).
+ * Solvers are fallback-only by default (CLAUDE_ROT_CAPTCHA_SOLVER_MODE=fallback).
* @param {{ attempt?: number, force?: boolean, stalled?: boolean }} [ctx]
*/
export async function captchaSolverAllowed(ctx = {}) {
- const mode = (process.env.CRS_CAPTCHA_SOLVER_MODE || 'fallback').toLowerCase();
+ const mode = (process.env.CLAUDE_ROT_CAPTCHA_SOLVER_MODE || 'fallback').toLowerCase();
if (mode === 'off' || mode === '0' || mode === 'never') return { allowed: false, reason: 'mode-off' };
if (mode === 'always' || mode === 'on') return { allowed: true, reason: 'mode-always' };
if (ctx.force) return { allowed: true, reason: 'force' };
if (ctx.stalled) return { allowed: true, reason: 'stalled' };
const healthy = await residentialEgressHealthy();
if (!healthy) return { allowed: true, reason: 'residential-down' };
- const after = Number(process.env.CRS_CAPTCHA_SOLVER_AFTER_ATTEMPTS || 1);
+ const after = Number(process.env.CLAUDE_ROT_CAPTCHA_SOLVER_AFTER_ATTEMPTS || 1);
const attempt = Number(ctx.attempt || 0);
if (attempt >= after) return { allowed: true, reason: `attempt>=${after}` };
return { allowed: false, reason: 'prefer-residential-browser' };
@@ -167,10 +167,11 @@ function proxyParams() {
// + 2026-07-30: solvers are fallback-only. When used, prefer EFG SOCKS
// so token IP matches residential PAC browser (not third-party US residential).
const preferEfg =
- process.env.CLAUDE_ROT_CAPTCHA_PREFER_EFG_SOCKS === '1' || process.env.CRS_OAUTH_EGRESS === 'efg-socks-reauth-only';
- if (preferEfg && process.env.CRS_CAPTCHA_SKIP_EFG_PROXY !== '1') {
- const host = process.env.CRS_EFG_SOCKS_HOST || '127.0.0.1';
- const port = process.env.CRS_EFG_SOCKS_PORT || '1089';
+ process.env.CLAUDE_ROT_CAPTCHA_PREFER_EFG_SOCKS === '1' ||
+ process.env.CLAUDE_ROT_OAUTH_EGRESS === 'efg-socks-reauth-only';
+ if (preferEfg && process.env.CLAUDE_ROT_CAPTCHA_SKIP_EFG_PROXY !== '1') {
+ const host = process.env.CLAUDE_ROT_EFG_SOCKS_HOST || '127.0.0.1';
+ const port = process.env.CLAUDE_ROT_EFG_SOCKS_PORT || '1089';
return {
proxy: `${host}:${port}`,
proxytype: 'SOCKS5',
@@ -180,7 +181,10 @@ function proxyParams() {
};
}
// Bright Data residential/ISP proxy as next solver IP path (fallback cascade)
- if (process.env.CRS_CAPTCHA_BRIGHT_PROXY === '1' || process.env.CRS_CAPTCHA_SOLVER_MODE === 'fallback') {
+ if (
+ process.env.CLAUDE_ROT_CAPTCHA_BRIGHT_PROXY === '1' ||
+ process.env.CLAUDE_ROT_CAPTCHA_SOLVER_MODE === 'fallback'
+ ) {
try {
// lazy sync require not available in ESM — use env-built superproxy if zones present
const customer = process.env.BRIGHT_DATA_CUSTOMER || process.env.BRIGHT_DATA_USERID;
@@ -697,15 +701,15 @@ export async function solveWithBrightDataCascade(challenge, log = () => {}, opts
HTTP_URL: process.env.TWOCAPTCHA_PROXY_HTTP_URL,
SOCKS_URL: process.env.TWOCAPTCHA_PROXY_SOCKS5_URL,
PROXY_URL: process.env.TWOCAPTCHA_PROXY_URL,
- OAUTH_EGRESS: process.env.CRS_OAUTH_EGRESS,
+ OAUTH_EGRESS: process.env.CLAUDE_ROT_OAUTH_EGRESS,
};
try {
// Prefer Bright Data HTTP proxy over EFG for this cascade only.
process.env.CLAUDE_ROT_CAPTCHA_PREFER_EFG_SOCKS = '0';
- if (process.env.CRS_OAUTH_EGRESS === 'efg-socks-reauth-only') {
+ if (process.env.CLAUDE_ROT_OAUTH_EGRESS === 'efg-socks-reauth-only') {
// proxyParams() treats this as prefer-EFG; clear for BD tier pass only.
- delete process.env.CRS_OAUTH_EGRESS;
+ delete process.env.CLAUDE_ROT_OAUTH_EGRESS;
}
process.env.CLAUDE_ROT_CAPTCHA_FORCE_PROXY = '1';
process.env.CLAUDE_ROT_CAPTCHA_USE_PROXY = '1';
@@ -740,7 +744,7 @@ export async function solveWithBrightDataCascade(challenge, log = () => {}, opts
restore('TWOCAPTCHA_PROXY_HTTP_URL', saved.HTTP_URL);
restore('TWOCAPTCHA_PROXY_SOCKS5_URL', saved.SOCKS_URL);
restore('TWOCAPTCHA_PROXY_URL', saved.PROXY_URL);
- restore('CRS_OAUTH_EGRESS', saved.OAUTH_EGRESS);
+ restore('CLAUDE_ROT_OAUTH_EGRESS', saved.OAUTH_EGRESS);
}
return null;
}
@@ -911,7 +915,7 @@ export async function solveCaptchaOnPage(page, log = () => {}, opts = {}) {
// Residential-first: wait for browser path before paying for solvers.
// After wait, if captcha still present → treat as stalled residual (solver allowed).
let residualAfterWait = false;
- const waitMs = Number(opts.browserWaitMs ?? process.env.CRS_CAPTCHA_BROWSER_WAIT_MS ?? 0);
+ const waitMs = Number(opts.browserWaitMs ?? process.env.CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS ?? 0);
if (waitMs > 0 && !opts.skipBrowserWait) {
log(`captcha: residential-first wait ${waitMs}ms before solver fallback`);
await new Promise((r) => setTimeout(r, waitMs));
@@ -925,7 +929,8 @@ export async function solveCaptchaOnPage(page, log = () => {}, opts = {}) {
}
const gate = await captchaSolverAllowed({
- attempt: opts.attempt ?? (residualAfterWait ? Number(process.env.CRS_CAPTCHA_SOLVER_AFTER_ATTEMPTS || 1) : 0),
+ attempt:
+ opts.attempt ?? (residualAfterWait ? Number(process.env.CLAUDE_ROT_CAPTCHA_SOLVER_AFTER_ATTEMPTS || 1) : 0),
force: opts.forceSolver || residualAfterWait,
stalled: opts.stalled || residualAfterWait,
});
diff --git a/claude-ops/scripts/account-rotation/claude-harness-env.mjs b/claude-ops/scripts/account-rotation/claude-harness-env.mjs
deleted file mode 100644
index 29332ec0..00000000
--- a/claude-ops/scripts/account-rotation/claude-harness-env.mjs
+++ /dev/null
@@ -1,90 +0,0 @@
-import { closeSync, constants, fstatSync, openSync, readFileSync } from 'node:fs';
-import { homedir } from 'node:os';
-import { join } from 'node:path';
-
-const ALLOWED_KEYS = new Set([
- 'ANTHROPIC_BASE_URL',
- 'ANTHROPIC_API_BASE',
- 'ANTHROPIC_AUTH_TOKEN',
- 'ANTHROPIC_API_KEY',
- 'CLAUDE_CODE_OAUTH_TOKEN',
- 'CRS_API_KEY',
- 'CRS_HARNESS_NAME',
-]);
-const REQUIRED_KEYS = new Set([
- 'ANTHROPIC_BASE_URL',
- 'ANTHROPIC_API_BASE',
- 'ANTHROPIC_AUTH_TOKEN',
- 'CLAUDE_CODE_OAUTH_TOKEN',
- 'CRS_API_KEY',
- 'CRS_HARNESS_NAME',
-]);
-const ALLOWED_BASES = new Set(['http://127.0.0.1:3005/api', 'http://127.0.0.1:8091/api']);
-const TOKEN_RE = /^cr_[A-Za-z0-9._~-]+$/;
-const HARNESS_RE = /^[A-Za-z0-9._-]+$/;
-
-function invalid() {
- return new Error('Claude CRS harness env validation failed');
-}
-
-export function claudeHarnessEnvPath(home = homedir()) {
- return join(home, '.claude', 'crs-keys', 'claude-cli.env');
-}
-
-export function loadClaudeHarnessEnv(options = {}) {
- const path = options.path || claudeHarnessEnvPath(options.home);
- let mode;
- let source;
- let fd;
- try {
- // Open once, then inspect and read the handle we opened. Checking the path
- // and then opening it separately lets the file be swapped in between the
- // two steps. O_NOFOLLOW makes the open itself fail on a symlink, so the
- // earlier lstat check is no longer needed.
- fd = openSync(path, constants.O_RDONLY | constants.O_NOFOLLOW);
- mode = fstatSync(fd).mode & 0o777;
- source = readFileSync(fd, 'utf8');
- } catch {
- throw invalid();
- } finally {
- if (fd !== undefined) {
- try {
- closeSync(fd);
- } catch {
- /* ignore */
- }
- }
- }
- if (mode !== 0o600) throw invalid();
-
- const values = {};
- for (const rawLine of source.split(/\r?\n/)) {
- if (!rawLine || rawLine.startsWith('#')) continue;
- const separator = rawLine.indexOf('=');
- if (separator <= 0) throw invalid();
- const name = rawLine.slice(0, separator);
- const value = rawLine.slice(separator + 1);
- if (!ALLOWED_KEYS.has(name) || Object.hasOwn(values, name)) throw invalid();
- values[name] = value;
- }
- if ([...REQUIRED_KEYS].some((name) => !Object.hasOwn(values, name))) {
- throw invalid();
- }
- // API_KEY optional in file; when present must match the harness cr_ key.
- if (Object.hasOwn(values, 'ANTHROPIC_API_KEY') && values.ANTHROPIC_API_KEY !== values.CRS_API_KEY) {
- throw invalid();
- }
- if (
- !ALLOWED_BASES.has(values.ANTHROPIC_BASE_URL) ||
- values.ANTHROPIC_API_BASE !== values.ANTHROPIC_BASE_URL ||
- !TOKEN_RE.test(values.ANTHROPIC_AUTH_TOKEN) ||
- values.CLAUDE_CODE_OAUTH_TOKEN !== values.ANTHROPIC_AUTH_TOKEN ||
- values.CRS_API_KEY !== values.ANTHROPIC_AUTH_TOKEN ||
- !HARNESS_RE.test(values.CRS_HARNESS_NAME)
- ) {
- throw invalid();
- }
- // Always surface API_KEY=cr_ so launch paths do not strip the only usable CRS credential.
- values.ANTHROPIC_API_KEY = values.CRS_API_KEY;
- return values;
-}
diff --git a/claude-ops/scripts/account-rotation/claude-settings-mode.mjs b/claude-ops/scripts/account-rotation/claude-settings-mode.mjs
index 75e08691..5c59fbcb 100644
--- a/claude-ops/scripts/account-rotation/claude-settings-mode.mjs
+++ b/claude-ops/scripts/account-rotation/claude-settings-mode.mjs
@@ -15,25 +15,6 @@ function claudeSettingsPath() {
return join(process.env.HOME || '', '.claude', 'settings.json');
}
-// ── CRS (Claude Relay Service) routing awareness ───────────────────────────────
-// When a box routes through CRS, ANTHROPIC_BASE_URL points at the local relay
-// (e.g. http://127.0.0.1:3005/api) and CLAUDE_CODE_OAUTH_TOKEN holds a static
-// `cr_…` relay token. Both must travel together — a `cr_` token with no base URL
-// 401s. The Bedrock/OAuth settings mutators below were written for the raw-OAuth /
-// Bedrock world and would silently strip ANTHROPIC_BASE_URL, stranding a CRS box.
-// These helpers detect that routing and snapshot/restore the pair so a mode flip
-// never breaks (or money-leaks past) a CRS-routed fleet.
-function isCrsToken(t) {
- return typeof t === 'string' && t.startsWith('cr_');
-}
-
-/** True when settings.env is wired for CRS relay routing (cr_ token + base URL). */
-function detectCrsRouting(env) {
- const token = env.CLAUDE_CODE_OAUTH_TOKEN;
- const base = env.ANTHROPIC_BASE_URL;
- return isCrsToken(token) && typeof base === 'string' && base.length > 0 ? { token, base } : null;
-}
-
function readSettings() {
try {
return JSON.parse(readFileSync(claudeSettingsPath(), 'utf8'));
@@ -247,19 +228,6 @@ export function resolveBedrockClaudeModelIds(region = 'us-east-1') {
/** Match use-bedrock.sh: Bedrock env + strip top-level model lists (OAuth catalog ids break Bedrock). */
export function persistBedrockClaudeSettings(region = 'us-east-1') {
- const s0 = readSettings();
- const env0 = s0.env && typeof s0.env === 'object' ? s0.env : {};
- // Defense-in-depth behind the activateBedrockFallback kill-switch: a CRS-routed
- // box must NEVER be flipped onto metered Bedrock — the free relay pool already
- // multiplexes every Max account. Refuse loudly so callers' try/catch logs a
- // failure instead of silently claiming "Bedrock active" + money-leaking.
- // Override with CLAUDE_ALLOW_BEDROCK_OVER_CRS=1 if ever genuinely intended.
- if (detectCrsRouting(env0) && process.env.CLAUDE_ALLOW_BEDROCK_OVER_CRS !== '1') {
- throw new Error(
- 'refusing Bedrock flip: box is CRS-routed (cr_ token + ANTHROPIC_BASE_URL). ' +
- 'CRS pool already covers capacity; Bedrock is metered. Set CLAUDE_ALLOW_BEDROCK_OVER_CRS=1 to override.',
- );
- }
const { primary, small, opus, fable } = resolveBedrockClaudeModelIds(region);
const aws = resolveWorkingAwsEnv(region);
const s = readSettings();
@@ -294,12 +262,9 @@ export function persistBedrockClaudeSettings(region = 'us-east-1') {
export function clearHardcodedModelsForOAuthClaudeSettings() {
const s = readSettings();
const env = s.env && typeof s.env === 'object' ? { ...s.env } : {};
- // Preserve CRS relay routing across the flip: a `cr_` token + ANTHROPIC_BASE_URL
- // must survive together. The generic clear below strips ANTHROPIC_BASE_URL (it
- // assumes raw OAuth, where base-url must be absent), which on a CRS box would
- // leave the cr_ token pointing at api.anthropic.com → 401, killing the relay.
- // Snapshot the pair before clearing, then restore it.
- const crs = detectCrsRouting(env);
+ // Raw OAuth means no base-URL override and no injected key: Claude Code uses
+ // its own credential. A pooled setup routes through CLIProxyAPI, which owns
+ // the seat files rather than an env pair inside settings.json.
for (const k of [
'CLAUDE_CODE_USE_BEDROCK',
'AWS_BEDROCK_REGION',
@@ -316,11 +281,6 @@ export function clearHardcodedModelsForOAuthClaudeSettings() {
]) {
delete env[k];
}
- if (crs) {
- // Restore the relay pair so the box keeps routing through CRS, not raw OAuth.
- env.ANTHROPIC_BASE_URL = crs.base;
- env.CLAUDE_CODE_OAUTH_TOKEN = crs.token;
- }
s.env = env;
delete s.model;
delete s.availableModels;
diff --git a/claude-ops/scripts/account-rotation/claude-stack.mjs b/claude-ops/scripts/account-rotation/claude-stack.mjs
index 649bee1e..5c44e222 100644
--- a/claude-ops/scripts/account-rotation/claude-stack.mjs
+++ b/claude-ops/scripts/account-rotation/claude-stack.mjs
@@ -97,30 +97,25 @@ function accountRotationStatus() {
}
}
+/** Local CLIProxyAPI, the supported multi-account backend. */
+function cliProxyStatus() {
+ const base = (process.env.CLIPROXYAPI_BASE_URL || 'http://127.0.0.1:8317').replace(/\/+$/, '');
+ const healthUrl = `${base}/health`;
+ return { baseUrl: base, healthUrl, healthy: curlOk(healthUrl) };
+}
+
function status() {
const route = routeStatus();
- const crsHealthUrl = route.state.crs?.healthUrl || process.env.CRS_HEALTH_URL || 'http://127.0.0.1:3005/health';
return {
- ok: !route.settings.mixedProvider,
+ ok: !route.settings.mixedProvider && !route.settings.legacyRelay,
route,
- crs: {
- healthUrl: crsHealthUrl,
- healthy: curlOk(crsHealthUrl),
- },
+ cliproxy: cliProxyStatus(),
docker: dockerStatus(),
launchd: {
accountRotation: launchctlStatus('com.claude-ops.account-rotation'),
- crsHealthWatch: launchctlStatus('com.claude-ops.crs-health-watch'),
- crsPriority: launchctlStatus('com.claude-ops.crs-priority'),
- crsFraTunnel: launchctlStatus('com.claude-ops.crs-fra-tunnel'),
opsDaemon: launchctlStatus('com.claude-ops.daemon'),
},
systemd: {
- crsCompose: systemdUserStatus('crs-compose.service'),
- crsTokenFeed: systemdUserStatus('crs-token-feed.service'),
- crsBedrockGuard: systemdUserStatus('crs-bedrock-guard.service'),
- crsTokenFeedTimer: systemdUserStatus('crs-token-feed.timer'),
- crsBedrockGuardTimer: systemdUserStatus('crs-bedrock-guard.timer'),
claudeAccountRotation: systemdUserStatus('claude-account-rotation.service'),
},
accounts: accountRotationStatus(),
@@ -130,20 +125,18 @@ function status() {
function doctor() {
const s = status();
const findings = [];
- const fraPrimary = s.route.state.crs?.authority === 'fra-primary';
if (s.route.settings.mixedProvider)
findings.push({
severity: 'error',
code: 'mixed_provider_env',
- detail: 'settings.json has both Bedrock and CRS/OAuth env',
+ detail: 'settings.json has both Bedrock and an OAuth/base-URL override',
});
- if (s.route.state.mode === 'crs-oauth' && !s.crs.healthy)
- findings.push({ severity: 'error', code: 'crs_unhealthy', detail: `${s.crs.healthUrl} is not healthy` });
- if (s.route.state.mode === 'crs-oauth' && s.route.settings.tokenKind !== 'crs-relay')
+ if (s.route.settings.legacyRelay)
findings.push({
severity: 'error',
- code: 'crs_auth_token_missing',
- detail: 'CRS token variable is not a relay token (set ANTHROPIC_AUTH_TOKEN or CLAUDE_CODE_OAUTH_TOKEN)',
+ code: 'legacy_relay_env',
+ detail:
+ 'settings.json still carries relay credentials from the removed CRS backend. Run `claude-stack route --mode oauth` to clear them.',
});
if (s.route.state.mode === 'bedrock-confirmed' && !s.route.bedrockConfirmationActive)
findings.push({
@@ -152,32 +145,16 @@ function doctor() {
detail: 'Bedrock route lacks active TTL confirmation',
});
if (!s.docker.ok) findings.push({ severity: 'warn', code: 'docker_unavailable', detail: s.docker.error });
+ if (!s.cliproxy.healthy)
+ findings.push({
+ severity: 'warn',
+ code: 'cliproxy_unavailable',
+ detail: `${s.cliproxy.healthUrl} is not answering (optional unless you pool multiple accounts)`,
+ });
if (process.platform === 'darwin') {
- if (!fraPrimary && !s.launchd.accountRotation.loaded)
+ if (!s.launchd.accountRotation.loaded)
findings.push({ severity: 'warn', code: 'account_rotation_not_loaded', detail: s.launchd.accountRotation.error });
- if (!s.launchd.crsHealthWatch.loaded)
- findings.push({ severity: 'warn', code: 'crs_health_watch_not_loaded', detail: s.launchd.crsHealthWatch.error });
- if (fraPrimary && !s.launchd.crsFraTunnel.loaded)
- findings.push({ severity: 'error', code: 'crs_fra_tunnel_not_loaded', detail: s.launchd.crsFraTunnel.error });
} else {
- if (s.systemd.crsCompose.active !== 'active')
- findings.push({
- severity: 'error',
- code: 'crs_compose_not_active',
- detail: s.systemd.crsCompose.error || s.systemd.crsCompose.active,
- });
- if (s.systemd.crsTokenFeedTimer.active !== 'active')
- findings.push({
- severity: 'warn',
- code: 'crs_token_feed_timer_not_active',
- detail: s.systemd.crsTokenFeedTimer.error || s.systemd.crsTokenFeedTimer.active,
- });
- if (s.systemd.crsBedrockGuardTimer.active !== 'active')
- findings.push({
- severity: 'warn',
- code: 'crs_bedrock_guard_timer_not_active',
- detail: s.systemd.crsBedrockGuardTimer.error || s.systemd.crsBedrockGuardTimer.active,
- });
if (s.systemd.claudeAccountRotation.active !== 'active')
findings.push({
severity: 'warn',
@@ -225,23 +202,12 @@ try {
const mode = valueOf('--mode');
const reason = valueOf('--reason', 'manual');
const ttlMinutes = Number.parseInt(valueOf('--ttl-minutes', '60'), 10);
- const baseUrl = valueOf('--base-url');
- const healthUrl = valueOf('--health-url');
- const crs =
- baseUrl || healthUrl
- ? {
- ...routeStatus().state.crs,
- ...(baseUrl ? { baseUrl } : {}),
- ...(healthUrl ? { healthUrl } : {}),
- }
- : undefined;
const state = setRouteMode(mode, {
reason,
ttlMinutes: Number.isFinite(ttlMinutes) ? ttlMinutes : 60,
confirmMetered: has('--confirm-metered-bedrock'),
region: valueOf('--region', process.env.AWS_BEDROCK_REGION || 'us-east-1'),
updatedBy: 'claude-stack',
- ...(crs ? { crs } : {}),
});
printJson({ ok: true, state, settings: routeStatus().settings });
} else if (command === 'apply') {
diff --git a/claude-ops/scripts/account-rotation/config.example.json b/claude-ops/scripts/account-rotation/config.example.json
index 7176715e..1bc7449e 100644
--- a/claude-ops/scripts/account-rotation/config.example.json
+++ b/claude-ops/scripts/account-rotation/config.example.json
@@ -4,7 +4,6 @@
"_account_schema_example": {
"email": "[email protected]",
"label": "optional-disambiguator",
- "crsAccountName": "pool-example",
"orgName": "Required Workspace or Personal Account Name",
"orgUuid": "required-organization-uuid",
"dashlaneTokenPath": "dl://[email protected]/password",
@@ -29,38 +28,14 @@
"autoRotate": true,
"refreshSession": false,
"notifyOnRotation": true,
- "crs": {
- "_comment": "Optional: claude-relay-service (CRS) pool integration. Off by default — set enabled=true and install via scripts/install-crs-priority-agent.sh. Map each rotator account to its CRS row with crsAccountName on the account entry (or crs.nameByVaultKey). Admin password is NOT stored here: set $CRS_ADMIN_PASSWORD or credential-store CRS-Admin-<adminUser>.",
- "enabled": false,
- "policy": "conservative",
- "_policy_note": "conservative (default) deprioritizes on utilization/session warnings; max-out keeps accounts schedulable until genuine rate-limit or overload (util % is telemetry only). Override with $CRS_POLICY.",
- "baseUrl": "http://127.0.0.1:3005",
+ "rotation": {
+ "_comment": "Optional rotation tunables. Multi-account seats and OAuth seat management run through CLIProxyAPI; this block holds only the rotator's own settings. A pre-existing \"crs\" block from an older install is still read.",
"fileVaultPath": "~/.claude/.credentials.json",
- "containerName": "crs-claude-relay-1",
- "adminUser": "cradmin",
- "adminPasswordEnv": "CRS_ADMIN_PASSWORD",
- "policy": "conservative",
- "nameByVaultKey": {
- "[email protected]": "pool-example"
- },
- "off5h": 90,
- "off7d": 95,
- "on5h": 70,
- "on7d": 85,
- "floor": 3,
- "freshMinutes": 15,
- "_thresholds_note": "Legacy conservative thresholds (off/on/floor) apply only when policy=conservative. Default max-out policy keeps accounts schedulable until genuine rate-limit or overload; utilization % is telemetry only. freshMinutes = max age of claudeUsage trusted for secondary signals.",
- "logDir": null,
- "logTzOffset": "+00:00",
- "_logDir_note": "Optional: crs-429-cooldown.mjs reads claude-relay-error-<date>.log from here (absolute path, no ~ expansion) for its optional lead signal. Defaults to <plugin-data-dir>/account-rotation/crs-logs if unset — point it at your CRS deployment's actual log directory. logTzOffset must match the timezone your CRS instance timestamps logs in.",
- "errorLogWindowMs": 300000,
- "errorLogCooldownMs": 300000,
- "_errorLog_note": "errorLogWindowMs: how far back crs-429-cooldown.mjs scans the log for a 429 hit. errorLogCooldownMs: how long a log-only hit holds an account before rateLimitStatus itself would (default 5m each).",
"stateDir": null,
- "_stateDir_note": "Optional absolute path (no ~ expansion) for crs-429-cooldown.mjs state/manual-disable files (crs-429-state.json, crs-manual-disable.json). Defaults to <plugin-data-dir>/account-rotation if unset.",
- "manualDisableEnabled": true,
- "cooldownEnabled": false,
- "_cooldownEnabled_note": "Off by default (or $CRS_COOLDOWN_ENABLED=1). Turns on crs-429-cooldown.mjs, a fast (1min-cadence) reconciler that holds an account out of rotation only on a real rateLimitStatus.isRateLimited=true from CRS (plus the optional error-log lead signal under logDir/logTzOffset above) — never on utilization. Own state file at <stateDir>/crs-429-state.json.",
+ "_stateDir_note": "Optional absolute path (no ~ expansion) for reconciler state files. Defaults to <plugin-data-dir>/account-rotation if unset.",
+ "proxyEnabled": false,
+ "proxyProvider": "2captcha,brightdata",
+ "_proxy_note": "Per-account egress proxy for the captcha cascade. Set CLAUDE_ROTATION_PROXY_ENABLED=1 with EFG_PROXY_URL or BRIGHTDATA_PROXY_URL, or configure providers here. Credentials never live in this file.",
"enableMagicLinkRecovery": false,
"_enableMagicLinkRecovery_note": "Deprecated and fail-closed. Unattended reauthentication is disabled; use the signed staged-enrollment workflow. No environment variable re-enables direct auth writes.",
"magicLinkRetryCooldownMs": 21600000,
@@ -70,6 +45,6 @@
"magicLinkHeaded": true,
"_magicLinkHeaded_note": "Pass CLAUDE_ROT_HEADED=1 to the rotate child (default true). Interactive captcha walls need a headed browser on CLAUDE_DESKTOP_DISPLAY. Override with $CLAUDE_ROT_HEADED=0 only for pure headless experiments.",
"magicLinkRotateTimeoutMs": 1200000,
- "_magicLinkRotateTimeoutMs_note": "Wall-clock ms before the autoloop kills a stuck rotate child (default 20m). Override with $CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS. Captcha cascade (visual + desktop-act) needs more room than a pure token refresh."
+ "_magicLinkRotateTimeoutMs_note": "Wall-clock ms before the autoloop kills a stuck rotate child (default 20m). Override with $CLAUDE_REAUTH_TIMEOUT_MS. Captcha cascade (visual + desktop-act) needs more room than a pure token refresh."
}
}
diff --git a/claude-ops/scripts/account-rotation/crs-401-refresher.mjs b/claude-ops/scripts/account-rotation/crs-401-refresher.mjs
deleted file mode 100644
index 3743ae34..00000000
--- a/claude-ops/scripts/account-rotation/crs-401-refresher.mjs
+++ /dev/null
@@ -1,11 +0,0 @@
-#!/usr/bin/env node
-// Retired. This file contains no network, credential, or account mutation code.
-// Use refresh-tokens.mjs and crs-token-feed.mjs for identity-verified recovery.
-
-const status = process.argv.includes('--status');
-if (status) {
- console.log('CRS 401 refresher: retired; use identity-verified token refresh and feed');
- process.exit(0);
-}
-console.error('CRS_401_REFRESHER_RETIRED_USE_VERIFIED_TOKEN_FEED');
-process.exit(1);
diff --git a/claude-ops/scripts/account-rotation/crs-429-cooldown.mjs b/claude-ops/scripts/account-rotation/crs-429-cooldown.mjs
deleted file mode 100644
index 419990df..00000000
--- a/claude-ops/scripts/account-rotation/crs-429-cooldown.mjs
+++ /dev/null
@@ -1,376 +0,0 @@
-#!/usr/bin/env node
-// crs-429-cooldown.mjs — per-account cooldown driven by real Anthropic API 429s.
-//
-// OPT-IN. Does nothing (exits 0) unless crs.cooldownEnabled is true (or
-// $CRS_COOLDOWN_ENABLED=1) — installing this plugin never silently starts a new
-// daemon. Enable it alongside crs-priority-daemon.mjs when you run a self-hosted
-// claude-relay-service (CRS) pool and want a fast, real-signal-only cooldown watcher
-// that runs on a tighter cadence than the priority daemon.
-//
-// Two signals, in priority order — both are things Anthropic (or CRS observing
-// Anthropic) actually told us, never a speculative/utilization-based guess:
-// 1. rateLimitStatus.isRateLimited = true on the account record (CRS derived
-// this from a real upstream response on an actual inference call).
-// 2. (optional) tail of today's CRS relay error log for 429 lines mentioning the
-// account name — a faster lead signal than #1 while CRS's own record catches
-// up. Reads from crs.logDir (or $CRS_LOG_DIR); a no-op on a fresh install
-// since that directory won't exist yet (this reconciler never assumes a
-// real CRS deployment's log layout).
-//
-// State is owned exclusively by THIS reconciler (own file, atomic write, single-
-// flight lock via crs-reconciler-state.mjs) so that:
-// - a process restart does not release a held account early;
-// - crs-priority-daemon.mjs (which owns utilization-based scheduling) can run on
-// its own cadence without clobbering this reconciler's cooldown record, and
-// vice versa. See crs-reconciler-state.mjs for why that separation exists.
-//
-// Cron/timer cadence: run this more often than crs-priority-daemon.mjs (e.g. every
-// 1 minute vs every 5) since it exists to react to a live 429 fast.
-//
-// CONFIG (config.json "crs" block; every key overridable by env — see
-// config.example.json for the full annotated schema):
-// cooldownEnabled default false — must be true (or $CRS_COOLDOWN_ENABLED=1)
-// baseUrl, adminUser, adminPasswordEnv, containerName — shared with the other
-// CRS reconcilers
-// stateDir default: <plugin-data-dir>/account-rotation. This
-// reconciler's own state file lives at
-// <stateDir>/crs-429-state.json (never shared with any
-// other reconciler's state file).
-// manualDisableEnabled default true — set false to skip the manual-disable
-// allow-list check entirely
-// manualDisablePath default: <stateDir>/crs-manual-disable.json (optional
-// allow-list of account ids/names to always leave alone)
-// logDir default: <plugin-data-dir>/account-rotation/crs-logs.
-// This reconciler reads claude-relay-error-<date>.log
-// from here for its optional lead signal. Unset -> that
-// signal is disabled
-// (rateLimitStatus alone is still a real Anthropic signal
-// and is sufficient on its own).
-// logTzOffset default: "+00:00" — offset appended to naive log
-// timestamps before Date.parse; must match your CRS
-// host's log timezone if it isn't UTC.
-// errorLogWindowMs default: 300000 (5m) — how far back to scan the log
-// errorLogCooldownMs default: 300000 (5m) — cooldown granted on a log hit
-//
-// CLI: (none)=one tick · --dry-run=log decisions, no writes · --status=print
-// current held/live accounts and exit.
-
-import { readFileSync, existsSync } from 'fs';
-import { join, dirname } from 'path';
-import { fileURLToPath } from 'url';
-import { homedir } from 'os';
-import { loadJsonState, saveJsonStateAtomic, withOwnStateLock } from './crs-reconciler-state.mjs';
-import { crsBaseUrl, loadRotationConfig, resolveCrsAdminPassword } from './crs-pool-config.mjs';
-import { resolveAccountsBackend } from './ops-accounts-backend.mjs';
-
-const __dirname = dirname(fileURLToPath(import.meta.url));
-const args = new Set(process.argv.slice(2));
-const DRY = args.has('--dry-run') || process.env.CRS_DRY === '1';
-const STATUS = args.has('--status');
-
-const ts = () => new Date().toISOString().slice(11, 19);
-const log = (m) => console.log(`${ts()} [crs-429] ${m}`);
-
-const cfg = loadRotationConfig();
-const C = cfg.crs || {};
-
-function truthy(v) {
- return ['1', 'true', 'yes', 'on'].includes(
- String(v ?? '')
- .trim()
- .toLowerCase(),
- );
-}
-
-const ENABLED = truthy(process.env.CRS_COOLDOWN_ENABLED) || C.cooldownEnabled === true;
-
-const BASE = crsBaseUrl(cfg);
-const ADMIN_USER = process.env.CRS_ADMIN_USER || C.adminUser || 'cradmin';
-const num = (v, d) => (v === undefined || v === null || v === '' || Number.isNaN(+v) ? d : +v);
-
-function expandHome(p) {
- return p ? p.replace(/^~(?=$|\/)/, homedir()) : p;
-}
-
-const PLUGIN_DATA_DIR =
- process.env.CLAUDE_PLUGIN_DATA_DIR || join(homedir(), '.claude', 'plugins', 'data', 'ops-ops-marketplace');
-const DEFAULT_STATE_DIR = join(PLUGIN_DATA_DIR, 'account-rotation');
-const STATE_DIR = expandHome(process.env.CRS_STATE_DIR || C.stateDir) || DEFAULT_STATE_DIR;
-
-const STATE_PATH = expandHome(process.env.CRS_COOLDOWN_STATE_PATH) || join(STATE_DIR, 'crs-429-state.json');
-const MANUAL_DISABLE_ENABLED = C.manualDisableEnabled !== false;
-const MANUAL_DISABLE_PATH =
- expandHome(process.env.CRS_MANUAL_DISABLE_PATH || C.manualDisablePath) || join(STATE_DIR, 'crs-manual-disable.json');
-// Unset by default: the error-log lead signal is opt-in extra sensitivity, not a
-// required capability. rateLimitStatus alone (signal #1) is a real Anthropic
-// signal and is sufficient on its own.
-// Defaults to a real (likely-absent-on-a-fresh-install) path rather than a null
-// "disabled" sentinel: scanErrorLog() below no-ops safely via existsSync() when
-// the directory/file isn't there, so this stays a safe no-op until you actually
-// point your CRS deployment's error-log directory at it.
-const DEFAULT_LOG_DIR = join(DEFAULT_STATE_DIR, 'crs-logs');
-const ERROR_LOG_DIR = expandHome(process.env.CRS_LOG_DIR || C.logDir) || DEFAULT_LOG_DIR;
-const ERROR_LOG_WINDOW_MS = num(process.env.CRS_ERROR_LOG_WINDOW_MS ?? C.errorLogWindowMs, 5 * 60_000);
-const ERROR_LOG_COOLDOWN_MS = num(process.env.CRS_ERROR_LOG_COOLDOWN_MS ?? C.errorLogCooldownMs, 5 * 60_000);
-const ERROR_LOG_TZ = process.env.CRS_LOG_TZ_OFFSET || C.logTzOffset || '+00:00';
-
-const jfetch = async (path, opts = {}) => {
- const r = await fetch(BASE + path, opts);
- const txt = await r.text();
- let body;
- try {
- body = JSON.parse(txt);
- } catch {
- body = txt;
- }
- return { status: r.status, body };
-};
-
-async function login() {
- const password = resolveCrsAdminPassword(cfg, { adminUser: ADMIN_USER });
- if (!password) throw new Error('CRS admin password unavailable (set $CRS_ADMIN_PASSWORD or see crs-pool-config.mjs)');
- const r = await jfetch('/web/auth/login', {
- method: 'POST',
- headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ username: ADMIN_USER, password }),
- });
- if (r.status !== 200 || !r.body?.token) throw new Error('login failed HTTP ' + r.status);
- return 'Bearer ' + r.body.token;
-}
-
-async function getAccounts(auth) {
- const r = await jfetch('/admin/claude-accounts', { headers: { Authorization: auth } });
- if (r.status !== 200) throw new Error('GET accounts failed HTTP ' + r.status);
- const list = Array.isArray(r.body) ? r.body : r.body.data || r.body.accounts || [];
- return list.filter((a) => a.platform === 'claude' && a.isActive !== false);
-}
-
-const toggle = async (auth, id, schedulable, metadata = {}) =>
- jfetch('/admin/claude-accounts/' + id + '/toggle-schedulable', {
- method: 'PUT',
- headers: { Authorization: auth, 'Content-Type': 'application/json' },
- body: JSON.stringify({ schedulable, ...metadata }),
- });
-
-const parseTs = (s) => {
- if (!s) return null;
- const t = Date.parse(s);
- return Number.isFinite(t) ? t : null;
-};
-
-function loadManualDisable() {
- if (!MANUAL_DISABLE_ENABLED) return new Set();
- try {
- if (!existsSync(MANUAL_DISABLE_PATH)) return new Set();
- const o = JSON.parse(readFileSync(MANUAL_DISABLE_PATH, 'utf8'));
- return new Set(Array.isArray(o) ? o : Object.keys(o));
- } catch {
- return new Set();
- }
-}
-
-// === optional signal #2: tail today's CRS relay error log for 429 lines =====
-function scanErrorLog(nowMs) {
- if (!ERROR_LOG_DIR) return new Map();
- const today = new Date(nowMs).toISOString().slice(0, 10);
- const logPath = join(ERROR_LOG_DIR, `claude-relay-error-${today}.log`);
- if (!existsSync(logPath)) return new Map();
- let lines;
- try {
- lines = readFileSync(logPath, 'utf8').split('\n').filter(Boolean);
- } catch {
- return new Map();
- }
- const recent = new Map(); // account name -> latest hit ts (ms)
- const windowStart = nowMs - ERROR_LOG_WINDOW_MS;
- for (const line of lines) {
- try {
- const e = JSON.parse(line);
- const m = e.msg || '';
- if (!m.includes('error status: 429')) continue;
- const rawTs = String(e.ts || '').replace(' ', 'T');
- const t = Date.parse(ERROR_LOG_TZ && !/[Zz]|[+-]\d\d:?\d\d$/.test(rawTs) ? rawTs + ERROR_LOG_TZ : rawTs);
- if (!Number.isFinite(t) || t < windowStart) continue;
- const acc = m.match(/Account:\s*([^\s"]+)/);
- if (!acc) continue;
- const cur = recent.get(acc[1]);
- if (!cur || t > cur) recent.set(acc[1], t);
- } catch {}
- }
- return recent;
-}
-
-async function tick() {
- const auth = await login();
- const accts = await getAccounts(auth);
- const state = loadJsonState(STATE_PATH, log);
- const nowMs = Date.now();
- const errorLogHits = scanErrorLog(nowMs);
- const manualDisable = loadManualDisable();
-
- if (STATUS) {
- const held = Object.entries(state).filter(([, s]) => s.cooldownUntil > nowMs);
- console.log(`CRS 429-cooldown @ ${BASE} — ${accts.length} claude accounts, ${held.length} held`);
- for (const [id, s] of held) {
- console.log(` ${s.name || id}: held until ${new Date(s.cooldownUntil).toISOString()} (${s.reason})`);
- }
- return;
- }
-
- const changes = [];
- for (const a of accts) {
- if (manualDisable.has(a.id) || manualDisable.has(a.name)) continue;
- const cur = a.schedulable !== false;
- const rl = a.rateLimitStatus?.isRateLimited === true;
- const rlMin = Math.max(a.rateLimitStatus?.minutesRemaining || 0, 0);
- const rlEnd = parseTs(a.rateLimitStatus?.rateLimitEndAt);
- const persistedRedisHoldEnd = parseTs(a.rateLimitEndAt || a.rateLimitStatus?.rateLimitEndAt);
- const rateLimitReason = a.rateLimitReason || a.rateLimitStatus?.reason || null;
- const transient429 = rateLimitReason === 'transient_default_cooldown';
- const errLogHit = errorLogHits.get(a.name);
-
- // Trust only real upstream signals here (never claudeUsage.*.utilization —
- // that cache is fine for crs-priority-daemon.mjs's global health checks but
- // lags live state by minutes and must not drive a per-account hold).
- let cooldownUntil = 0;
- let reason = null;
- if (rl && !transient429) {
- const t = rlEnd && rlEnd > nowMs ? rlEnd : nowMs + Math.max(rlMin, 1) * 60_000;
- cooldownUntil = Math.max(cooldownUntil, t);
- reason = `api-rl ${Math.max(rlMin, 1)}m remaining${rateLimitReason ? ` (${rateLimitReason})` : ''}`;
- }
- if (errLogHit && rl && rateLimitReason && !transient429) {
- const t = errLogHit + ERROR_LOG_COOLDOWN_MS;
- if (t > cooldownUntil) {
- cooldownUntil = t;
- reason = `429 in error log @ ${new Date(errLogHit).toISOString()}`;
- }
- }
-
- const prevHeld = state[a.id]?.cooldownUntil || 0;
-
- if (cooldownUntil > nowMs) {
- state[a.id] = { name: a.name, cooldownUntil, reason };
- const holdMissingInRedis = !persistedRedisHoldEnd || persistedRedisHoldEnd < cooldownUntil - 1000;
- if (cur || holdMissingInRedis) {
- if (DRY) {
- changes.push({ name: a.name, action: cur ? '[dry]OFF' : '[dry]HOLD', reason, http: '-' });
- } else {
- const r = await toggle(auth, a.id, false, {
- rateLimitEndAt: new Date(cooldownUntil).toISOString(),
- rateLimitReason: reason || 'api-rl',
- });
- if (r.status < 200 || r.status >= 300) {
- state[a.id] = {
- ...state[a.id],
- retryAction: 'hold',
- retryAfter: nowMs + 60_000,
- lastToggleStatus: r.status,
- };
- } else {
- delete state[a.id].retryAction;
- delete state[a.id].retryAfter;
- delete state[a.id].lastToggleStatus;
- }
- changes.push({
- name: a.name,
- action: cur ? 'OFF' : 'HOLD',
- reason: `cooldown until ${new Date(cooldownUntil).toISOString().slice(11, 19)} (${reason})`,
- http: r.status,
- });
- }
- }
- } else if (prevHeld > nowMs) {
- // Persisted hold still wins — releasing before the recorded upstream
- // reset just re-hammers the same account.
- state[a.id] = { name: a.name, cooldownUntil: prevHeld, reason: state[a.id]?.reason || 'persisted-api-rl-bench' };
- const holdMissingInRedis = !persistedRedisHoldEnd || persistedRedisHoldEnd < prevHeld - 1000;
- if ((cur || holdMissingInRedis) && !DRY) {
- const r = await toggle(auth, a.id, false, {
- rateLimitEndAt: new Date(prevHeld).toISOString(),
- rateLimitReason: state[a.id]?.reason || 'persisted-api-rl-bench',
- });
- if (r.status < 200 || r.status >= 300) {
- state[a.id] = {
- ...state[a.id],
- retryAction: 'hold',
- retryAfter: nowMs + 60_000,
- lastToggleStatus: r.status,
- };
- } else {
- delete state[a.id].retryAction;
- delete state[a.id].retryAfter;
- delete state[a.id].lastToggleStatus;
- }
- changes.push({
- name: a.name,
- action: cur ? 'OFF' : 'HOLD',
- reason: `persisted cooldown until ${new Date(prevHeld).toISOString().slice(11, 19)}`,
- http: r.status,
- });
- } else if ((cur || holdMissingInRedis) && DRY) {
- changes.push({ name: a.name, action: '[dry]HOLD', reason: 'persisted cooldown', http: '-' });
- }
- } else if (prevHeld && prevHeld <= nowMs) {
- if (!cur) {
- if (DRY) {
- changes.push({ name: a.name, action: '[dry]ON', reason: 'cooldown expired', http: '-' });
- } else {
- const r = await toggle(auth, a.id, true);
- if (r.status >= 200 && r.status < 300) {
- delete state[a.id];
- } else {
- state[a.id] = {
- ...state[a.id],
- retryAction: 'release',
- retryAfter: nowMs + 60_000,
- lastToggleStatus: r.status,
- };
- }
- changes.push({ name: a.name, action: 'ON', reason: 'cooldown expired', http: r.status });
- }
- } else {
- delete state[a.id];
- }
- }
- }
-
- if (!DRY) saveJsonStateAtomic(STATE_PATH, state);
- const held = Object.values(state).filter((s) => s.cooldownUntil > nowMs);
- const live = accts.length - held.length;
- log(
- `tick: ${changes.length} change(s). live=${live}/${accts.length} | held=[${held
- .map((h) => `${h.name}@${new Date(h.cooldownUntil).toISOString().slice(11, 19)}`)
- .join(',')}]`,
- );
- for (const c of changes) log(` ${c.name}: ${c.action} (${c.reason || ''}) [HTTP ${c.http}]`);
-}
-
-async function main() {
- const backend = resolveAccountsBackend({ env: process.env, cfgBackend: C.backend });
- if (backend === 'local') {
- log('backend=local — CRS 429 cooldown no-op (use seat-policy-tick / exhaustedUntil on seat-state)');
- return;
- }
- if (!ENABLED && !STATUS) {
- log('crs.cooldownEnabled is not true (and $CRS_COOLDOWN_ENABLED!=1) — skipping tick (opt-in required)');
- return;
- }
- const outcome = await withOwnStateLock(STATE_PATH, tick, log);
- if (outcome.skipped) log('skipped tick: previous run still holds the state lock');
-}
-
-main().catch((e) => {
- const message = e?.message || String(e);
- if (
- message === 'fetch failed' ||
- message.includes('ECONNREFUSED') ||
- message.includes('ECONNRESET') ||
- message.includes('ETIMEDOUT')
- ) {
- log('WARN transient CRS fetch failure; skipping this tick');
- process.exit(0);
- }
- log('ERROR: ' + message);
- process.exit(1);
-});
diff --git a/claude-ops/scripts/account-rotation/crs-429-cooldown.sh b/claude-ops/scripts/account-rotation/crs-429-cooldown.sh
deleted file mode 100755
index 75f8ee36..00000000
--- a/claude-ops/scripts/account-rotation/crs-429-cooldown.sh
+++ /dev/null
@@ -1,21 +0,0 @@
-#!/bin/bash
-# crs-429-cooldown.sh — single-flight wrapper for crs-429-cooldown.mjs.
-# Invoked once-per-tick by launchd/systemd (recommended: every 60s — this
-# reconciler is meant to react fast to a real rate-limit hit).
-set -uo pipefail
-
-source "$HOME/.claude/scripts/lib/once.sh" 2>/dev/null || true
-type claude_once >/dev/null 2>&1 && { claude_once crs-429-cooldown 20 || exit 0; }
-
-DIR="$HOME/.claude/scripts/account-rotation"
-LOG="$DIR/crs-429-cooldown.log"
-NODE="$(command -v node || echo /opt/homebrew/bin/node)"
-
-export CLAUDE_PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$HOME/.claude/plugins/marketplaces/ops-marketplace/claude-ops}"
-
-if [ -f "$LOG" ]; then
- LOGSIZE="$(stat -c%s "$LOG" 2>/dev/null || stat -f%z "$LOG" 2>/dev/null || echo 0)"
- [ "$LOGSIZE" -gt 2097152 ] && mv "$LOG" "$LOG.1"
-fi
-
-"$NODE" "$DIR/crs-429-cooldown.mjs" >> "$LOG" 2>&1
diff --git a/claude-ops/scripts/account-rotation/crs-bedrock-guard.mjs b/claude-ops/scripts/account-rotation/crs-bedrock-guard.mjs
deleted file mode 100644
index eb23d244..00000000
--- a/claude-ops/scripts/account-rotation/crs-bedrock-guard.mjs
+++ /dev/null
@@ -1,116 +0,0 @@
-#!/usr/bin/env node
-/**
- * crs-bedrock-guard.mjs — WARN when the CRS relay actually serves via the
- * bedrock-fallback account (i.e. the OAuth Max pool got exhausted and CRS fell
- * back to paid AWS Bedrock). App-level signal: polls the bedrock-fallback
- * account's usage.total.requests and alerts when it climbs.
- *
- * Pairs with the network-level bedrock-net-monitor (catches Bedrock from ANY
- * source). Runs via crs-bedrock-guard.timer (every 5 min).
- */
-import { readFileSync, writeFileSync, renameSync, appendFileSync, mkdirSync } from 'fs';
-import { execSync, spawnSync } from 'child_process';
-import { join } from 'path';
-
-const HOME = process.env.HOME;
-const CRS = 'http://127.0.0.1:3005';
-const STATE = join(HOME, '.claude', '.crs-bedrock-guard-state.json');
-const ALERT = join(HOME, '.claude', '.bedrock-alert.json');
-const LOGDIR = join(HOME, '.claude', 'logs');
-const LOG = join(LOGDIR, 'bedrock-alerts.log');
-
-function log(m) {
- try {
- mkdirSync(LOGDIR, { recursive: true });
- appendFileSync(LOG, `[${new Date().toISOString()}] [crs-guard] ${m}\n`);
- } catch {}
- console.log(m);
-}
-function loadState() {
- try {
- return JSON.parse(readFileSync(STATE, 'utf8'));
- } catch {
- return {};
- }
-}
-function saveState(s) {
- const t = `${STATE}.tmp.${process.pid}`;
- writeFileSync(t, JSON.stringify(s, null, 2));
- renameSync(t, STATE);
-}
-
-function alert(payload) {
- try {
- const t = `${ALERT}.tmp.${process.pid}`;
- writeFileSync(t, JSON.stringify(payload, null, 2));
- renameSync(t, ALERT);
- } catch {}
- log(`🚨 ALERT: ${payload.message}`);
- try {
- spawnSync('notify-send', ['-u', 'critical', '🚨 CRS→Bedrock fallback', payload.message], { timeout: 3000 });
- } catch {}
-}
-
-async function main() {
- let pw = '';
- try {
- pw = execSync(
- `docker inspect crs-claude-relay-1 --format '{{range .Config.Env}}{{println .}}{{end}}' | sed -n 's/^ADMIN_PASSWORD=//p'`,
- { timeout: 8000 },
- )
- .toString()
- .trim();
- } catch {}
- if (!pw) {
- log('CRS not reachable (no admin pw) — skip');
- return;
- }
- const login = await fetch(`${CRS}/web/auth/login`, {
- method: 'POST',
- headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ username: 'cradmin', password: pw }),
- })
- .then((r) => r.json())
- .catch(() => ({}));
- const tok = login.token || login.data?.token;
- if (!tok) {
- log('CRS login failed — skip');
- return;
- }
- const resp = await fetch(`${CRS}/admin/bedrock-accounts`, { headers: { Authorization: `Bearer ${tok}` } })
- .then((r) => r.json())
- .catch(() => ({}));
- const accts = resp.data || resp.accounts || resp || [];
- const be = (Array.isArray(accts) ? accts : []).find((a) => a.platform === 'bedrock' || a.type === 'bedrock');
- if (!be) {
- log('no bedrock account registered — skip');
- return;
- }
-
- const totalReq = be.usage?.total?.requests ?? 0;
- const dailyReq = be.usage?.daily?.requests ?? 0;
- const dailyCost = be.usage?.daily?.cost ?? 0;
- const st = loadState();
- const prev = st.totalRequests ?? 0;
-
- if (totalReq > prev) {
- const delta = totalReq - prev;
- alert({
- ts: new Date().toISOString(),
- source: 'crs-pool-fallback',
- account: be.name,
- message: `CRS served ${delta} request(s) via AWS Bedrock fallback (OAuth pool was exhausted). total=${totalReq}, today=${dailyReq} req / $${dailyCost}. Check why the 10-account Max pool ran dry.`,
- deltaRequests: delta,
- totalRequests: totalReq,
- dailyRequests: dailyReq,
- dailyCost,
- });
- } else {
- log(`ok — bedrock total=${totalReq} (no new fallback traffic)`);
- }
- saveState({ totalRequests: totalReq, lastCheck: new Date().toISOString() });
-}
-main().catch((e) => {
- log(`fatal: ${e.message}`);
- process.exit(1);
-});
diff --git a/claude-ops/scripts/account-rotation/crs-egress-failover.sh b/claude-ops/scripts/account-rotation/crs-egress-failover.sh
deleted file mode 100644
index bbab3d99..00000000
--- a/claude-ops/scripts/account-rotation/crs-egress-failover.sh
+++ /dev/null
@@ -1,61 +0,0 @@
-#!/usr/bin/env bash
-set -u
-
-DROPIN="$HOME/.config/systemd/user/crs-priority.service.d/40-mac-residential-egress.conf"
-LOCK="${XDG_RUNTIME_DIR:-/tmp}/crs-egress-failover.lock"
-EFG_PROXY='http://127.0.0.1:1088'
-UDM_PROXY='socks5h://127.0.0.1:1087'
-MAC_PROXY='socks5h://127.0.0.1:1086'
-TARGET='https://api.anthropic.com'
-
-mkdir -p "$(dirname "$DROPIN")"
-exec 9>"$LOCK"
-flock -n 9 || exit 0
-
-proxy_healthy() {
- local proxy="$1" status
- status=$(curl -ksS --max-time 15 --proxy "$proxy" -o /dev/null -w '%{http_code}' "$TARGET" 2>/dev/null) || return 1
- case "$status" in
- 2*|3*|4*|5*) return 0 ;;
- *) return 1 ;;
- esac
-}
-
-if proxy_healthy "$UDM_PROXY"; then
- desired='socks5://127.0.0.1:1087'
-elif proxy_healthy "$EFG_PROXY"; then
- desired='http://127.0.0.1:1088'
-elif proxy_healthy "$MAC_PROXY"; then
- desired='socks5://127.0.0.1:1086'
-else
- exit 1
-fi
-
-current=$(python3 - "$DROPIN" <<'PY'
-import pathlib, sys
-p = pathlib.Path(sys.argv[1])
-try:
- for line in p.read_text().splitlines():
- if line.startswith('Environment=EFG_PROXY_URL='):
- print(line.split('=', 2)[2])
- break
-except FileNotFoundError:
- pass
-PY
-)
-
-if [ "$current" = "$desired" ]; then
- exit 0
-fi
-
-tmp="$DROPIN.tmp.$$"
-cat > "$tmp" <<EOF
-[Service]
-# Scoped to live Anthropic quota probes only. Never changes host-wide routing.
-Environment=EFG_PROXY_URL=$desired
-EOF
-chmod 600 "$tmp"
-mv -f "$tmp" "$DROPIN"
-systemctl --user daemon-reload
-systemctl --user restart crs-priority.service
-printf 'switched_proxy=%s\n' "$desired"
diff --git a/claude-ops/scripts/account-rotation/crs-health-watch.mjs b/claude-ops/scripts/account-rotation/crs-health-watch.mjs
deleted file mode 100755
index 13718d01..00000000
--- a/claude-ops/scripts/account-rotation/crs-health-watch.mjs
+++ /dev/null
@@ -1,438 +0,0 @@
-#!/usr/bin/env node
-// crs-health-watch.mjs — fleet-level CRS failsafe (single tick; launchd-driven).
-//
-// PROBLEM: the global CRS default lives in ~/.claude/settings.json `env`
-// (ANTHROPIC_BASE_URL=CRS + CLAUDE_CODE_OAUTH_TOKEN=cr_…). settings.json env is
-// applied at `claude` startup and OVERRIDES any childEnv a respawn sets — so the
-// per-session health-gate in bg-respawn.mjs CANNOT protect new/respawned sessions
-// when the relay is down: they'd boot with base=CRS pointing at a dead upstream and
-// wedge on connection errors. This watch is the authority over settings.json.
-//
-// POLICY (hysteresis, no flap):
-// • Probe the health URL from claude-routing-state each tick.
-// • DOWN for DOWN_STRIKES consecutive ticks → FALLBACK: strip the two CRS env keys
-// from settings.json (new/respawned sessions go direct keychain auth). Drop a
-// marker so we know we're in fallback.
-// • UP for UP_STRIKES consecutive ticks while in fallback → RESTORE: re-add the two
-// CRS env keys. Clear the marker.
-// • Otherwise hold. State (counters + mode) persisted to crs-health-watch.state.json.
-//
-// It NEVER force-respawns running sessions (avoids thrash). It only flips the global
-// default so the fleet self-heals to direct on the next respawn/new session, and back
-// to CRS once the relay is solidly up. Manual rollback is independent (see status doc).
-//
-// CLI: (none)=tick · --status=print mode+counters+current env state · --restore=force
-// restore CRS env now · --fallback=force fail-closed mode now.
-
-import { readFileSync, writeFileSync, existsSync } from 'fs';
-import { execFileSync } from 'child_process';
-import { dirname, join } from 'path';
-import { fileURLToPath } from 'url';
-import { homedir } from 'os';
-import { applyRouteToSettings, readRouteState, setRouteMode } from './route-state.mjs';
-import { healCrsRelay } from './crs-heal-relay.mjs';
-
-const __dirname = dirname(fileURLToPath(import.meta.url));
-const args = new Set(process.argv.slice(2));
-
-const SETTINGS = join(homedir(), '.claude', 'settings.json');
-const STATE = join(__dirname, 'crs-health-watch.state.json');
-const MARKER = join(__dirname, 'crs-fallback-active'); // legacy marker present while route is fail-closed
-const REBOOT_COOLDOWN_MARKER = join(__dirname, 'crs-reboot-cooldown'); // holds last OS-wedge reboot epoch ms
-
-// dev-us EC2 (us-east-1) serves CRS for the entire fleet. Prefer the canonical
-// instance-id sync file; fall back to the known id from project config.
-const DEV_US_INSTANCE_ID = (() => {
- const f = join(homedir(), '.claude', 'sync', 'dev-us-instance.id');
- try {
- const v = readFileSync(f, 'utf8').trim();
- if (/^i-[0-9a-f]+$/.test(v)) return v;
- } catch {}
- return 'i-01b4d5132c1b167bf';
-})();
-
-function routeCrsConfig() {
- try {
- return readRouteState().crs || {};
- } catch {
- return {};
- }
-}
-
-const ROUTE_CRS = routeCrsConfig();
-const CRS_BASE = process.env.CRS_BASE_URL || ROUTE_CRS.baseUrl || 'http://127.0.0.1:3000/api';
-const HEALTH_URL = process.env.CRS_HEALTH_URL || ROUTE_CRS.healthUrl || CRS_BASE.replace(/\/api\/?$/, '/health');
-const CRS_SMOKE_URL = process.env.CRS_SMOKE_URL || `${CRS_BASE}/v1/messages?beta=true`;
-const CRS_SMOKE_MODEL = process.env.CRS_SMOKE_MODEL || 'claude-sonnet-4-6';
-// Phase 1b: default ON (was opt-in ==='1' which caused silent-healthy when inference=false).
-const CRS_ENABLE_INFERENCE_SMOKE = process.env.CRS_ENABLE_INFERENCE_SMOKE !== '0';
-const DOWN_STRIKES = +(process.env.CRS_DOWN_STRIKES || 3); // ~3 min at 60s tick
-const UP_STRIKES = +(process.env.CRS_UP_STRIKES || 1);
-// OS-wedge auto-reboot threshold. Must be >= DOWN_STRIKES so we only reboot AFTER
-// fail-closed has already protected the fleet (~5 min sustained-down at 60s tick).
-const REBOOT_STRIKES = Math.max(DOWN_STRIKES, +(process.env.CRS_REBOOT_STRIKES || 5));
-const REBOOT_COOLDOWN_MS = 30 * 60 * 1000; // at most one auto-reboot per 30 min
-
-const ts = () => new Date().toISOString().slice(11, 19);
-const log = (m) => console.log(`${ts()} [crs-health-watch] ${m}`);
-
-function readJson(p, d) {
- try {
- return JSON.parse(readFileSync(p, 'utf8'));
- } catch {
- return d;
- }
-}
-function loadState() {
- return readJson(STATE, { down: 0, up: 0, mode: 'crs' });
-}
-function saveState(s) {
- try {
- writeFileSync(STATE, JSON.stringify(s, null, 2));
- } catch (e) {
- log(`state write failed: ${e.message}`);
- }
-}
-
-function probe() {
- try {
- const code = execFileSync('curl', ['-sf', '-o', '/dev/null', '-w', '%{http_code}', '--max-time', '3', HEALTH_URL], {
- encoding: 'utf8',
- timeout: 5000,
- }).trim();
- // Phase 1b: also fail probe on CRS-reported degraded (inference smoke is the separate hard check for "inference=false")
- let body = '';
- try {
- body = execFileSync('curl', ['-sS', '--max-time', '2', HEALTH_URL], { encoding: 'utf8', timeout: 3000 }).trim();
- } catch {}
- if (body && /"status"\s*:\s*"(degraded|unhealthy)"/i.test(body)) return false;
- return code === '200';
- } catch {
- return false;
- }
-}
-
-function probeInference() {
- let key = '';
- try {
- key = readFileSync(join(__dirname, '.crkey'), 'utf8').trim();
- } catch {}
- if (!key.startsWith('cr_')) return false;
- const body = JSON.stringify({
- model: CRS_SMOKE_MODEL,
- max_tokens: 1,
- stream: false,
- messages: [{ role: 'user', content: 'ping' }],
- });
- try {
- const code = execFileSync(
- 'curl',
- [
- '-sS',
- '-o',
- '/dev/null',
- '-w',
- '%{http_code}',
- '--max-time',
- '25',
- '-H',
- `Authorization: Bearer ${key}`,
- '-H',
- 'Content-Type: application/json',
- '-H',
- 'anthropic-version: 2023-06-01',
- '-d',
- body,
- CRS_SMOKE_URL,
- ],
- { encoding: 'utf8', timeout: 30_000 },
- ).trim();
- // 529 means CRS selected a valid account and reached Anthropic, but the
- // upstream is temporarily overloaded. That is not a CRS auth/routing fault.
- return /^2\d\d$/.test(code) || code === '529';
- } catch {
- return false;
- }
-}
-
-function setEnvMode(mode) {
- try {
- if (mode === 'direct') {
- setRouteMode('fail-closed', {
- reason: 'crs-health-watch: CRS unavailable',
- updatedBy: 'crs-health-watch',
- });
- } else {
- setRouteMode('crs-oauth', {
- reason: 'crs-health-watch: CRS recovered',
- updatedBy: 'crs-health-watch',
- });
- }
- return true;
- } catch (e) {
- log(`route/settings write failed: ${e.message}`);
- return false;
- }
-}
-
-function currentEnvMode() {
- const s = readJson(SETTINGS, { env: {} });
- const base = s.env?.ANTHROPIC_BASE_URL || '';
- const tok = s.env?.CLAUDE_CODE_OAUTH_TOKEN || '';
- const bedrock = s.env?.CLAUDE_CODE_USE_BEDROCK || '';
- if (/127\.0\.0\.1:(3000|3005)|:(3000|3005)\/api/.test(base) && tok.startsWith('cr_')) return 'crs';
- if (bedrock === '1') return 'bedrock';
- if (!base && !tok) return 'direct';
- return 'mixed';
-}
-
-// SSM PingStatus for the dev-us instance. "ConnectionLost" => the SSM agent itself
-// is unreachable, i.e. an OS-level wedge (reboot can help). "Online" => OS is fine and
-// CRS is merely app-down (reboot is unwarranted). Returns the literal string, or an
-// "ERROR:<msg>" sentinel on AWS failure (treated as not-ConnectionLost by callers).
-function ssmPingStatus(instanceId) {
- try {
- return execFileSync(
- 'aws',
- [
- 'ssm',
- 'describe-instance-information',
- '--region',
- 'us-east-1',
- '--filters',
- `Key=InstanceIds,Values=${instanceId}`,
- '--query',
- 'InstanceInformationList[0].PingStatus',
- '--output',
- 'text',
- ],
- { encoding: 'utf8', timeout: 30_000 }, // generous: aws CLI cold-starts each tick; this read decides the reboot
- ).trim();
- } catch (e) {
- return `ERROR:${e.message}`;
- }
-}
-
-// Milliseconds left on the auto-reboot cooldown (0 = no cooldown / clear to reboot).
-function rebootCooldownRemainingMs() {
- try {
- const last = +readFileSync(REBOOT_COOLDOWN_MARKER, 'utf8').trim();
- if (!Number.isFinite(last)) return 0;
- const rem = REBOOT_COOLDOWN_MS - (Date.now() - last);
- return rem > 0 ? rem : 0;
- } catch {
- return 0;
- }
-}
-
-// Fire an EC2 reboot ONLY on a double-confirmed OS wedge: CRS sustained-down past
-// REBOOT_STRIKES, already fail-closed (fleet protected), SSM says ConnectionLost, and
-// no reboot in the last 30 min. Conservative by design — every gate must hold.
-// Are we in fail-closed mode? Read the marker instead of statting it: a stat is
-// a check, and the claim in tick() must not hang off a check of the same path.
-// A failed read tells us the same thing without one.
-function markerPresent() {
- try {
- readFileSync(MARKER, 'utf8');
- return true;
- } catch {
- return false;
- }
-}
-
-function maybeAutoReboot(st, inFallback, healthy) {
- if (healthy || st.down < REBOOT_STRIKES || !inFallback) return;
-
- const cooldown = rebootCooldownRemainingMs();
- if (cooldown > 0) {
- log(
- `reboot: conditions met (down x${st.down}) but cooldown active (${Math.ceil(cooldown / 60000)}m remaining) → skip`,
- );
- return;
- }
- const ping = ssmPingStatus(DEV_US_INSTANCE_ID);
- if (ping !== 'ConnectionLost') {
- log(
- `reboot: down x${st.down} + fail-closed, but SSM PingStatus=${ping} (not ConnectionLost) → app-level, NOT rebooting`,
- );
- return;
- }
- try {
- execFileSync('aws', ['ec2', 'reboot-instances', '--region', 'us-east-1', '--instance-ids', DEV_US_INSTANCE_ID], {
- encoding: 'utf8',
- timeout: 20_000,
- });
- writeFileSync(REBOOT_COOLDOWN_MARKER, String(Date.now()));
- log(
- `🔁 OS-WEDGE AUTO-REBOOT: CRS down x${st.down} + SSM ConnectionLost → reboot-instances ${DEV_US_INSTANCE_ID} (30m cooldown armed)`,
- );
- } catch (e) {
- log(`WARN: auto-reboot failed: ${e.message}`);
- }
-}
-
-function main() {
- if (args.has('--dry-run-reboot')) {
- const st = loadState();
- const healthOk = probe();
- const inferenceOk = CRS_ENABLE_INFERENCE_SMOKE && healthOk ? probeInference() : false;
- const healthy = healthOk; // mirrors the live tick's health gate
- const inFallback = existsSync(MARKER);
- const ping = ssmPingStatus(DEV_US_INSTANCE_ID);
- const cooldownRemMs = rebootCooldownRemainingMs();
- const wouldReboot =
- !healthy && st.down >= REBOOT_STRIKES && inFallback && ping === 'ConnectionLost' && cooldownRemMs === 0;
- console.log(
- JSON.stringify(
- {
- dryRunReboot: true,
- instanceId: DEV_US_INSTANCE_ID,
- healthy,
- healthOk,
- inferenceOk,
- stDown: st.down,
- rebootStrikes: REBOOT_STRIKES,
- downThresholdMet: st.down >= REBOOT_STRIKES,
- inFallback,
- ssmPingStatus: ping,
- ssmConnectionLost: ping === 'ConnectionLost',
- cooldownRemainingMin: Math.ceil(cooldownRemMs / 60000),
- cooldownActive: cooldownRemMs > 0,
- WOULD_REBOOT: wouldReboot,
- },
- null,
- 2,
- ),
- );
- return;
- }
-
- if (args.has('--status')) {
- const st = loadState();
- const health = probe();
- const route = readRouteState();
- console.log(
- JSON.stringify(
- {
- state: st,
- routeMode: route.mode,
- fallbackActive: existsSync(MARKER),
- settingsEnvMode: currentEnvMode(),
- healthy: health,
- inferenceHealthy: CRS_ENABLE_INFERENCE_SMOKE && health ? probeInference() : false,
- inferenceSmokeEnabled: CRS_ENABLE_INFERENCE_SMOKE,
- smokeModel: CRS_SMOKE_MODEL || null,
- },
- null,
- 2,
- ),
- );
- return;
- }
- if (args.has('--restore')) {
- if (probe() && (!CRS_ENABLE_INFERENCE_SMOKE || probeInference()) && setEnvMode('crs')) {
- try {
- execFileSync('rm', ['-f', MARKER]);
- } catch {}
- saveState({ down: 0, up: 0, mode: 'crs' });
- log('FORCED restore → CRS env');
- } else {
- setEnvMode('direct');
- // Forced path: overwrite whatever is there, but keep the marker owner-only.
- writeFileSync(MARKER, String(Date.now()), { mode: 0o600 });
- saveState({ down: 0, up: 0, mode: 'fail-closed' });
- log('FORCED restore refused: CRS health or inference smoke failed → fail-closed env');
- process.exitCode = 1;
- }
- return;
- }
- if (args.has('--fallback')) {
- setEnvMode('direct');
- // Forced path: overwrite whatever is there, but keep the marker owner-only.
- writeFileSync(MARKER, String(Date.now()), { mode: 0o600 });
- saveState({ down: 0, up: 0, mode: 'fail-closed' });
- log('FORCED fallback → fail-closed env');
- return;
- }
-
- try {
- applyRouteToSettings(readRouteState());
- } catch (e) {
- log(`route apply skipped: ${e.message}`);
- }
-
- const healthOk = probe();
- const inferenceOk = CRS_ENABLE_INFERENCE_SMOKE ? probeInference() : false;
- // Phase 1b: inference=false is hard-fail (triggers heal + affects route fallback) instead of silent healthy.
- if (CRS_ENABLE_INFERENCE_SMOKE && !inferenceOk) {
- healCrsRelay('inference-false');
- }
- const healthy = healthOk && (!CRS_ENABLE_INFERENCE_SMOKE || inferenceOk);
- const st = loadState();
- if (healthy) {
- st.up += 1;
- st.down = 0;
- } else {
- st.down += 1;
- st.up = 0;
- }
- if (CRS_ENABLE_INFERENCE_SMOKE && !inferenceOk && healthOk) {
- st.inferenceDown = (st.inferenceDown || 0) + 1;
- } else if (CRS_ENABLE_INFERENCE_SMOKE && inferenceOk) {
- st.inferenceDown = 0;
- }
- const inFallback = markerPresent();
-
- let acted = false;
- if (!healthy && st.down >= DOWN_STRIKES) {
- // Creating the marker is what claims the switch, so nothing here acts on the
- // reading taken above: 'wx' creates it in one step, and EEXIST means another
- // watcher already went fail-closed — the outcome we wanted anyway.
- let claimed = true;
- try {
- writeFileSync(MARKER, String(Date.now()), { flag: 'wx', mode: 0o600 });
- } catch (err) {
- if (err.code !== 'EEXIST') throw err;
- claimed = false;
- }
- if (claimed) {
- if (setEnvMode('direct')) {
- st.mode = 'fail-closed';
- acted = true;
- log(
- `CRS DOWN x${st.down} (>=${DOWN_STRIKES}) → FAIL-CLOSED: stripped provider env from settings.json; new/respawned sessions must not silently use direct or Bedrock`,
- );
- } else {
- // The env rewrite refused, so give the claim back rather than leave a
- // marker claiming we are fail-closed while settings.json still says CRS.
- try {
- execFileSync('rm', ['-f', MARKER]);
- } catch {}
- }
- }
- } else if (healthy && st.up >= UP_STRIKES && inFallback) {
- if (setEnvMode('crs')) {
- try {
- execFileSync('rm', ['-f', MARKER]);
- } catch {}
- st.mode = 'crs';
- acted = true;
- log(
- `CRS UP x${st.up} (>=${UP_STRIKES}) → RESTORE: re-added CRS env to settings.json; new/respawned sessions route via relay again`,
- );
- }
- }
- if (!acted) {
- log(
- `tick: health=${healthOk} inference=${CRS_ENABLE_INFERENCE_SMOKE ? inferenceOk : 'disabled'} down=${st.down} up=${st.up} fallback=${inFallback} envMode=${currentEnvMode()}`,
- );
- }
-
- // OS-wedge auto-reboot (double-confirmed via SSM, 30m cooldown). Only fires when CRS
- // is sustained-down past REBOOT_STRIKES, we're already fail-closed, and the SSM agent
- // itself is unreachable (true OS wedge, not an app-level CRS failure).
- maybeAutoReboot(st, inFallback, healthy);
- saveState(st);
-}
-
-main();
diff --git a/claude-ops/scripts/account-rotation/crs-priority-daemon.mjs b/claude-ops/scripts/account-rotation/crs-priority-daemon.mjs
deleted file mode 100755
index 34949ff9..00000000
--- a/claude-ops/scripts/account-rotation/crs-priority-daemon.mjs
+++ /dev/null
@@ -1,901 +0,0 @@
-#!/usr/bin/env node
-// crs-priority-daemon.mjs — utilization-driven account prioritization for a
-// claude-relay-service (CRS) pool.
-//
-// CRS load-balances Claude requests across many claude.ai accounts at once. Each
-// account carries a `schedulable` flag; CRS only routes to schedulable accounts.
-// This daemon flips that flag from LIVE signals so the pool AVOIDS near-maxed
-// accounts (cutting 429/529s) and RE-ENABLES them once their windows recover —
-// the relay-pool analogue of the keychain rotator's "rotate to the coolest
-// account". One tick per invocation; a launchd/systemd timer drives the cadence
-// (see scripts/install-crs-priority-agent.sh). The wrapper crs-priority-daemon.sh
-// holds the single-flight lock.
-//
-// SIGNALS (real-time first — claudeUsage.* is often null/stale for pool accounts
-// and must NEVER drive a re-enable on its own):
-// rateLimitStatus.isRateLimited / opusRateLimitStatus.isRateLimited (hard cap hit)
-// overloadStatus.isOverloaded (upstream 529)
-// sessionWindow.sessionWindowStatus (allowed | allowed_warning | …)
-// claudeUsage.{fiveHour,sevenDay,sevenDayOpus}.utilization (fresh secondary only)
-//
-// LIVE QUOTA POLLER (60-120s): hybrid header-derived (CRS rateLimit* from relay
-// upstream headers) + targeted /oauth/usage probe per account (throttled, cached).
-// Clears stale cooldowns on rateLimitResetAt expiry. Feeds live headroom back
-// to CRS scheduler via account PUT for accurate LB/concurrency.
-//
-// POLICY (configurable via crs.policy or $CRS_POLICY):
-// conservative (default) — deprioritize on fresh high utilization, sessionWindow
-// warnings, rate limits, and overload; FLOOR keeps minimum pool size; dedup by
-// organizationUuid (same claude.ai quota pool).
-// max-out — schedulable=TRUE for every account with a fresh OAuth token unless
-// CRS would get a hard error (genuine rate-limit or 529 after stale-cache scrub).
-// Util % and sessionWindow are telemetry only; dedup by login email.
-//
-// CONFIG (config.json "crs" block; every key overridable by env):
-// enabled, policy, baseUrl, adminUser, adminPasswordEnv, off5h, off7d, on5h, on7d,
-// floor, freshMinutes, weeklyReconcile. Per-account crsAccountName (or
-// crs.nameByVaultKey) maps vault keys to CRS admin account names for token-feed +
-// live quota lookup. Admin password resolves from (1) $CRS_ADMIN_PASSWORD, (2) the
-// configured env var, (3) credential-store.
-//
-// WEEKLY-CAP RECONCILIATION (crs.weeklyReconcile, default on; $CRS_WEEKLY_RECONCILE=0
-// to disable): some CRS deployments persist a weeklyRateLimitEndAt timestamp on an
-// account once a genuine 7-day cap is hit, and hold schedulable=false until it
-// passes. That timestamp can go stale (wrong upstream reset, clock skew, a reset
-// that already happened) and park a perfectly usable account. Each tick this daemon
-// reconciles the persisted hold against live utilization: if the account's real 7d
-// utilization has dropped back under `on7d`, the hold is cleared even though the
-// timestamp hasn't passed yet. It also records a hold when live utilization proves a
-// genuine breach of `off7d` and the source data included an authoritative reset time.
-// Entirely a no-op on accounts/deployments that never set weeklyRateLimitEndAt.
-//
-// FLOOR (crs.floor / $CRS_FLOOR, default 2): minimum number of accounts the
-// conservative policy will keep schedulable even if that means reverting some
-// soft (utilization/sessionWindow) deprioritizations. Tune this to your own pool
-// size — the default is a conservative floor for a small pool, not a target.
-//
-// BACKEND (crs.backend / $CRS_PRIORITY_BACKEND): crs (default) | local (file seat-state).
-// CLI: (none)=one live tick · --dry-run=log decisions, no writes · --status=print
-// current schedulable + utilization table and exit · --once (alias for tick).
-
-import { execFileSync } from 'child_process';
-import { readFileSync, writeFileSync, mkdirSync, renameSync, existsSync } from 'fs';
-import { dirname, join } from 'path';
-import { fileURLToPath } from 'url';
-import { homedir, platform } from 'os';
-import {
- buildCrsNameMaps,
- crsBaseUrl,
- crsFileVaultPath,
- crsPolicy,
- loadRotationConfig,
- vaultLookupKeysForEmail,
-} from './crs-pool-config.mjs';
-import { resolveAccountsBackend, dualWriteEnabled, mergeSeatsIntoState } from './ops-accounts-backend.mjs';
-
-const __dirname = dirname(fileURLToPath(import.meta.url));
-const PLUGIN_ROOT = process.env.CLAUDE_PLUGIN_ROOT || join(__dirname, '..', '..');
-const CRED_STORE_CANDIDATES = [
- process.env.OPS_CREDENTIAL_STORE,
- join(homedir(), '.claude', 'plugins', 'cache', 'ops-marketplace', 'ops', 'current', 'lib', 'credential-store.sh'),
- join(homedir(), '.claude', 'plugins', 'marketplaces', 'ops-marketplace', 'claude-ops', 'lib', 'credential-store.sh'),
- join(PLUGIN_ROOT, 'lib', 'credential-store.sh'),
-].filter(Boolean);
-function credentialStorePath() {
- for (const p of CRED_STORE_CANDIDATES) {
- if (existsSync(p)) return p;
- }
- return CRED_STORE_CANDIDATES[CRED_STORE_CANDIDATES.length - 1];
-}
-const EXEC_QUIET = { encoding: 'utf8', stdio: ['pipe', 'pipe', 'ignore'] };
-const DATA_DIR =
- process.env.CLAUDE_PLUGIN_DATA_DIR || join(homedir(), '.claude', 'plugins', 'data', 'ops-ops-marketplace');
-// Dual-mode: backend=crs | local | auto (OPS_ACCOUNTS_BACKEND)
-const SEAT_STATE_PATH = process.env.CRS_SEAT_STATE_PATH || join(DATA_DIR, 'seat-state', 'claude-priority.json');
-const OPS_SEAT_STATE_PATH =
- process.env.OPS_ACCOUNTS_STATE_PATH || join(DATA_DIR, 'account-rotation', 'seat-state.json');
-
-const args = new Set(process.argv.slice(2));
-const DRY = args.has('--dry-run') || process.env.CRS_DRY === '1';
-const STATUS = args.has('--status');
-
-const ts = () => new Date().toISOString().slice(11, 19);
-const log = (m) => console.log(`${ts()} [crs-priority] ${m}`);
-
-// ── config ───────────────────────────────────────────────────────────────────
-const cfg = loadRotationConfig();
-const C = cfg.crs || {};
-const BACKEND_WANT = resolveAccountsBackend({
- env: process.env,
- cfgBackend: process.env.CRS_PRIORITY_BACKEND || C.backend || 'auto',
-});
-// local|file → local tick; crs → CRS only; auto → CRS with local fallback
-const BACKEND = BACKEND_WANT === 'local' ? 'local' : BACKEND_WANT === 'crs' ? 'crs' : 'auto';
-
-const { vaultKeyByCrsName } = buildCrsNameMaps(cfg);
-const num = (v, d) => (v === undefined || v === null || v === '' || Number.isNaN(+v) ? d : +v);
-
-const BASE = crsBaseUrl(cfg);
-const ADMIN_USER = process.env.CRS_ADMIN_USER || C.adminUser || 'cradmin';
-const OFF_5H = num(process.env.CRS_OFF_5H ?? C.off5h, 90);
-const OFF_7D = num(process.env.CRS_OFF_7D ?? C.off7d, 95);
-const ON_5H = num(process.env.CRS_ON_5H ?? C.on5h, 70);
-const ON_7D = num(process.env.CRS_ON_7D ?? C.on7d, 85);
-// Minimum-schedulable-accounts safety margin. Default is intentionally small and
-// generic — set crs.floor (or $CRS_FLOOR) to whatever fraction makes sense for your
-// own pool size; do not assume this default fits a large fleet.
-const FLOOR = num(process.env.CRS_FLOOR ?? C.floor, 2);
-const FRESH_MIN = num(process.env.CRS_FRESH_MIN ?? C.freshMinutes, 15);
-const CRS_POLICY = crsPolicy(cfg);
-const TOKEN_MIN_FRESH_MS = num(process.env.CRS_TOKEN_MIN_FRESH_MS ?? C.tokenMinFreshMs, 5 * 60_000);
-const STALE_RL_MAX_MINS = num(process.env.CRS_STALE_RL_MINUTES, 300);
-const LIVE_USAGE_TTL_MS = num(process.env.CRS_LIVE_USAGE_TTL_MS, 90_000);
-// Opt-in-by-default, safe-no-op guard: only touches accounts that already carry a
-// weeklyRateLimitEndAt field, so a fresh install with no such field never sees this
-// reconciler do anything. Set crs.weeklyReconcile=false / $CRS_WEEKLY_RECONCILE=0
-// to disable outright.
-const WEEKLY_RECONCILE = process.env.CRS_WEEKLY_RECONCILE === '0' ? false : C.weeklyReconcile !== false;
-const IS_LINUX = platform() === 'linux';
-const liveUsageCache = new Map();
-const LINUX_CRED_PATH = crsFileVaultPath(cfg);
-const CRS_CONTAINER = process.env.CRS_CONTAINER || C.containerName || 'crs-claude-relay-1';
-
-function adminPasswordFromDocker() {
- if (!IS_LINUX) return null;
- try {
- const out = execFileSync(
- 'docker',
- ['inspect', CRS_CONTAINER, '--format', '{{range .Config.Env}}{{println .}}{{end}}'],
- EXEC_QUIET,
- );
- const m = out.match(/^ADMIN_PASSWORD=(.+)$/m);
- return m?.[1]?.trim() || null;
- } catch {
- return null;
- }
-}
-
-function adminPassword() {
- if (process.env.CRS_ADMIN_PASSWORD) return process.env.CRS_ADMIN_PASSWORD;
- const envName = C.adminPasswordEnv || process.env.CRS_ADMIN_PASSWORD_ENV;
- if (envName && process.env[envName]) return process.env[envName];
- const dockerPw = adminPasswordFromDocker();
- if (dockerPw) return dockerPw;
- try {
- const cred = credentialStorePath();
- const acct = process.env.CLAUDE_ROTATOR_KEYCHAIN_ACCOUNT || process.env.USER || 'claude-ops';
- return execFileSync('bash', [cred, 'get', `CRS-Admin-${ADMIN_USER}`, acct], EXEC_QUIET).trim();
- } catch {}
- throw new Error(
- `CRS admin password unavailable — set $CRS_ADMIN_PASSWORD, or store it via:\n` +
- ` bash "${credentialStorePath()}" set CRS-Admin-${ADMIN_USER} "$USER" '<password>'`,
- );
-}
-
-// ── live per-account quota poller (hybrid header + targeted probe) ──────────────
-// 60-120s friendly: CRS /admin accounts gives header-derived rateLimitStatus etc;
-// we do targeted read-only /oauth/usage probe (via stored OAuth) for u5/u7 headroom.
-// Hybrid: body + response headers. TTL + throttle. Falls back to cache on error.
-// Never drives re-enable from stale; feeds headroom to scheduler after collection.
-const USAGE_TOKEN_SVC = process.env.CRS_USAGE_TOKEN_PREFIX || 'Claude-Rotation-';
-
-function accountVaultKey(a) {
- return a.subscriptionInfo?.email || vaultKeyByCrsName[a.name] || null;
-}
-
-function vaultLookupKeys(email) {
- return vaultLookupKeysForEmail(email, cfg.accounts || []);
-}
-
-function readOauthTokenFromFileVault(email) {
- if (!email || !existsSync(LINUX_CRED_PATH)) return null;
- try {
- const store = JSON.parse(readFileSync(LINUX_CRED_PATH, 'utf8'));
- for (const key of vaultLookupKeys(email)) {
- const t = store[`${USAGE_TOKEN_SVC}${key}`]?.claudeAiOauth?.accessToken;
- if (t) return t;
- }
- } catch {}
- return null;
-}
-
-function readOauthToken(email) {
- if (!email) return null;
- const fileTok = readOauthTokenFromFileVault(email);
- if (fileTok) return fileTok;
- const acct = process.env.CLAUDE_ROTATOR_KEYCHAIN_ACCOUNT || process.env.USER || 'claude-ops';
- const cred = credentialStorePath();
- for (const get of [
- () => execFileSync('bash', [cred, 'get', `${USAGE_TOKEN_SVC}${email}`, acct], EXEC_QUIET),
- () =>
- execFileSync(
- 'security',
- ['find-generic-password', '-a', acct, '-s', `${USAGE_TOKEN_SVC}${email}`, '-w'],
- EXEC_QUIET,
- ),
- ]) {
- try {
- const j = JSON.parse(get().trim());
- const t = j?.claudeAiOauth?.accessToken;
- if (t) return t;
- } catch {}
- }
- for (const key of vaultLookupKeys(email)) {
- if (key === email) continue;
- for (const get of [
- () => execFileSync('bash', [cred, 'get', `${USAGE_TOKEN_SVC}${key}`, acct], EXEC_QUIET),
- () =>
- execFileSync(
- 'security',
- ['find-generic-password', '-a', acct, '-s', `${USAGE_TOKEN_SVC}${key}`, '-w'],
- EXEC_QUIET,
- ),
- ]) {
- try {
- const j = JSON.parse(get().trim());
- const t = j?.claudeAiOauth?.accessToken;
- if (t) return t;
- } catch {}
- }
- }
- return null;
-}
-
-function accountTokenFresh(a, now = Date.now()) {
- const tokenExp = Number(a.tokenExpiresAt || a.expiresAt || 0);
- return Number.isFinite(tokenExp) && tokenExp > now + TOKEN_MIN_FRESH_MS;
-}
-
-function rateLimitLooksStale(a, now = Date.now()) {
- if (a.status === 'error' && accountTokenFresh(a, now)) return true;
-
- const inspect = (st, resetAtFallback) => {
- if (!st?.isRateLimited) return false;
- const resetAt = st.resetAt || resetAtFallback;
- const resetMs = resetAt ? Date.parse(resetAt) : NaN;
- const mins = st.minutesRemaining ?? null;
- if (Number.isFinite(resetMs) && resetMs <= now) return true;
- if (mins === 0) return true;
- if (typeof mins === 'number' && mins > STALE_RL_MAX_MINS) return true;
- if (accountTokenFresh(a, now) && typeof mins === 'number' && mins > 60) return true;
- return false;
- };
-
- // Explicit: clear stale cooldowns when rateLimitResetAt (or embedded resetAt) has passed
- if (resetAtPassed(a.rateLimitStatus, a.rateLimitResetAt, now)) return true;
- if (resetAtPassed(a.opusRateLimitStatus, a.opusRateLimitStatus?.resetAt, now)) return true;
-
- if (inspect(a.rateLimitStatus, a.rateLimitResetAt)) return true;
- if (inspect(a.opusRateLimitStatus, a.opusRateLimitStatus?.resetAt)) return true;
-
- const ov = a.overloadStatus;
- if (ov?.isOverloaded) {
- const resetMs = ov.resetAt ? Date.parse(ov.resetAt) : NaN;
- if (Number.isFinite(resetMs) && resetMs <= now) return true;
- if (accountTokenFresh(a, now)) return true;
- }
- return false;
-}
-
-function resetAtPassed(st, fallback, now = Date.now()) {
- const resetAt = st?.resetAt || fallback;
- if (!resetAt) return false;
- const ms = Date.parse(resetAt);
- return Number.isFinite(ms) && ms <= now;
-}
-async function liveUsage(email) {
- if (!email) return null;
- const cached = liveUsageCache.get(email);
- if (cached && Date.now() - cached.at < LIVE_USAGE_TTL_MS) return cached.data;
- const token = readOauthToken(email);
- if (!token) return null;
- try {
- const r = await fetch('https://api.anthropic.com/api/oauth/usage', {
- method: 'GET',
- headers: { Authorization: `Bearer ${token}`, 'anthropic-beta': 'oauth-2025-04-20' },
- signal: AbortSignal.timeout(5000),
- });
- if (!r.ok) return cached?.data ?? null;
- // Hybrid: body (utilization) + headers (rate reset signals when present)
- const headers = {};
- r.headers.forEach((v, k) => {
- headers[k.toLowerCase()] = v;
- });
- const d = await r.json();
- const data = {
- u5: d?.five_hour?.utilization ?? null,
- u7: d?.seven_day?.utilization ?? null,
- resets5: d?.five_hour?.resets_at ?? null,
- // ISO string when the API supplies it — used by the weekly-cap reconciler
- // below to record/clear an authoritative reset time.
- resets7: d?.seven_day?.resets_at ?? null,
- // capture any reset header for hybrid RL detection if body alone insufficient
- resetHeader: headers['retry-after'] || headers['anthropic-ratelimit-unified-tokens-reset'] || null,
- };
- liveUsageCache.set(email, { at: Date.now(), data });
- return data;
- } catch {
- return cached?.data ?? null;
- }
-}
-
-function authenticationUnavailable(a) {
- const status = String(a.status || '').toLowerCase();
- const detail = [a.lastError, a.error, a.errorMessage, a.statusMessage, a.statusReason].filter(Boolean).join(' ');
- return (
- status === 'auth_repair' ||
- /\b(401|403)\b|authentication[_ -]?error|invalid (api )?key|invalid.*token|revoked/i.test(detail)
- );
-}
-
-function genuineRateLimit(a, now = Date.now()) {
- const flagged = !!a.rateLimitStatus?.isRateLimited || !!a.opusRateLimitStatus?.isRateLimited;
- return flagged && !rateLimitLooksStale(a, now);
-}
-
-function liveQuotaExhaustion(a, now = Date.now()) {
- const usage = a._liveUsage;
- if (!usage) return null;
- const windows = [
- { name: '5h', utilization: usage.u5, resetsAt: usage.resets5 },
- { name: '7d', utilization: usage.u7, resetsAt: usage.resets7 },
- ].filter(({ utilization, resetsAt }) => {
- const resetMs = resetsAt ? Date.parse(resetsAt) : NaN;
- return typeof utilization === 'number' && utilization >= 100 && Number.isFinite(resetMs) && resetMs > now;
- });
- if (!windows.length) return null;
- const resetAt = new Date(Math.min(...windows.map(({ resetsAt }) => Date.parse(resetsAt)))).toISOString();
- return { windows: windows.map(({ name }) => name), resetAt };
-}
-
-function genuineOverload(a, now = Date.now()) {
- if (!a.overloadStatus?.isOverloaded) return false;
- const resetMs = a.overloadStatus.resetAt ? Date.parse(a.overloadStatus.resetAt) : NaN;
- if (Number.isFinite(resetMs) && resetMs <= now) return false;
- if (accountTokenFresh(a, now)) {
- const mins = a.overloadStatus.minutesRemaining;
- if (typeof mins === 'number' && mins > STALE_RL_MAX_MINS) return false;
- }
- return true;
-}
-
-// A persisted weekly-cap hold (see WEEKLY-CAP RECONCILIATION above). Unlike the
-// soft utilization/sessionWindow signals, this is treated as a hard hold — the
-// FLOOR safety margin never reverts it, since it represents a genuine 7-day
-// exhaustion, not a heuristic guess.
-function weeklyCapActive(a, now = Date.now()) {
- const endMs = a.weeklyRateLimitEndAt ? Date.parse(a.weeklyRateLimitEndAt) : NaN;
- return Number.isFinite(endMs) && endMs > now;
-}
-
-// ── http ─────────────────────────────────────────────────────────────────────
-async function jfetch(path, opts = {}) {
- const r = await fetch(`${BASE}${path}`, opts);
- const txt = await r.text();
- let body;
- try {
- body = JSON.parse(txt);
- } catch {
- body = txt;
- }
- return { status: r.status, body };
-}
-
-async function login() {
- const r = await jfetch('/web/auth/login', {
- method: 'POST',
- headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ username: ADMIN_USER, password: adminPassword() }),
- });
- if (r.status !== 200 || !r.body?.token) throw new Error(`login failed (HTTP ${r.status})`);
- return { Authorization: `Bearer ${r.body.token}`, 'Content-Type': 'application/json' };
-}
-
-async function getAccounts(auth) {
- const r = await jfetch('/admin/claude-accounts', { headers: auth });
- if (r.status !== 200) throw new Error(`GET claude-accounts failed (HTTP ${r.status})`);
- const list = Array.isArray(r.body) ? r.body : r.body.data || r.body.accounts || [];
- return list.filter((a) => a.platform === 'claude' && a.isActive !== false);
-}
-
-async function clearStaleCooldowns(auth, accts) {
- const now = Date.now();
- let cleared = 0;
- for (const a of accts) {
- if (!rateLimitLooksStale(a, now)) continue;
- const mins = a.rateLimitStatus?.minutesRemaining ?? '?';
- const resetPassed =
- resetAtPassed(a.rateLimitStatus, a.rateLimitResetAt, now) ||
- resetAtPassed(a.opusRateLimitStatus, a.opusRateLimitStatus?.resetAt, now);
- if (DRY) {
- log(`[dry] clear stale RL ${a.name} (mins=${mins}${resetPassed ? ', resetAt passed' : ''})`);
- cleared++;
- continue;
- }
- const res = await jfetch(`/admin/claude-accounts/${a.id}/reset-status`, { method: 'POST', headers: auth });
- if (res.status >= 200 && res.status < 300) {
- cleared++;
- log(
- `${a.name}: cleared stale RL/error cache (was mins=${mins}${resetPassed ? ' — rateLimitResetAt passed' : ''})`,
- );
- }
- }
- return cleared;
-}
-
-async function recoverSchedulableAfterClear(auth, accts) {
- const now = Date.now();
- let enabled = 0;
- for (const a of accts) {
- if (a.schedulable !== false) continue;
- if (a.rateLimitStatus?.isRateLimited || a.opusRateLimitStatus?.isRateLimited) continue;
- if (a.overloadStatus?.isOverloaded) continue;
- if (!accountTokenFresh(a, now)) continue;
- if (DRY) {
- log(`[dry] re-enable ${a.name} after stale clear`);
- enabled++;
- continue;
- }
- const put = await jfetch(`/admin/claude-accounts/${a.id}/toggle-schedulable`, {
- method: 'PUT',
- headers: auth,
- body: JSON.stringify({ schedulable: true }),
- });
- if (put.status >= 200 && put.status < 300) {
- enabled++;
- log(`${a.name}: re-enabled schedulable after stale-cache recovery`);
- }
- }
- return enabled;
-}
-
-// Feed live headroom (from targeted quota probe) back into CRS account record.
-// This supplies the scheduler with fresh per-account utilization/headroom so
-// load-balancing, concurrency caps, and selection use authoritative 5h/7d numbers
-// instead of stale claudeUsage cache.
-async function feedLiveHeadroom(auth, a, lu) {
- if (!lu || DRY) return false;
- const update = {
- claudeUsage: {
- fiveHour: { utilization: lu.u5 ?? null, updatedAt: new Date().toISOString() },
- sevenDay: { utilization: lu.u7 ?? null, updatedAt: new Date().toISOString() },
- },
- };
- try {
- const r = await jfetch(`/admin/claude-accounts/${a.id}`, {
- method: 'PUT',
- headers: auth,
- body: JSON.stringify(update),
- });
- return r.status >= 200 && r.status < 300;
- } catch {
- return false;
- }
-}
-
-// ── policy ───────────────────────────────────────────────────────────────────
-const WARN_STATUSES = new Set(['allowed_warning', 'warning', 'blocked', 'exceeded', 'limited', 'stopped']);
-
-function decideMaxOut(accts, nowMs) {
- return accts.map((a) => {
- const cu = a.claudeUsage || {};
- const lu = a._liveUsage;
- const u5 = lu?.u5 ?? cu.fiveHour?.utilization;
- const u7 = lu?.u7 ?? cu.sevenDay?.utilization;
- const sw = a.sessionWindow?.sessionWindowStatus || null;
- const cur = a.schedulable !== false;
- const unavailable = authenticationUnavailable(a);
- const quota = liveQuotaExhaustion(a, nowMs);
- const rl = Boolean(quota);
- const overloaded = genuineOverload(a, nowMs);
-
- let desired = true;
- let reason = `max-out (5h=${u5 ?? '?'} 7d=${u7 ?? '?'} sw=${sw ?? 'none'})`;
- if (unavailable) {
- desired = false;
- reason = 'authentication unavailable; staged re-auth required';
- } else if (quota) {
- desired = false;
- reason = `quota exhausted (${quota.windows.join('+')}); auto-reschedule ${quota.resetAt}`;
- } else if (overloaded) {
- reason = 'overloaded(529), staying schedulable';
- }
- return {
- a,
- cur,
- desired,
- reason,
- soft: false,
- sw,
- u5: u5 ?? '?',
- u7: u7 ?? '?',
- unavailable,
- rl,
- overloaded,
- quota,
- };
- });
-}
-
-function decideConservative(accts, nowMs) {
- const decisions = accts.map((a) => {
- const cu = a.claudeUsage || {};
- const lu = a._liveUsage;
- const u5 = lu?.u5 ?? cu.fiveHour?.utilization;
- const u7 = lu?.u7 ?? cu.sevenDay?.utilization;
- const sw = a.sessionWindow?.sessionWindowStatus || null;
- const cur = a.schedulable !== false;
- const unavailable = authenticationUnavailable(a);
- const quota = liveQuotaExhaustion(a, nowMs);
- const rl = Boolean(quota);
- const overloaded = genuineOverload(a, nowMs);
-
- let desired = true;
- let reason = 'scheduled unless live quota is exhausted';
- let soft = false;
- if (unavailable) {
- desired = false;
- reason = 'authentication unavailable; staged re-auth required';
- } else if (quota) {
- desired = false;
- reason = `quota exhausted (${quota.windows.join('+')}); auto-reschedule ${quota.resetAt}`;
- } else if (overloaded) {
- reason = 'overloaded(529), staying schedulable';
- }
- return { a, cur, desired, reason, soft, sw, u5: u5 ?? '?', u7: u7 ?? '?', unavailable, rl, overloaded, quota };
- });
-
- let usable = decisions.filter((d) => d.desired && !d.unavailable && !d.rl && !d.overloaded).length;
- if (usable < FLOOR) {
- const unum = (d) => (typeof d.u5 === 'number' ? d.u5 : 50);
- const revertable = decisions
- .filter((d) => d.desired === false && d.soft && !d.unavailable && !d.rl && !d.overloaded)
- .sort((x, y) => unum(x) - unum(y));
- for (const d of revertable) {
- if (usable >= FLOOR) break;
- d.desired = true;
- d.reason = `FLOOR(${FLOOR}): held schedulable despite ${d.reason}`;
- usable++;
- }
- const final = decisions.filter((d) => d.desired && !d.unavailable && !d.rl && !d.overloaded).length;
- if (final < FLOOR)
- log(`WARNING: only ${final} usable account(s) (< floor ${FLOOR}) — pool is capacity-constrained`);
- }
-
- const byUuid = {};
- for (const d of decisions) {
- const uuid = d.a.subscriptionInfo?.organizationUuid;
- if (!uuid || !d.desired) continue;
- (byUuid[uuid] ||= []).push(d);
- }
- for (const group of Object.values(byUuid)) {
- if (group.length < 2) continue;
- const unum = (d) => (typeof d.u5 === 'number' ? d.u5 : 50);
- group.sort((x, y) => (x.sw === 'allowed' ? 0 : 1) - (y.sw === 'allowed' ? 0 : 1) || unum(x) - unum(y));
- for (const d of group.slice(1)) {
- d.desired = false;
- d.reason = `dedup: same org ${group[0].a.subscriptionInfo.organizationUuid.slice(0, 8)} as ${group[0].a.name}`;
- }
- }
- return decisions;
-}
-
-function dedupByLogin(decisions) {
- const byLogin = {};
- for (const d of decisions) {
- const login = accountVaultKey(d.a);
- if (!login || !d.desired) continue;
- (byLogin[login] ||= []).push(d);
- }
- for (const [login, group] of Object.entries(byLogin)) {
- if (group.length < 2) continue;
- const unum = (d) => (typeof d.u5 === 'number' ? d.u5 : 100);
- group.sort((x, y) => unum(x) - unum(y));
- for (const d of group.slice(1)) {
- d.desired = false;
- d.reason = `dedup: same login ${login} as ${group[0].a.name}`;
- }
- }
- return decisions;
-}
-
-function decide(accts) {
- const nowMs = Date.now();
- if (CRS_POLICY === 'max-out' || CRS_POLICY === 'maxout') {
- return dedupByLogin(decideMaxOut(accts, nowMs));
- }
- return decideConservative(accts, nowMs);
-}
-
-// ── local seat-state backend (backend=local) ─────────────────────────────────
-function loadSeatStateFile() {
- try {
- if (!existsSync(SEAT_STATE_PATH)) return { seats: {}, updatedAt: null };
- return JSON.parse(readFileSync(SEAT_STATE_PATH, 'utf8'));
- } catch {
- return { seats: {}, updatedAt: null };
- }
-}
-
-function writeSeatStateFile(payload) {
- mkdirSync(dirname(SEAT_STATE_PATH), { recursive: true });
- writeFileSync(SEAT_STATE_PATH, JSON.stringify(payload, null, 2) + '\n', { mode: 0o600 });
-}
-
-/** Build synthetic CRS-shaped accounts from rotation config vault keys. */
-function loadLocalAccounts() {
- const prev = loadSeatStateFile();
- const accounts = Array.isArray(cfg.accounts) ? cfg.accounts : [];
- const out = [];
- for (const a of accounts) {
- if (!a || typeof a !== 'object') continue;
- if (a.disabled || a.autoAuthDisabled) continue;
- const email = a.email || a.label;
- if (!email || typeof email !== 'string') continue;
- const key = email.trim().toLowerCase();
- const tok = readOauthToken(key);
- const prevSeat = prev.seats?.[key] || {};
- let expiresAt = 0;
- if (tok?.expiresAt) expiresAt = Date.parse(tok.expiresAt);
- else if (tok?.expiry_date) expiresAt = Number(tok.expiry_date);
- else if (tok?.expires_at) expiresAt = Date.parse(tok.expires_at);
- else expiresAt = Number(prevSeat.tokenExpiresAt || 0);
- out.push({
- id: key,
- name: key,
- email: key,
- schedulable: prevSeat.schedulable !== false,
- tokenExpiresAt: Number.isFinite(expiresAt) ? expiresAt : 0,
- claudeUsage: prevSeat.claudeUsage || null,
- sessionWindow: prevSeat.sessionWindow || null,
- rateLimitStatus: prevSeat.rateLimitStatus || null,
- overloadStatus: prevSeat.overloadStatus || null,
- weeklyRateLimitEndAt: prevSeat.weeklyRateLimitEndAt || null,
- subscriptionInfo: prevSeat.subscriptionInfo || null,
- _local: true,
- });
- }
- return out;
-}
-
-function persistLocalDecisions(decisions) {
- const seats = {};
- for (const d of decisions) {
- const key = accountVaultKey(d.a);
- seats[key] = {
- schedulable: d.desired,
- reason: d.reason,
- u5: d.u5,
- sw: d.sw,
- tokenExpiresAt: d.a.tokenExpiresAt || null,
- claudeUsage: d.a._liveUsage
- ? {
- fiveHour: { utilization: d.a._liveUsage.u5 ?? null, updatedAt: new Date().toISOString() },
- sevenDay: { utilization: d.a._liveUsage.u7 ?? null, updatedAt: new Date().toISOString() },
- updatedAt: new Date().toISOString(),
- }
- : d.a.claudeUsage || null,
- updatedAt: new Date().toISOString(),
- };
- }
- const payload = {
- backend: 'local',
- updatedAt: new Date().toISOString(),
- seatStatePath: SEAT_STATE_PATH,
- seats,
- };
- if (!DRY) writeSeatStateFile(payload);
- return payload;
-}
-
-async function runLocalTick() {
- log(`backend=local seat-state=${SEAT_STATE_PATH}`);
- let accts = loadLocalAccounts();
- if (!accts.length) {
- log('no local accounts (config.accounts empty or all disabled)');
- return;
- }
- let liveN = 0;
- for (const a of accts) {
- const key = accountVaultKey(a);
- a._liveUsage = await liveUsage(key);
- if (a._liveUsage) liveN++;
- await new Promise((r) => setTimeout(r, 150));
- }
- const decisions = decide(accts);
- if (STATUS) {
- console.log(
- `LOCAL seats @ ${SEAT_STATE_PATH} — ${decisions.filter((d) => d.cur).length}/${decisions.length} schedulable`,
- );
- for (const d of decisions.sort((a, b) => (a.cur === b.cur ? 0 : a.cur ? -1 : 1))) {
- const flags = [d.rl && 'RL', d.overloaded && 'OVERLOAD', d.sw].filter(Boolean).join(' ');
- console.log(
- ` ${d.cur ? '●' : '○'} ${d.a.name.padEnd(26)} sched=${d.cur} 5h=${String(d.u5).padStart(3)}% ${flags}`,
- );
- }
- return;
- }
- let changed = 0;
- for (const d of decisions) {
- if (d.desired === d.cur) continue;
- changed++;
- if (DRY) log(`[dry] ${d.a.name}: ${d.cur}→${d.desired} (${d.reason})`);
- else log(`${d.a.name}: schedulable ${d.cur}→${d.desired} (${d.reason}) [local]`);
- }
- persistLocalDecisions(decisions);
- const on = decisions.filter((d) => d.desired).map((d) => d.a.name);
- const off = decisions.filter((d) => !d.desired).map((d) => d.a.name);
- log(
- `tick(local): ${changed} change(s). live-quota=${liveN}/${decisions.length} schedulable=${on.length} [${on.join(',')}] | off=[${off.join(',')}] wrote=${!DRY}`,
- );
-}
-
-/** Mirror CRS decisions into provider-shaped seat-state (ops-accounts dual-write). */
-function dualWriteFromCrsDecisions(decisions) {
- if (!dualWriteEnabled()) return;
- if (DRY) {
- log(`[dry] dual-write ${decisions.length} decision(s) → ${OPS_SEAT_STATE_PATH}`);
- return;
- }
- try {
- let state = { version: 1, providers: {} };
- if (existsSync(OPS_SEAT_STATE_PATH)) {
- try {
- state = JSON.parse(readFileSync(OPS_SEAT_STATE_PATH, 'utf8'));
- } catch {
- /* keep empty */
- }
- }
- const seats = decisions.map((d) => {
- const email = d.a?.subscriptionInfo?.email || vaultKeyByCrsName[d.a?.name] || d.a?.name || null;
- const u5 = d.u5;
- const u7 = d.a?._liveUsage?.u7;
- return {
- email,
- schedulable: d.desired !== false,
- util5h: typeof u5 === 'number' ? u5 : null,
- util7d: typeof u7 === 'number' ? u7 : null,
- };
- });
- const next = mergeSeatsIntoState(state, seats, { provider: 'claude' });
- mkdirSync(dirname(OPS_SEAT_STATE_PATH), { recursive: true });
- const tmp = OPS_SEAT_STATE_PATH + '.tmp';
- writeFileSync(tmp, JSON.stringify(next, null, 2) + '\n', { mode: 0o600 });
- renameSync(tmp, OPS_SEAT_STATE_PATH);
- log(`dual-write: mirrored ${seats.filter((s) => s.email).length} seat(s) → ${OPS_SEAT_STATE_PATH}`);
- } catch (e) {
- log(`dual-write skipped: ${e.message}`);
- }
-}
-
-// ── main ─────────────────────────────────────────────────────────────────────
-async function main() {
- log(`backend want=${BACKEND_WANT}`);
- if (BACKEND_WANT === 'local' || BACKEND === 'local' || BACKEND === 'file') {
- await runLocalTick();
- return;
- }
- try {
- if (C.enabled === false && !STATUS && !DRY) {
- log('crs.enabled=false — skipping tick');
- return;
- }
- const auth = await login();
- let accts = await getAccounts(auth);
- if (!accts.length) {
- log('no active claude accounts');
- return;
- }
- const cleared = await clearStaleCooldowns(auth, accts);
- if (cleared) {
- accts = await getAccounts(auth);
- log(`stale-cooldown: cleared ${cleared} account(s), reloaded pool`);
- const reenabled = await recoverSchedulableAfterClear(auth, accts);
- if (reenabled) {
- accts = await getAccounts(auth);
- log(`stale-cooldown: re-enabled ${reenabled} schedulable account(s)`);
- }
- }
- // Live per-account quota poller (60-120s cadence): hybrid of CRS-provided
- // rateLimitStatus / sessionWindow / overload (populated from upstream response
- // headers in the relay) + targeted /oauth/usage probe for precise u5/u7 headroom.
- // Throttled sequential probes keep it polite under 60-120s window; feeds headroom
- // to scheduler after.
- let liveN = 0;
- for (const a of accts) {
- const key = accountVaultKey(a);
- a._liveUsage = await liveUsage(key);
- if (a._liveUsage) liveN++;
- // Throttle ~150ms between probes (fits 60-120s hybrid header+probe cadence for ~13 accts)
- await new Promise((r) => setTimeout(r, 150));
- }
- // Feed fresh headroom/utilization from targeted probes into CRS so scheduler
- // (unifiedClaudeScheduler etc) sees live quota headroom for selection/LB.
- let fed = 0;
- for (const a of accts) {
- if (a._liveUsage && (await feedLiveHeadroom(auth, a, a._liveUsage))) fed++;
- }
- if (fed) log(`headroom feed: updated ${fed} account(s) in CRS for scheduler`);
-
- // Weekly-cap reconciliation (see WEEKLY-CAP RECONCILIATION in the header comment).
- // Safe no-op on any account/deployment that never sets weeklyRateLimitEndAt.
- if (WEEKLY_RECONCILE && !DRY && !STATUS) {
- for (const a of accts) {
- const u7 = a._liveUsage?.u7;
- const resets7 = a._liveUsage?.resets7;
- if (typeof u7 !== 'number') continue;
- if (u7 >= 100 && resets7 && !a.weeklyRateLimitEndAt) {
- const put = await jfetch(`/admin/claude-accounts/${a.id}`, {
- method: 'PUT',
- headers: auth,
- body: JSON.stringify({ weeklyRateLimitEndAt: resets7 }),
- }).catch(() => null);
- if (put && put.status >= 200 && put.status < 300) {
- a.weeklyRateLimitEndAt = resets7;
- log(`${a.name}: recorded weekly-cap hold until ${resets7} (live 7d=${u7}%)`);
- }
- } else if (u7 < ON_7D && a.weeklyRateLimitEndAt) {
- const put = await jfetch(`/admin/claude-accounts/${a.id}`, {
- method: 'PUT',
- headers: auth,
- body: JSON.stringify({ weeklyRateLimitEndAt: null }),
- }).catch(() => null);
- if (put && put.status >= 200 && put.status < 300) {
- log(`${a.name}: cleared stale weekly-cap hold (live 7d=${u7}% < ${ON_7D}%)`);
- a.weeklyRateLimitEndAt = null;
- }
- }
- }
- }
-
- const decisions = decide(accts);
-
- if (STATUS) {
- console.log(`CRS pool @ ${BASE} — ${decisions.filter((d) => d.cur).length}/${decisions.length} schedulable`);
- for (const d of decisions.sort((a, b) => (a.cur === b.cur ? 0 : a.cur ? -1 : 1))) {
- const flags = [d.rl && 'RL', d.overloaded && 'OVERLOAD', d.sw].filter(Boolean).join(' ');
- console.log(
- ` ${d.cur ? '●' : '○'} ${d.a.name.padEnd(26)} sched=${d.cur} 5h=${String(d.u5).padStart(3)}% ${flags}`,
- );
- }
- return;
- }
-
- let changed = 0;
- for (const d of decisions) {
- if (d.desired === d.cur) continue;
- changed++;
- if (DRY) {
- log(`[dry] ${d.a.name}: ${d.cur}→${d.desired} (${d.reason})`);
- continue;
- }
- const put = await jfetch(`/admin/claude-accounts/${d.a.id}/toggle-schedulable`, {
- method: 'PUT',
- headers: auth,
- body: JSON.stringify({ schedulable: d.desired }),
- });
- log(`${d.a.name}: schedulable ${d.cur}→${d.desired} (${d.reason}) [HTTP ${put.status}]`);
- }
- const on = decisions.filter((d) => d.desired).map((d) => d.a.name);
- const off = decisions
- .filter((d) => !d.desired)
- .map((d) => `${d.a.name}(${d.unavailable ? 'AUTH' : d.sw || (d.rl ? 'RL' : '?')})`);
- if (!on.length) log('WARNING: all active Claude accounts unavailable; leaving pool honestly unschedulable');
- log(
- `tick: ${changed} change(s). live-quota=${liveN}/${decisions.length} schedulable=${on.length} [${on.join(',')}] | off=[${off.join(',')}]`,
- );
- dualWriteFromCrsDecisions(decisions);
- } catch (e) {
- if (BACKEND_WANT === 'crs') {
- log(`ERROR: ${e.message}`);
- process.exit(1);
- }
- log(`CRS unavailable (${e.message}) — falling back to local seat-state`);
- await runLocalTick();
- }
-}
-
-main().catch((e) => {
- log(`ERROR: ${e.message}`);
- process.exit(1);
-});
diff --git a/claude-ops/scripts/account-rotation/crs-priority-daemon.sh b/claude-ops/scripts/account-rotation/crs-priority-daemon.sh
deleted file mode 100755
index 6cb71eff..00000000
--- a/claude-ops/scripts/account-rotation/crs-priority-daemon.sh
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/bin/bash
-# crs-priority-daemon.sh — single-flight wrapper for crs-priority-daemon.mjs.
-# Invoked once-per-tick by launchd/systemd (StartInterval/OnCalendar).
-set -uo pipefail
-
-source "$HOME/.claude/scripts/lib/once.sh" 2>/dev/null || true
-type claude_once >/dev/null 2>&1 && { claude_once crs-priority-daemon 30 || exit 0; }
-
-DIR="$HOME/.claude/scripts/account-rotation"
-LOG="$DIR/crs-priority-daemon.log"
-NODE="$(command -v node || echo /opt/homebrew/bin/node)"
-CRS_CONTAINER="${CRS_CONTAINER:-crs-claude-relay-1}"
-
-export CLAUDE_PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$HOME/.claude/plugins/marketplaces/ops-marketplace/claude-ops}"
-CRED_STORE="$CLAUDE_PLUGIN_ROOT/lib/credential-store.sh"
-KEYCHAIN_ACCOUNT="${CLAUDE_ROTATOR_KEYCHAIN_ACCOUNT:-$USER}"
-
-if [ -z "${CRS_ADMIN_PASSWORD:-}" ]; then
- if [ -f "$CRED_STORE" ]; then
- CRS_ADMIN_PASSWORD="$(bash "$CRED_STORE" get "CRS-Admin-cradmin" "$KEYCHAIN_ACCOUNT" 2>/dev/null || true)"
- fi
- if [ -z "${CRS_ADMIN_PASSWORD:-}" ] && command -v security >/dev/null 2>&1; then
- CRS_ADMIN_PASSWORD="$(security find-generic-password -a "$USER" -s CRS-Admin-cradmin -w 2>/dev/null || true)"
- fi
- if [ -z "${CRS_ADMIN_PASSWORD:-}" ] && command -v docker >/dev/null 2>&1; then
- CRS_ADMIN_PASSWORD="$(docker inspect "$CRS_CONTAINER" --format '{{range .Config.Env}}{{println .}}{{end}}' 2>/dev/null | sed -n 's/^ADMIN_PASSWORD=//p' | head -1 || true)"
- fi
- export CRS_ADMIN_PASSWORD
-fi
-
-# Probe local CRS (Mac 8091/18091 or dev 3005); tolerate down (priority is advisory)
-for p in ${CRS_HEALTH_PORTS:-8091 18091 3005 13005}; do
- curl -sf -o /dev/null --max-time 3 "http://127.0.0.1:${p}/health" && break
-done || true
-
-if [ -f "$LOG" ]; then
- LOGSIZE="$(stat -c%s "$LOG" 2>/dev/null || stat -f%z "$LOG" 2>/dev/null || echo 0)"
- [ "$LOGSIZE" -gt 2097152 ] && mv "$LOG" "$LOG.1"
-fi
-
-"$NODE" "$DIR/crs-priority-daemon.mjs" >> "$LOG" 2>&1
diff --git a/claude-ops/scripts/account-rotation/crs-token-feed.mjs b/claude-ops/scripts/account-rotation/crs-token-feed.mjs
deleted file mode 100644
index f12dff66..00000000
--- a/claude-ops/scripts/account-rotation/crs-token-feed.mjs
+++ /dev/null
@@ -1,326 +0,0 @@
-#!/usr/bin/env node
-/**
- * crs-token-feed.mjs — propagate rotation vault tokens into a local CRS pool.
- *
- * On headless Linux hosts the macOS keychain refresh path is unavailable; this
- * feeder reads ~/.claude/.credentials.json (or crs.fileVaultPath), optionally
- * refreshes expiring tokens, and PUTs claudeAiOauth into mapped CRS accounts.
- *
- * Map each rotator account to a CRS admin account name via crsAccountName in
- * config.json (or crs.nameByVaultKey). Runs via crs-token-feed.timer when installed.
- *
- * node crs-token-feed.mjs # refresh-if-needed + propagate all
- * node crs-token-feed.mjs --dry-run # report only
- * node crs-token-feed.mjs --status # show vault vs CRS state
- */
-import { readFileSync, appendFileSync } from 'fs';
-import { join, dirname } from 'path';
-import { fileURLToPath } from 'url';
-import { execFileSync, spawnSync } from 'child_process';
-import { foreignActiveKeys } from './account-leases.mjs';
-import { assertCrsInvariant } from './route-state.mjs';
-import { acquireRefreshLock, claimRefreshPace } from './crs-refresh-lock.mjs';
-import {
- buildCrsNameMaps,
- crsBaseUrl,
- crsFileVaultPath,
- loadRotationConfig,
- resolveConfigPath,
-} from './crs-pool-config.mjs';
-import { resolveAccountsBackend } from './ops-accounts-backend.mjs';
-import { withAuthWriterLock } from './auth-writer-coordination.mjs';
-import { verifyRefreshedTokenIdentity } from './token-identity.mjs';
-import { automatedAuthAllowed } from './auto-auth-policy.mjs';
-import { publishCredentialFileCas, snapshotCredentialFile } from './credential-file-publication.mjs';
-
-const __dirname = dirname(fileURLToPath(import.meta.url));
-const LOG_PATH = join(__dirname, 'rotation.log');
-const CLIENT_ID = '9d1c250a-e61b-44d9-88ed-5944d1962f5e';
-const TOKEN_ENDPOINT = 'https://platform.claude.com/v1/oauth/token';
-const BUFFER_MS = 2 * 3_600_000; // refresh if expiring within 2h
-const INTER_DELAY_MS = 1_500;
-
-const args = process.argv.slice(2);
-const DRY = args.includes('--dry-run');
-const STATUS = args.includes('--status');
-
-function log(msg) {
- const line = `[${new Date().toISOString()}] [crs-feed] ${msg}`;
- console.log(line);
- try {
- // Operational log data is written only to the fixed plugin-local log path.
- // CodeQL[js/http-to-file-access]
- appendFileSync(LOG_PATH, line + '\n');
- } catch {}
-}
-function accountKey(a) {
- return a.label || a.email;
-}
-const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
-
-function makeVaultOps(fileVaultPath) {
- const snapshots = new WeakMap();
- return {
- load() {
- try {
- const snapshot = snapshotCredentialFile(fileVaultPath);
- const value = snapshot.absent ? {} : JSON.parse(snapshot.bytes.toString('utf8'));
- snapshots.set(value, snapshot);
- return value;
- } catch {
- return {};
- }
- },
- save(v) {
- const snapshot = snapshots.get(v);
- if (!snapshot) throw new Error('CRS_VAULT_SNAPSHOT_REQUIRED');
- publishCredentialFileCas(fileVaultPath, Buffer.from(JSON.stringify(v, null, 2)), snapshot);
- },
- };
-}
-
-async function oauthRefresh(refreshToken) {
- try {
- const res = await fetch(TOKEN_ENDPOINT, {
- method: 'POST',
- headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ grant_type: 'refresh_token', refresh_token: refreshToken, client_id: CLIENT_ID }),
- });
- const body = await res.json().catch(() => ({}));
- if (res.ok && body.access_token) {
- return {
- ok: true,
- oauth: {
- accessToken: body.access_token,
- refreshToken: body.refresh_token || refreshToken,
- expiresAt: body.expires_in ? Date.now() + body.expires_in * 1000 : Date.now() + 8 * 3_600_000,
- subscriptionType: body.subscription_type,
- rateLimitTier: body.rate_limit_tier,
- },
- };
- }
- return { ok: false, status: res.status, error: body?.error?.message || body?.error?.type || `HTTP ${res.status}` };
- } catch (e) {
- return { ok: false, error: e.message };
- }
-}
-
-async function crsLogin(crsBase, crsContainer, adminUser = 'cradmin') {
- let pw = '';
- try {
- // The container name came from config/env and used to be interpolated into a shell
- // pipeline. Run docker directly with argv and do the ADMIN_PASSWORD extraction in JS,
- // so no part of this can be interpreted as shell syntax.
- const envDump = execFileSync(
- 'docker',
- ['inspect', crsContainer, '--format', '{{range .Config.Env}}{{println .}}{{end}}'],
- { timeout: 8000, encoding: 'utf8' },
- );
- for (const line of envDump.split('\n')) {
- if (line.startsWith('ADMIN_PASSWORD=')) {
- pw = line.slice('ADMIN_PASSWORD='.length).trim();
- break;
- }
- }
- } catch {}
- if (!pw) return null;
- if (!process.env.CRS_ADMIN_PASSWORD) process.env.CRS_ADMIN_PASSWORD = pw;
- try {
- const r = await fetch(`${crsBase}/web/auth/login`, {
- method: 'POST',
- headers: { 'Content-Type': 'application/json' },
- body: JSON.stringify({ username: adminUser, password: pw }),
- }).then((x) => x.json());
- const tok = r.token || r.data?.token;
- return tok ? { 'Content-Type': 'application/json', Authorization: `Bearer ${tok}` } : null;
- } catch {
- return null;
- }
-}
-
-async function main() {
- if (!process.env.CLAUDE_ROTATOR_CONFIG) throw new Error('CLAUDE_ROTATOR_CONFIG_REQUIRED');
- if (!process.env.CLAUDE_AUTH_COORDINATION_CONFIG) throw new Error('AUTH_COORDINATION_CONFIG_REQUIRED');
- const earlyCfg = loadRotationConfig();
- const backend = resolveAccountsBackend({ env: process.env, cfgBackend: earlyCfg.crs?.backend });
- if (backend === 'local') {
- log('backend=local — CRS token-feed no-op (CLI seats read vault/keychain directly; no pool to feed)');
- return;
- }
- assertCrsInvariant(process.env, 'crs-token-feed:main');
- const configPath = resolveConfigPath();
- if (!configPath) {
- log('no rotation config found — set CRS_CONFIG or install account-rotation config.json');
- process.exit(1);
- }
- const config = earlyCfg;
- const { nameByVaultKey } = buildCrsNameMaps(config);
- const fileVault = crsFileVaultPath(config);
- const crsBase = crsBaseUrl(config);
- const crsContainer = process.env.CRS_CONTAINER || config.crs?.containerName || 'crs-claude-relay-1';
- const adminUser = process.env.CRS_ADMIN_USER || config.crs?.adminUser || 'cradmin';
- const vault = makeVaultOps(fileVault);
- const H = await crsLogin(crsBase, crsContainer, adminUser);
- if (!H) {
- log('CRS login failed (relay down or no admin pw) — aborting');
- process.exit(1);
- }
- const acctsResp = await fetch(`${crsBase}/admin/claude-accounts`, { headers: H }).then((r) => r.json());
- const byName = Object.fromEntries((acctsResp.data || acctsResp.accounts || acctsResp).map((a) => [a.name, a]));
- const now = Date.now();
- // Contention guard: never refresh an account the OTHER host holds the lease on
- // (refresh tokens are single-use; double-refresh → 400s on the other machine).
- let foreign = new Set();
- try {
- foreign = new Set(foreignActiveKeys());
- } catch (e) {
- log(`lease check failed (${e.message}) — proceeding propagate-only`);
- }
- if (foreign.size) log(`foreign-leased (refresh-skipped): ${[...foreign].join(', ')}`);
-
- if (STATUS) {
- for (const a of config.accounts) {
- const key = accountKey(a);
- const crsName = nameByVaultKey[key];
- const e = vault.load()[`Claude-Rotation-${key}`]?.claudeAiOauth;
- const min = e?.expiresAt ? Math.floor((e.expiresAt - now) / 60000) : 'n/a';
- const crs = byName[crsName];
- console.log(
- ` ${key} -> ${crsName || '?'}: vault_min=${min} crs=${crs ? crs.status + '/sched=' + crs.schedulable : 'MISSING'}`,
- );
- }
- return;
- }
-
- let refreshed = 0,
- propagated = 0,
- skipped = 0,
- missing = 0;
- for (let i = 0; i < config.accounts.length; i++) {
- const a = config.accounts[i];
- const key = accountKey(a);
- const crsName = nameByVaultKey[key];
- if (!crsName || !byName[crsName]) {
- missing++;
- continue;
- }
- const crs = byName[crsName];
-
- await withAuthWriterLock(async () => {
- const vaultData = vault.load();
- const entry = vaultData[`Claude-Rotation-${key}`]?.claudeAiOauth;
- if (!entry?.accessToken) {
- log(`${key}: no vault token — skip`);
- skipped++;
- return;
- }
-
- let oauth = entry;
- const expiring = !oauth.expiresAt || oauth.expiresAt < now + BUFFER_MS;
- if (expiring && foreign.has(key)) {
- log(`${key}: expiring but foreign-leased — refresh deferred to owner host, propagating current token`);
- } else if (expiring && oauth.refreshToken && !DRY && automatedAuthAllowed(a)) {
- if (i > 0) await sleep(INTER_DELAY_MS);
- const release = acquireRefreshLock(key);
- if (!release) {
- log(`${key}: refresh lock held elsewhere — propagate-only`);
- } else {
- try {
- const paceWaitMs = claimRefreshPace(key);
- if (paceWaitMs === null) {
- log(`${key}: refresh pacing lease held elsewhere — propagate-only`);
- return;
- }
- if (paceWaitMs > 0) await sleep(paceWaitMs);
- const r = await withAuthWriterLock(async () => {
- // Re-read under the canonical writer lock: refresh tokens are
- // single-use and the resulting file update is one transaction.
- const currentVault = vault.load();
- const current = currentVault[`Claude-Rotation-${key}`]?.claudeAiOauth;
- if (!current?.refreshToken) return { ok: false, error: 'vault token changed' };
- const result = await oauthRefresh(current.refreshToken);
- if (result.ok) {
- await verifyRefreshedTokenIdentity(a, result.oauth.accessToken);
- const updated = { ...current, ...result.oauth, scopes: current.scopes || [] };
- currentVault[`Claude-Rotation-${key}`] = {
- claudeAiOauth: updated,
- mcpOAuth: currentVault[`Claude-Rotation-${key}`]?.mcpOAuth || {},
- };
- vault.save(currentVault);
- result.oauth = updated;
- }
- return result;
- });
- if (r.ok) {
- oauth = r.oauth;
- refreshed++;
- log(`${key}: refreshed (min_left=${Math.floor((oauth.expiresAt - now) / 60000)})`);
- } else if (r.status === 400) {
- log(`${key}: refresh 400 (rotated elsewhere) — propagating existing vault token`);
- } else {
- log(`${key}: refresh failed (${r.error}) — propagating existing vault token`);
- }
- } finally {
- release();
- }
- }
- } else if (expiring && oauth.refreshToken && !DRY) {
- log(`${key}: automated refresh denied — propagate-only`);
- }
-
- // propagate whatever fresh token we have (don't push already-expired)
- if ((oauth.expiresAt || 0) < now + 60_000) {
- log(`${key}: vault token expired, no refresh — CRS left as-is`);
- skipped++;
- return;
- }
- if (DRY) {
- log(`${key}: [dry] would PUT -> ${crsName}`);
- return;
- }
- try {
- await verifyRefreshedTokenIdentity(a, oauth.accessToken);
- const update = { claudeAiOauth: oauth };
- if ('proxy' in crs) update.proxy = crs.proxy;
- if ('maxConcurrency' in crs) update.maxConcurrency = crs.maxConcurrency;
- if ('schedulable' in crs) update.schedulable = crs.schedulable;
- // The credential is sent only to the configured CRS administrative endpoint.
- // CodeQL[js/file-access-to-http]
- const put = await fetch(`${crsBase}/admin/claude-accounts/${crs.id}`, {
- method: 'PUT',
- headers: H,
- body: JSON.stringify(update),
- });
- if (put.ok) {
- if (crs.schedulable !== false) {
- // This fixed CRS control request contains no credential-file bytes.
- // CodeQL[js/file-access-to-http]
- await fetch(`${crsBase}/admin/claude-accounts/${crs.id}/reset-status`, {
- method: 'POST',
- headers: H,
- }).catch(() => {});
- }
- propagated++;
- } else {
- log(`${key}: CRS PUT ${put.status}`);
- }
- } catch (e) {
- log(`${key}: CRS PUT error ${e.message}`);
- }
- });
- }
- log(`feed complete: ${refreshed} refreshed, ${propagated} propagated, ${skipped} skipped, ${missing} unmapped`);
- if (!DRY && propagated > 0 && process.env.CRS_FEED_SKIP_PRIORITY !== '1') {
- const pr = spawnSync(process.execPath, [join(__dirname, 'crs-priority-daemon.mjs')], {
- env: { ...process.env, CRS_ADMIN_PASSWORD: process.env.CRS_ADMIN_PASSWORD || '' },
- timeout: 120_000,
- encoding: 'utf8',
- });
- if (pr.status === 0) log('priority tick after feed: ok');
- else log(`priority tick after feed: exit=${pr.status} ${(pr.stderr || pr.stdout || '').slice(0, 200)}`);
- }
-}
-main().catch((e) => {
- log(`fatal: ${e.message}`);
- process.exit(1);
-});
diff --git a/claude-ops/scripts/account-rotation/crs-token-feed.sh b/claude-ops/scripts/account-rotation/crs-token-feed.sh
deleted file mode 100755
index fd9693e7..00000000
--- a/claude-ops/scripts/account-rotation/crs-token-feed.sh
+++ /dev/null
@@ -1,8 +0,0 @@
-#!/usr/bin/env bash
-# Single-flight wrapper for crs-token-feed.mjs (NEVER STACK rule).
-set -euo pipefail
-source "$HOME/.claude/scripts/lib/once.sh" 2>/dev/null || true
-if command -v claude_once >/dev/null 2>&1; then
- claude_once crs-token-feed 60 || exit 0
-fi
-exec /usr/local/bin/node "$HOME/.claude/scripts/account-rotation/crs-token-feed.mjs" "$@"
diff --git a/claude-ops/scripts/account-rotation/daemon.mjs b/claude-ops/scripts/account-rotation/daemon.mjs
index 6fadb4da..111e0e9c 100644
--- a/claude-ops/scripts/account-rotation/daemon.mjs
+++ b/claude-ops/scripts/account-rotation/daemon.mjs
@@ -1252,8 +1252,8 @@ async function dynamicRefresh(config, state) {
const key = accountKey(account);
if (key === state.activeAccount) continue; // Don't refresh active account mid-session
if (account.disabled === true) continue; // hands-off: a disabled account is owned
- // elsewhere (e.g. a CRS relay pool). Refreshing its OAuth token rotates the
- // refresh token and INVALIDATES the copy CRS holds → CRS "Invalid API key" 401s.
+ // elsewhere (e.g. a shared proxy pool). Refreshing its OAuth token rotates
+ // the refresh token and INVALIDATES the copy that pool holds → 401s there.
const tokenJson = readStoredToken(account);
if (!tokenJson) continue;
diff --git a/claude-ops/scripts/account-rotation/grok-cli-auth-proxy.py b/claude-ops/scripts/account-rotation/grok-cli-auth-proxy.py
index a58ccb09..860d4d8e 100644
--- a/claude-ops/scripts/account-rotation/grok-cli-auth-proxy.py
+++ b/claude-ops/scripts/account-rotation/grok-cli-auth-proxy.py
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""OpenAI-compatible local proxy backed by SuperGrok CLI OAuth (multi-account).
-Ported into claude-ops for no-CRS installs. Host path ~/.local/bin is not required.
+Ported into claude-ops. Host path ~/.local/bin is not required.
Configure seats via GROK_PREFERRED_EMAILS / GROK_SLOT_FILES / ~/.grok/auth-slots.
Reads OAuth grants from ~/.grok/auth.json plus ~/.grok/auth-slots/*.json, refreshes
diff --git a/claude-ops/scripts/account-rotation/launch-claude.mjs b/claude-ops/scripts/account-rotation/launch-claude.mjs
index 1aac9a28..5cceca9a 100755
--- a/claude-ops/scripts/account-rotation/launch-claude.mjs
+++ b/claude-ops/scripts/account-rotation/launch-claude.mjs
@@ -1,5 +1,4 @@
#!/usr/bin/env node
-import { loadClaudeHarnessEnv } from './claude-harness-env.mjs';
import { normalizeClaudeModelArgs } from './model-args.mjs';
import { readFileSync, existsSync } from 'node:fs';
import { spawnSync } from 'node:child_process';
@@ -149,15 +148,6 @@ function withBypassPermissions(args) {
}
const args = withBypassPermissions(rawArgs);
-const isModelLaunch = shouldBypassPermissions(rawArgs);
-// `agents` is a control subcommand (no tool-permission bypass) but it is the
-// agent-hub interactive surface and MUST keep CRS harness pairing. Stripping
-// CRS keys here forces session-router onto raw OAuth + direct-oauth.settings
-// and surfaces "login required / 401 Invalid bearer token" when vault tokens
-// are stale. Pure path: agent-hub-claude-crs; this keeps the launch-efficient
-// / ls-attach path on CRS too.
-const launchCmd = firstCommand(rawArgs);
-const needsCrsHarness = isModelLaunch || launchCmd === 'agents';
const settingsEnv = (() => {
try {
return JSON.parse(readFileSync(CLAUDE_SETTINGS_PATH, 'utf8'))?.env || {};
@@ -187,33 +177,19 @@ for (const key of [
delete baseEnv.CLAUDE_CODE_USE_BEDROCK;
delete baseEnv.ANTHROPIC_MODEL;
-if (needsCrsHarness) {
- let harnessEnv;
- try {
- harnessEnv = loadClaudeHarnessEnv({ home });
- } catch (error) {
- console.error(error.message);
- process.exit(2);
- }
- Object.assign(baseEnv, harnessEnv);
- // Keep API_KEY=cr_ (derived from harness). Stripping it left BASE→CRS with no key.
- baseEnv.ANTHROPIC_API_KEY = baseEnv.ANTHROPIC_API_KEY || baseEnv.CRS_API_KEY || baseEnv.ANTHROPIC_AUTH_TOKEN;
- // Agent-hub TUI: stay on CRS relay; do not per-account OAuth rewrite.
- if (launchCmd === 'agents') {
- baseEnv.CLAUDE_SESSION_ROUTING = '0';
- }
-} else {
- for (const key of [
- 'ANTHROPIC_BASE_URL',
- 'ANTHROPIC_API_BASE',
- 'ANTHROPIC_API_KEY',
- 'ANTHROPIC_AUTH_TOKEN',
- 'CLAUDE_CODE_OAUTH_TOKEN',
- 'CRS_API_KEY',
- 'CRS_HARNESS_NAME',
- ]) {
- delete baseEnv[key];
- }
+// Launch on Claude Code's own OAuth credential. Any inherited base-URL override
+// or injected key would take precedence over it, so drop them: a pooled setup
+// routes through CLIProxyAPI, which owns its seat files rather than this env.
+for (const key of [
+ 'ANTHROPIC_BASE_URL',
+ 'ANTHROPIC_API_BASE',
+ 'ANTHROPIC_API_KEY',
+ 'ANTHROPIC_AUTH_TOKEN',
+ 'CLAUDE_CODE_OAUTH_TOKEN',
+ 'CRS_API_KEY',
+ 'CRS_HARNESS_NAME',
+]) {
+ delete baseEnv[key];
}
function readJson(path) {
diff --git a/claude-ops/scripts/account-rotation/magic-link-autoloop.mjs b/claude-ops/scripts/account-rotation/magic-link-autoloop.mjs
index 255b8d4e..a735c563 100644
--- a/claude-ops/scripts/account-rotation/magic-link-autoloop.mjs
+++ b/claude-ops/scripts/account-rotation/magic-link-autoloop.mjs
@@ -7,15 +7,15 @@
* needs-reauth and stops. This reconciler, if enabled,
* dispatches ONE unattended browser re-auth per tick (serial, single-flight).
*
- * OPT-IN. Does nothing (exits 0) unless crs.enableMagicLinkRecovery is true
- * (or $CRS_ENABLE_MAGIC_LINK=1) — installing this plugin never silently
- * starts unattended re-auth attempts.
+ * OPT-IN. Does nothing (exits 0) unless rotation.enableMagicLinkRecovery is true
+ * (or $CLAUDE_ROTATION_ENABLE_MAGIC_LINK=1) — installing this plugin never
+ * silently starts unattended re-auth attempts.
*
* Direct reauthentication dispatch has been removed. This reconciler now only
* reports status and a staged-enrollment handoff.
*
* Concurrency (hard rules):
- * 1. Single-flight lock (crs-reconciler-state.mjs) — one tick fleet-wide.
+ * 1. Single-flight lock (reconciler-state.mjs) — one tick fleet-wide.
* 2. Exactly ONE account per tick, always serial.
* 3. Skips the tick entirely if rotate.mjs's own .rotating lock is held —
* never contends with a live rotation. Agents must not spawn parallel
@@ -26,26 +26,26 @@
*
* CLI: (none)=one tick · --dry-run · --status
*/
-import { join, dirname } from 'path';
-import { fileURLToPath } from 'url';
+import { join } from 'path';
import { homedir } from 'os';
-import { existsSync, readFileSync } from 'fs';
-import { loadRotationConfig, buildCrsNameMaps, crsFileVaultPath } from './crs-pool-config.mjs';
-import { loadJsonState, saveJsonStateAtomic, withOwnStateLock } from './crs-reconciler-state.mjs';
+import { existsSync } from 'fs';
+import { loadRotationConfig, rotationSection } from './rotation-config.mjs';
+import { loadJsonState } from './reconciler-state.mjs';
import { resolveReauthTimeoutMs } from './reauth-env.mjs';
-const __dirname = dirname(fileURLToPath(import.meta.url));
const args = new Set(process.argv.slice(2));
-const DRY = args.has('--dry-run');
const STATUS = args.has('--status');
-const C = loadRotationConfig()?.crs || {};
-const ENABLED = process.env.CRS_ENABLE_MAGIC_LINK === '1' || C.enableMagicLinkRecovery === true;
+const C = rotationSection(loadRotationConfig());
+const ENABLED =
+ process.env.CLAUDE_ROTATION_ENABLE_MAGIC_LINK === '1' ||
+ process.env.CRS_ENABLE_MAGIC_LINK === '1' ||
+ C.enableMagicLinkRecovery === true;
const RETRY_COOLDOWN_MS = Number(
- process.env.CRS_MAGIC_LINK_RETRY_COOLDOWN_MS ?? C.magicLinkRetryCooldownMs ?? 21_600_000,
+ process.env.CLAUDE_ROTATION_MAGIC_LINK_RETRY_COOLDOWN_MS ?? C.magicLinkRetryCooldownMs ?? 21_600_000,
);
const ROTATE_TIMEOUT_MS = resolveReauthTimeoutMs(process.env, C);
-const DISPATCH = process.env.CRS_MAGIC_LINK_DISPATCH || C.magicLinkDispatch || 'setup';
+const DISPATCH = process.env.CLAUDE_REAUTH_DISPATCH || C.magicLinkDispatch || 'setup';
function expandHome(p) {
if (!p) return p;
@@ -54,96 +54,22 @@ function expandHome(p) {
const DEFAULT_DATA_DIR =
process.env.CLAUDE_PLUGIN_DATA_DIR || join(homedir(), '.claude', 'plugins', 'data', 'ops-ops-marketplace');
-const STATE_DIR = expandHome(process.env.CRS_STATE_DIR || C.stateDir) || join(DEFAULT_DATA_DIR, 'account-rotation');
-const STATE_PATH = join(STATE_DIR, 'crs-magic-link-state.json');
-const REFRESHER_STATE_PATH = join(STATE_DIR, 'crs-401-state.json');
-const ROTATE_SCRIPT = join(__dirname, 'rotate.mjs');
-const ROTATING_LOCK = join(__dirname, '.rotating');
-
-function log(msg) {
- console.log(`[${new Date().toISOString()}] [magic-link-autoloop] ${msg}`);
-}
-
-function pidAlive(pid) {
- if (!pid) return false;
- try {
- process.kill(pid, 0);
- return true;
- } catch {
- return false;
- }
-}
-
-/** rotate.mjs owns this lock file for any live rotation — never contend with it. */
-function rotatingLockBusy() {
- if (!existsSync(ROTATING_LOCK)) return false;
- try {
- const raw = readFileSync(ROTATING_LOCK, 'utf8').trim();
- // Support both "pid …" plain text and JSON {"pid":N}
- let pid = 0;
- if (raw.startsWith('{')) {
- pid = parseInt(JSON.parse(raw)?.pid || '0', 10) || 0;
- } else {
- const parts = raw.split(/\s+/);
- pid = parseInt(parts[1] || parts[parts.length - 1] || '0', 10) || 0;
- }
- return pidAlive(pid);
- } catch {
- return false;
- }
-}
-
-/** Read the legacy needs-reauth state file, READ-ONLY, during migration. */
-function needsReauthKeysFromRefresher() {
- const state = loadJsonState(REFRESHER_STATE_PATH, log);
- const out = new Set();
- for (const [key, entry] of Object.entries(state || {})) {
- if (entry && typeof entry === 'object' && entry.needsReauth) out.add(key);
- }
- return out;
-}
-
-/** Fallback when no needs-reauth record exists: vault missing refresh_token. */
-function needsReauthKeysFromVault(vaultKeys) {
- const cfg = loadRotationConfig();
- const vaultPath = crsFileVaultPath(cfg) || join(homedir(), '.claude', '.credentials.json');
- if (!existsSync(vaultPath)) return new Set();
- let vault;
- try {
- vault = JSON.parse(readFileSync(vaultPath, 'utf8'));
- } catch {
- return new Set();
- }
- const out = new Set();
- for (const key of vaultKeys) {
- const entry = vault?.[`Claude-Rotation-${key}`] || vault?.[key];
- const oauth = entry?.claudeAiOauth || entry?.oauth || entry;
- const hasRefresh = !!(oauth?.refreshToken || oauth?.refresh_token);
- if (!entry || !hasRefresh) out.add(key);
- }
- return out;
+const STATE_DIR =
+ expandHome(process.env.CLAUDE_ROTATION_STATE_DIR || process.env.CRS_STATE_DIR || C.stateDir) ||
+ join(DEFAULT_DATA_DIR, 'account-rotation');
+
+/** Prefer the current filename; fall back to a legacy file already on disk. */
+function statePath(current, legacy) {
+ const currentPath = join(STATE_DIR, current);
+ const legacyPath = join(STATE_DIR, legacy);
+ if (!existsSync(currentPath) && existsSync(legacyPath)) return legacyPath;
+ return currentPath;
}
-function pickCandidate(now, state) {
- const cfg = loadRotationConfig();
- const maps = buildCrsNameMaps(cfg);
- const byKey = maps?.nameByVaultKey || cfg?.crs?.nameByVaultKey || {};
- const vaultKeys = Object.keys(byKey);
- if (!vaultKeys.length) return null;
-
- const fromRefresher = needsReauthKeysFromRefresher();
- const needy = fromRefresher.size ? fromRefresher : needsReauthKeysFromVault(vaultKeys);
-
- const eligible = [...needy].filter((key) => {
- const last = state[key];
- if (last?.lastAttemptAt && now - last.lastAttemptAt < RETRY_COOLDOWN_MS) return false;
- return true;
- });
- if (!eligible.length) return null;
+const STATE_PATH = statePath('magic-link-state.json', 'crs-magic-link-state.json');
- // Oldest-attempted (or never-attempted) first — no account-name heuristics.
- eligible.sort((a, b) => (state[a]?.lastAttemptAt || 0) - (state[b]?.lastAttemptAt || 0));
- return eligible[0];
+function log(msg) {
+ console.log(`[${new Date().toISOString()}] [magic-link-autoloop] ${msg}`);
}
async function tick() {
@@ -175,7 +101,7 @@ async function main() {
return;
}
if (!ENABLED) {
- log('disabled (crs.enableMagicLinkRecovery is false) — no-op, exiting');
+ log('disabled (rotation.enableMagicLinkRecovery is false) — no-op, exiting');
return;
}
await tick();
diff --git a/claude-ops/scripts/account-rotation/magic-link-autoloop.sh b/claude-ops/scripts/account-rotation/magic-link-autoloop.sh
index 6bd12382..29d3da08 100755
--- a/claude-ops/scripts/account-rotation/magic-link-autoloop.sh
+++ b/claude-ops/scripts/account-rotation/magic-link-autoloop.sh
@@ -19,7 +19,7 @@ fi
# Logs live under plugin data dir when available (cross-platform).
DATA_DIR="${CLAUDE_PLUGIN_DATA_DIR:-${HOME:-}/.claude/plugins/data/ops-ops-marketplace}"
-LOG_DIR="${CRS_LOG_DIR:-$DATA_DIR/logs}"
+LOG_DIR="${CLAUDE_ROTATION_LOG_DIR:-$DATA_DIR/logs}"
mkdir -p "$LOG_DIR" 2>/dev/null || true
LOG="${MAGIC_LINK_AUTOLOOP_LOG:-$LOG_DIR/magic-link-autoloop.log}"
@@ -39,8 +39,8 @@ export CLAUDE_ROT_VISUAL_CAPTCHA="${CLAUDE_ROT_VISUAL_CAPTCHA:-1}"
export CLAUDE_ROT_DESKTOP_ACT="${CLAUDE_ROT_DESKTOP_ACT:-1}"
export CLAUDE_ROT_VNC_AGENT="${CLAUDE_ROT_VNC_AGENT:-1}"
export CLAUDE_ROT_CAPTCHA_MAX_ATTEMPTS="${CLAUDE_ROT_CAPTCHA_MAX_ATTEMPTS:-4}"
-export CRS_CAPTCHA_BROWSER_WAIT_MS="${CRS_CAPTCHA_BROWSER_WAIT_MS:-8000}"
-export CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS="${CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS:-1200000}"
+export CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS="${CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS:-8000}"
+export CLAUDE_REAUTH_TIMEOUT_MS="${CLAUDE_REAUTH_TIMEOUT_MS:-1200000}"
# Clear sticky one-shot skip flags from interactive debug sessions
if [[ "${CLAUDE_ROT_SKIP_TOKEN_SOLVERS:-0}" == "1" && "${CLAUDE_ROT_ALLOW_SKIP_TOKEN_SOLVERS:-0}" != "1" ]]; then
echo "magic-link-autoloop: clearing CLAUDE_ROT_SKIP_TOKEN_SOLVERS=1 (set CLAUDE_ROT_ALLOW_SKIP_TOKEN_SOLVERS=1 to keep)" >&2
diff --git a/claude-ops/scripts/account-rotation/oauth-keep-alive-policy.mjs b/claude-ops/scripts/account-rotation/oauth-keep-alive-policy.mjs
index 07ee6a8e..63978790 100644
--- a/claude-ops/scripts/account-rotation/oauth-keep-alive-policy.mjs
+++ b/claude-ops/scripts/account-rotation/oauth-keep-alive-policy.mjs
@@ -1,7 +1,7 @@
/**
* Shared OAuth token freshness policy — single source of truth for "when is
* a Claude account token stale enough to need a proactive refresh" across
- * refresh-tokens.mjs, crs-token-feed.mjs, and any other refresh call site.
+ * refresh-tokens.mjs and any other refresh call site.
*/
/** Proactively refresh once remaining TTL drops below this. */
diff --git a/claude-ops/scripts/account-rotation/ops-accounts-backend.mjs b/claude-ops/scripts/account-rotation/ops-accounts-backend.mjs
index 04226585..3f9740fa 100644
--- a/claude-ops/scripts/account-rotation/ops-accounts-backend.mjs
+++ b/claude-ops/scripts/account-rotation/ops-accounts-backend.mjs
@@ -1,13 +1,17 @@
/**
- * ops-accounts-backend.mjs — resolve CRS vs local seat-state backend.
+ * ops-accounts-backend.mjs — resolve the seat-state backend.
*
* Env / config:
- * OPS_ACCOUNTS_BACKEND | crs.backend
- * local | seat-state | file → local seat-state (no CRS)
- * crs → CRS admin API only
- * auto (default) → try CRS, fall back to local
+ * OPS_ACCOUNTS_BACKEND | rotation.backend
+ * local | seat-state | file → local seat-state file
+ * cliproxy | proxy → CLIProxyAPI seat inventory
+ * auto (default) → try CLIProxyAPI, fall back to local
*
- * Dual-write (CRS path → seat-state.json) is on by default unless
+ * The retired relay values (crs, relay, claude-relay) resolve to 'local' so an
+ * install that still exports one keeps reading its own seat state instead of
+ * failing on an unknown backend.
+ *
+ * Dual-write into seat-state.json is on by default unless
* OPS_ACCOUNTS_DUAL_WRITE=0.
*/
export function resolveAccountsBackend({ env = process.env, cfgBackend } = {}) {
@@ -18,8 +22,11 @@ export function resolveAccountsBackend({ env = process.env, cfgBackend } = {}) {
if (raw === 'local' || raw === 'seat-state' || raw === 'file' || raw === 'seatstate') {
return 'local';
}
+ if (raw === 'cliproxy' || raw === 'cliproxyapi' || raw === 'proxy') {
+ return 'cliproxy';
+ }
if (raw === 'crs' || raw === 'relay' || raw === 'claude-relay') {
- return 'crs';
+ return 'local';
}
return 'auto';
}
@@ -29,7 +36,7 @@ export function dualWriteEnabled({ env = process.env } = {}) {
}
/**
- * Mirror CRS (or local) decisions into seat-state shape.
+ * Mirror backend decisions into seat-state shape.
* @param {Array<{ email: string, schedulable: boolean, util5h?: number|null, util7d?: number|null, lastError?: string|null }>} seats
* @param {object} opts
*/
diff --git a/claude-ops/scripts/account-rotation/ops-accounts-gateway.mjs b/claude-ops/scripts/account-rotation/ops-accounts-gateway.mjs
index 85eb437d..9208aefa 100644
--- a/claude-ops/scripts/account-rotation/ops-accounts-gateway.mjs
+++ b/claude-ops/scripts/account-rotation/ops-accounts-gateway.mjs
@@ -1,6 +1,6 @@
#!/usr/bin/env node
/**
- * ops-accounts-gateway.mjs — thin optional OpenAI-compat gateway (no CRS).
+ * ops-accounts-gateway.mjs — thin optional OpenAI-compat gateway.
*
* Skeleton (Phase D):
* - GET /health, /v1/health
@@ -12,7 +12,7 @@
* State: file seat-state only. No Redis. No admin SPA.
*
* Env:
- * OPS_ACCOUNTS_GATEWAY_PORT default 3005 (CRS-compat port)
+ * OPS_ACCOUNTS_GATEWAY_PORT default 3005
* OPS_ACCOUNTS_GATEWAY_HOST default 127.0.0.1
* OPS_ACCOUNTS_GATEWAY_KEY if set, require Bearer / x-api-key / api-key
* OPS_ACCOUNTS_STATE_PATH seat-state.json path
@@ -71,7 +71,7 @@ export function classifyRoute(method, urlPath) {
if (path === '/health' || path === '/v1/health') return { kind: 'health', path };
if (path === '/v1/models' || path === '/models') return { kind: 'models', path };
if (path === '/accounts' || path === '/v1/accounts') return { kind: 'accounts', path };
- // Grok hop: anything mentioning grok, or xAI-shaped image routes often used via CRS
+ // Grok hop: anything mentioning grok, or xAI-shaped image routes
if (/\/grok(\/|$)/i.test(path) || path.startsWith('/v1/images') || path.startsWith('/images')) {
return { kind: 'grok-proxy', path };
}
@@ -148,7 +148,7 @@ async function proxyToGrok(req, res, urlPath) {
return;
}
const target = new URL(urlPath, GROK_PROXY_URL);
- // Map CRS-style /grok/v1/... → proxy /v1/...
+ // Map /grok/v1/... → proxy /v1/...
let p = target.pathname;
p = p.replace(/^\/grok(?=\/|$)/i, '');
if (!p.startsWith('/v1') && p !== '/health' && p !== '/accounts') {
@@ -262,7 +262,7 @@ async function handleClaude(req, res, route) {
}
// Skeleton: do not invent OAuth vault reads here. Either forward to CLAUDE_UPSTREAM_URL
// (operator-supplied Anthropic-compat) or return 501 with the chosen seat so harnesses
- // can prove routing without CRS.
+ // can prove routing end to end.
let bodyBuf = Buffer.alloc(0);
try {
bodyBuf = await readBody(req);
diff --git a/claude-ops/scripts/account-rotation/provision-coordination.mjs b/claude-ops/scripts/account-rotation/provision-coordination.mjs
index d31935a0..dfc6d79c 100755
--- a/claude-ops/scripts/account-rotation/provision-coordination.mjs
+++ b/claude-ops/scripts/account-rotation/provision-coordination.mjs
@@ -43,7 +43,6 @@ const PATH_FLAGS = [
'macos-plist',
'runtime-inventory',
'authoritative-consumer-inventory',
- 'linux-token-feed-unit',
'linux-refresh-unit',
];
@@ -734,8 +733,6 @@ function rendered(plan) {
[p['macos-env']]: env,
[p['linux-unit']]:
`[Unit]\nDescription=Claude account rotation daemon\nAfter=network-online.target\n\n[Service]\nType=simple\nEnvironmentFile=${p['linux-env']}\nExecStart=/usr/bin/env node ${resolve(dirname(fileURLToPath(import.meta.url)), 'daemon.mjs')}\nRestart=on-failure\nRestartSec=30\n\n[Install]\nWantedBy=default.target\n`,
- [p['linux-token-feed-unit']]:
- `[Service]\nType=oneshot\nEnvironmentFile=${p['linux-env']}\nExecStart=/bin/bash ${resolve(dirname(fileURLToPath(import.meta.url)), 'crs-token-feed.sh')}\n`,
[p['linux-refresh-unit']]:
`[Service]\nType=oneshot\nEnvironmentFile=${p['linux-env']}\nExecStart=/usr/bin/env node ${resolve(dirname(fileURLToPath(import.meta.url)), 'refresh-tokens.mjs')}\n`,
[p['macos-plist']]: macosPlist,
@@ -808,7 +805,6 @@ function provisionTargetPaths(paths) {
paths['linux-env'],
paths['macos-env'],
paths['linux-unit'],
- paths['linux-token-feed-unit'],
paths['linux-refresh-unit'],
paths['macos-plist'],
]);
diff --git a/claude-ops/scripts/account-rotation/reauth-env.mjs b/claude-ops/scripts/account-rotation/reauth-env.mjs
index 60b5ddf0..0c325132 100644
--- a/claude-ops/scripts/account-rotation/reauth-env.mjs
+++ b/claude-ops/scripts/account-rotation/reauth-env.mjs
@@ -2,7 +2,7 @@
* Portable env builder for unattended re-auth (magic-link-autoloop → rotate.mjs).
*
* No host paths, no account emails, no operator names. Everything is driven by
- * process.env (and optional crs.* config keys). Callers merge the returned
+ * process.env (and optional rotation config keys). Callers merge the returned
* object into child_process.spawn env.
*
* See CAPTCHA-CASCADE.md for the captcha stack contract.
@@ -15,7 +15,7 @@ import { homedir } from 'os';
* Prefer explicit reauth seat vars over the ambient DISPLAY of a headless unit.
*/
export function resolveReauthDisplay(env = process.env) {
- return env.CLAUDE_DESKTOP_DISPLAY || env.DESKTOP_ACT_DISPLAY || env.CRS_REAUTH_DISPLAY || env.DISPLAY || ':1';
+ return env.CLAUDE_DESKTOP_DISPLAY || env.DESKTOP_ACT_DISPLAY || env.CLAUDE_REAUTH_DISPLAY || env.DISPLAY || ':1';
}
/**
@@ -43,23 +43,23 @@ export function resolveReauthPath(env = process.env) {
/**
* @param {object} [opts]
* @param {NodeJS.ProcessEnv} [opts.env]
- * @param {Record<string, unknown>} [opts.crs] optional loadRotationConfig().crs
+ * @param {Record<string, unknown>} [opts.rotation] optional rotationSection(loadRotationConfig())
* @returns {NodeJS.ProcessEnv} env fragment to merge into spawn
*/
export function buildReauthChildEnv(opts = {}) {
const env = opts.env || process.env;
- const crs = opts.crs || {};
+ const cfg = opts.rotation || opts.crs || {};
const display = resolveReauthDisplay(env);
// Dispatch mode: public plugin default is rotate --setup; forks may set magic-link.
- const dispatch = env.CRS_MAGIC_LINK_DISPATCH || crs.magicLinkDispatch || 'setup';
+ const dispatch = env.CLAUDE_REAUTH_DISPATCH || env.CRS_MAGIC_LINK_DISPATCH || cfg.magicLinkDispatch || 'setup';
- const headed = env.CLAUDE_ROT_HEADED || (String(crs.magicLinkHeaded ?? '1') === '0' ? '0' : '1');
+ const headed = env.CLAUDE_ROT_HEADED || (String(cfg.magicLinkHeaded ?? '1') === '0' ? '0' : '1');
const fragment = {
CLAUDE_ROTATION_MAGIC_LINK_AUTO: '1',
CLAUDE_ROT_STEAL_LOCK: '0',
- CRS_MAGIC_LINK_DISPATCH: String(dispatch),
+ CLAUDE_REAUTH_DISPATCH: String(dispatch),
// Seat for headed Chrome + desktop-act (must match reauth browser DISPLAY)
DISPLAY: display,
@@ -75,7 +75,7 @@ export function buildReauthChildEnv(opts = {}) {
CLAUDE_ROT_DESKTOP_ACT: env.CLAUDE_ROT_DESKTOP_ACT || '1',
CLAUDE_ROT_VNC_AGENT: env.CLAUDE_ROT_VNC_AGENT || '1',
CLAUDE_ROT_CAPTCHA_MAX_ATTEMPTS: env.CLAUDE_ROT_CAPTCHA_MAX_ATTEMPTS || '4',
- CRS_CAPTCHA_BROWSER_WAIT_MS: env.CRS_CAPTCHA_BROWSER_WAIT_MS || '8000',
+ CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS: env.CLAUDE_ROT_CAPTCHA_BROWSER_WAIT_MS || '8000',
// Never leave a one-shot "skip solvers" flag sticky in the reconciler env
CLAUDE_ROT_SKIP_TOKEN_SOLVERS: env.CLAUDE_ROT_SKIP_TOKEN_SOLVERS || '0',
@@ -92,7 +92,7 @@ export function buildReauthChildEnv(opts = {}) {
'DESKTOP_ACT_MAX_ITERATIONS',
'CLAUDE_PLUGIN_ROOT',
'CLAUDE_PLUGIN_DATA_DIR',
- 'CRS_REAUTH_CDP_PORT',
+ 'CLAUDE_REAUTH_CDP_PORT',
]) {
if (env[key]) fragment[key] = env[key];
}
@@ -104,12 +104,9 @@ export function buildReauthChildEnv(opts = {}) {
* Wall-clock budget for one unattended reauth child.
* Longer than a pure token refresh: visual + desktop-act layers need room.
*/
-export function resolveReauthTimeoutMs(env = process.env, crs = {}) {
+export function resolveReauthTimeoutMs(env = process.env, cfg = {}) {
return Number(
- env.CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS ??
- env.CRS_MAGIC_LOOP_ROTATE_TIMEOUT_MS ??
- crs.magicLinkRotateTimeoutMs ??
- 20 * 60_000,
+ env.CLAUDE_REAUTH_TIMEOUT_MS ?? env.CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS ?? cfg.magicLinkRotateTimeoutMs ?? 20 * 60_000,
);
}
diff --git a/claude-ops/scripts/account-rotation/crs-reconciler-state.mjs b/claude-ops/scripts/account-rotation/reconciler-state.mjs
similarity index 84%
rename from claude-ops/scripts/account-rotation/crs-reconciler-state.mjs
rename to claude-ops/scripts/account-rotation/reconciler-state.mjs
index 1ed92480..d7689b68 100644
--- a/claude-ops/scripts/account-rotation/crs-reconciler-state.mjs
+++ b/claude-ops/scripts/account-rotation/reconciler-state.mjs
@@ -1,9 +1,9 @@
#!/usr/bin/env node
/**
- * crs-reconciler-state.mjs — shared own-state-file helper for CRS pool reconcilers.
+ * reconciler-state.mjs — shared own-state-file helper for rotation reconcilers.
*
- * Each reconciler that watches a claude-relay-service (CRS) account pool (cooldown
- * tracking, token-refresh tracking, priority scoring, ...) MUST persist to its OWN
+ * Each reconciler that watches the account pool (cooldown tracking, token-refresh
+ * tracking, priority scoring, ...) MUST persist to its OWN
* state file — never a file shared with another reconciler. Sharing one file was the
* root cause of a documented production incident: reconciler A's non-atomic write
* clobbered reconciler B's cooldown data mid-tick, silently erasing real rate-limit
@@ -21,7 +21,7 @@
* (older than LOCK_STALE_MS, e.g. left behind by a crashed process) is reclaimed
* rather than wedging the reconciler forever.
*
- * Nothing here talks to CRS or knows about account schedules — it is pure state I/O so
+ * Nothing here talks to a provider or knows about account schedules — it is pure state I/O so
* every reconciler can `import` the same tiny, well-tested primitive instead of each
* re-implementing tmp+rename by hand.
*/
@@ -36,7 +36,7 @@ export function loadJsonState(path, log = console.error) {
try {
return JSON.parse(readFileSync(path, 'utf8'));
} catch (e) {
- log(`[crs-reconciler-state] WARN corrupt state at ${path}, resetting: ${e.message}`);
+ log(`[reconciler-state] WARN corrupt state at ${path}, resetting: ${e.message}`);
return {};
}
}
@@ -93,13 +93,13 @@ function tryAcquireLock(lockDir, log) {
}
}
if (ageMs < LOCK_STALE_MS) {
- log(`[crs-reconciler-state] lock held (age ${Math.round(ageMs / 1000)}s) at ${lockDir} — skipping this tick`);
+ log(`[reconciler-state] lock held (age ${Math.round(ageMs / 1000)}s) at ${lockDir} — skipping this tick`);
return false;
}
try {
rmdirSync(lockDir);
mkdirSync(lockDir);
- log(`[crs-reconciler-state] reclaimed stale lock (age ${Math.round(ageMs / 1000)}s) at ${lockDir}`);
+ log(`[reconciler-state] reclaimed stale lock (age ${Math.round(ageMs / 1000)}s) at ${lockDir}`);
return true;
} catch {
return false;
diff --git a/claude-ops/scripts/account-rotation/crs-refresh-lock.mjs b/claude-ops/scripts/account-rotation/refresh-lock.mjs
similarity index 89%
rename from claude-ops/scripts/account-rotation/crs-refresh-lock.mjs
rename to claude-ops/scripts/account-rotation/refresh-lock.mjs
index e7c3a532..c8f4982d 100644
--- a/claude-ops/scripts/account-rotation/crs-refresh-lock.mjs
+++ b/claude-ops/scripts/account-rotation/refresh-lock.mjs
@@ -9,15 +9,19 @@ const DEFAULT_LOCK_DIR = join(
'account-rotation',
'refresh-locks',
);
-const LOCK_DIR = process.env.CRS_REFRESH_LOCK_DIR || DEFAULT_LOCK_DIR;
-const LOCK_TTL_MS = (Number(process.env.CRS_REFRESH_LOCK_TTL_SEC) || 120) * 1000;
+// CLAUDE_REFRESH_* are the current names. The CRS_REFRESH_* names are read as a
+// fallback so an install that still exports the old ones keeps its pacing.
+const envValue = (name) => process.env[`CLAUDE_REFRESH_${name}`] ?? process.env[`CRS_REFRESH_${name}`];
+
+const LOCK_DIR = envValue('LOCK_DIR') || DEFAULT_LOCK_DIR;
+const LOCK_TTL_MS = (Number(envValue('LOCK_TTL_SEC')) || 120) * 1000;
const GUARD_TTL_MS = 10_000;
const PRODUCTION_MIN_PACE_MS = 1_000;
const MIN_PACE_MS = Math.max(
- process.env.CRS_REFRESH_TEST_ALLOW_ZERO_PACE === '1' ? 0 : PRODUCTION_MIN_PACE_MS,
- Number(process.env.CRS_REFRESH_MIN_PACE_MS) || PRODUCTION_MIN_PACE_MS,
+ envValue('TEST_ALLOW_ZERO_PACE') === '1' ? 0 : PRODUCTION_MIN_PACE_MS,
+ Number(envValue('MIN_PACE_MS')) || PRODUCTION_MIN_PACE_MS,
);
-const MAX_JITTER_MS = Math.max(0, Math.min(5_000, Number(process.env.CRS_REFRESH_JITTER_MS) || 500));
+const MAX_JITTER_MS = Math.max(0, Math.min(5_000, Number(envValue('JITTER_MS')) || 500));
function safeKey(key) {
return String(key).replace(/[^a-zA-Z0-9_.@-]/g, '_');
diff --git a/claude-ops/scripts/account-rotation/refresh-tokens.mjs b/claude-ops/scripts/account-rotation/refresh-tokens.mjs
index b55f9340..4ccd856f 100644
--- a/claude-ops/scripts/account-rotation/refresh-tokens.mjs
+++ b/claude-ops/scripts/account-rotation/refresh-tokens.mjs
@@ -4,9 +4,9 @@
* access tokens for all vault accounts before they expire.
*
* Doctrine: OAuth tokens must never become stale.
- * - launchd every 15m (com.sam.crs-rotation-refresh)
+ * - launchd every 15m (com.claude-ops.rotation-refresh)
* - refresh when remaining TTL < 6h (see oauth-keep-alive-policy.mjs)
- * - dead refresh_token → mark needsReauth for magic-link-autoloop
+ * - dead refresh_token → mark needsReauth for staged enrollment
*
* Usage:
* node refresh-tokens.mjs # Refresh tokens under keep-alive floor
@@ -21,7 +21,7 @@ import { userInfo } from 'os';
import { fileURLToPath } from 'url';
import { execFileSync, spawnSync } from 'child_process';
import { fetchWithProxyFallback } from './proxy-helper.mjs';
-import { acquireRefreshLock, claimRefreshPace } from './crs-refresh-lock.mjs';
+import { acquireRefreshLock, claimRefreshPace } from './refresh-lock.mjs';
import { readRotationToken, reconcileRemoteRotationVault, writeRotationTokenCoordinated } from './rotation-vault.mjs';
import { withAuthWriterLock } from './auth-writer-coordination.mjs';
import { verifyRefreshedTokenIdentity } from './token-identity.mjs';
@@ -39,8 +39,17 @@ const __dirname = dirname(fileURLToPath(import.meta.url));
const CONFIG_PATH = process.env.CLAUDE_ROTATOR_CONFIG || join(__dirname, 'config.json');
const STATE_PATH = join(__dirname, 'state.json');
const LOG_PATH = join(__dirname, 'rotation.log');
-const NEEDS_REAUTH_PATH = join(__dirname, '.crs-token-refresher-state.json');
-const AUTOLOOP_STATE_PATH = join(__dirname, '.crs-magic-autoloop-state.json');
+/** Prefer the current filename; keep reading a pre-existing legacy file so an
+ * in-place upgrade does not lose its needs-reauth flags. */
+function statePath(current, legacy) {
+ const currentPath = join(__dirname, current);
+ const legacyPath = join(__dirname, legacy);
+ if (!existsSync(currentPath) && existsSync(legacyPath)) return legacyPath;
+ return currentPath;
+}
+
+const NEEDS_REAUTH_PATH = statePath('.token-refresher-state.json', '.crs-token-refresher-state.json');
+const AUTOLOOP_STATE_PATH = statePath('.magic-autoloop-state.json', '.crs-magic-autoloop-state.json');
const KEYCHAIN_SERVICE = 'Claude Code-credentials';
const KEYCHAIN_ACCOUNT = process.env.USER || process.env.LOGNAME || userInfo().username;
@@ -113,26 +122,6 @@ function writeStoredToken(account, json, capability) {
return writeRotationTokenCoordinated(account, json, capability);
}
-function syncStoredTokenToCrs(account) {
- if (process.env.CLAUDE_ROTATION_SKIP_CRS_SYNC === '1') return;
- const key = accountKey(account);
- try {
- // JSON.stringify only adds double quotes, and $(...) is still expanded inside shell
- // double quotes, so the account key must never reach a shell. Pass it as an argv entry.
- const res = spawnSync('node', [join(__dirname, 'sync-crs-account.mjs'), key], {
- timeout: 45_000,
- encoding: 'utf8',
- });
- if (res.status !== 0) {
- throw new Error(`${res.stderr || res.stdout || `exit ${res.status}`}`.trim());
- }
- const out = `${res.stdout || ''}${res.stderr || ''}`.trim();
- log(out.split('\n').slice(-1)[0] || `${key}: CRS sync complete`);
- } catch (err) {
- log(`${key}: ⚠ CRS sync failed — ${String(err.message || err).slice(0, 180)}`);
- }
-}
-
function readState() {
try {
return JSON.parse(readFileSync(STATE_PATH, 'utf8'));
@@ -482,7 +471,6 @@ for (let i = 0; i < accountsOrdered.length; i++) {
// Save to vault
await writeStoredToken(account, JSON.stringify(updated), writerCapability);
- syncStoredTokenToCrs(account);
clearNeedsReauth(key);
const newHoursLeft = ((result.expiresAt - Date.now()) / 3_600_000).toFixed(1);
log(`${key}: ✓ refreshed (${newHoursLeft}h remaining)`);
diff --git a/claude-ops/scripts/account-rotation/rotate.mjs b/claude-ops/scripts/account-rotation/rotate.mjs
index 2bc52361..f6a72a82 100644
--- a/claude-ops/scripts/account-rotation/rotate.mjs
+++ b/claude-ops/scripts/account-rotation/rotate.mjs
@@ -124,7 +124,7 @@ const CLIPROXYAPI_CONFIG = process.env.CLIPROXYAPI_CONFIG || join(CLIPROXYAPI_DI
const CLIPROXYAPI_AUTH_DIR =
process.env.CLIPROXYAPI_AUTH_DIR || process.env.CLIPROXY_AUTH_DIR || join(CLIPROXYAPI_DIR, 'auths');
const CLIPROXY_REPLICA_SYNC =
- process.env.CLIPROXY_REPLICA_SYNC || join(homedir(), '.local', 'bin', 'crsproxy-replica-sync.py');
+ process.env.CLIPROXY_REPLICA_SYNC || join(homedir(), '.local', 'bin', 'cliproxy-replica-sync.py');
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
@@ -850,17 +850,17 @@ function stopClaudeDaemon() {
// one with `source ~/.claude/scripts/account-rotation/use-bedrock.sh`.
async function activateBedrockFallback(reason, dryRun = false) {
// Kill-switch — mirror daemon.mjs activateBedrockFallbackFromDaemon (disabled
- // 2026-05-08, Max/CRS plan handles capacity; Bedrock is metered AWS = $600-1.2k/mo
+ // 2026-05-08, the Max plan handles capacity; Bedrock is metered AWS = $600-1.2k/mo
// bleed). BLOCKED by default; set CLAUDE_DISABLE_BEDROCK_FALLBACK=0 to re-enable.
// Authoritative here regardless of caller so the rotate path can never silently
- // flip a CRS/Max-routed box onto metered Bedrock (the bedrockOnExhausted opt
+ // flip a Max-routed box onto metered Bedrock (the bedrockOnExhausted opt
// defaults ON, unlike the daemon — this is the backstop).
if (process.env.CLAUDE_DISABLE_BEDROCK_FALLBACK !== '0') {
log(`[bedrock-fallback] BLOCKED by CLAUDE_DISABLE_BEDROCK_FALLBACK (reason was: ${reason})`);
if (!dryRun) {
notify(
'Bedrock Fallback BLOCKED',
- 'Kill-switch active — Max/CRS-only mode. Wait for reset window or rotate manually.',
+ 'Kill-switch active — Max-only mode. Wait for reset window or rotate manually.',
);
}
return false;
diff --git a/claude-ops/scripts/account-rotation/crs-pool-config.mjs b/claude-ops/scripts/account-rotation/rotation-config.mjs
similarity index 74%
rename from claude-ops/scripts/account-rotation/crs-pool-config.mjs
rename to claude-ops/scripts/account-rotation/rotation-config.mjs
index e869fcce..0f3a246f 100644
--- a/claude-ops/scripts/account-rotation/crs-pool-config.mjs
+++ b/claude-ops/scripts/account-rotation/rotation-config.mjs
@@ -1,13 +1,17 @@
#!/usr/bin/env node
/**
- * crs-pool-config.mjs — shared CRS + rotation config resolution for public installs.
+ * rotation-config.mjs — shared rotation config resolution for public installs.
*
- * Vault key ↔ CRS account name mapping is NEVER hardcoded in repo scripts.
- * Configure per account via `crsAccountName` and/or optional `crs.nameByVaultKey`.
+ * Reads the rotator config file and exposes the generic settings the rotation
+ * scripts need: seat vault path, credential-store lookup, and the optional
+ * per-account egress proxy descriptor.
+ *
+ * Multi-account seats live in CLIProxyAPI. There is no relay pool here.
+ * The settings block is `rotation`; a legacy `crs` block is still read so an
+ * existing config keeps working, but new configs should use `rotation`.
*/
import { readFileSync, existsSync } from 'fs';
-import { execFileSync } from 'child_process';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { homedir } from 'os';
@@ -19,12 +23,13 @@ const DATA_DIR =
export const CONFIG_CANDIDATES = [
process.env.CLAUDE_ROTATOR_CONFIG,
- process.env.CRS_CONFIG,
join(homedir(), '.claude', 'scripts', 'account-rotation', 'config.json'),
join(DATA_DIR, 'account-rotation', 'config.json'),
join(homedir(), '.claude', 'plugins', 'data', 'ops', 'account-rotation', 'config.json'),
join(PLUGIN_ROOT, 'scripts', 'account-rotation', 'config.json'),
join(__dirname, 'config.json'),
+ // Legacy env name kept last so an older install still resolves its config.
+ process.env.CRS_CONFIG,
].filter(Boolean);
export function resolveConfigPath() {
@@ -38,73 +43,35 @@ export function resolveConfigPath() {
export function loadRotationConfig() {
const path = resolveConfigPath();
- if (!path) return { crs: {}, accounts: [] };
+ if (!path) return { rotation: {}, accounts: [] };
try {
- return { crs: {}, accounts: [], ...JSON.parse(readFileSync(path, 'utf8')) };
+ return { rotation: {}, accounts: [], ...JSON.parse(readFileSync(path, 'utf8')) };
} catch {
- return { crs: {}, accounts: [] };
+ return { rotation: {}, accounts: [] };
}
}
/**
- * @returns {{ nameByVaultKey: Record<string,string>, vaultKeyByCrsName: Record<string,string> }}
+ * The tunables block of the rotator config. `rotation` is the current key;
+ * `crs` is the legacy key an older install may still have on disk.
*/
-export function buildCrsNameMaps(config = loadRotationConfig()) {
- const nameByVaultKey = {};
- const vaultKeyByCrsName = {};
-
- const overrides = config.crs?.nameByVaultKey;
- if (overrides && typeof overrides === 'object') {
- for (const [vaultKey, crsName] of Object.entries(overrides)) {
- if (!vaultKey || !crsName) continue;
- nameByVaultKey[vaultKey] = crsName;
- if (!vaultKeyByCrsName[crsName]) vaultKeyByCrsName[crsName] = vaultKey;
- }
- }
-
- for (const a of config.accounts || []) {
- const crsName = a.crsAccountName || a.crsName;
- if (!crsName) continue;
- const keys = [];
- if (a.email) keys.push(a.email);
- if (a.label) keys.push(a.label);
- for (const k of keys) {
- if (!k) continue;
- nameByVaultKey[k] = crsName;
- }
- if (!vaultKeyByCrsName[crsName]) {
- vaultKeyByCrsName[crsName] = a.email || a.label || null;
- }
- }
-
- for (const [vaultKey, crsName] of Object.entries(nameByVaultKey)) {
- if (crsName && !vaultKeyByCrsName[crsName]) vaultKeyByCrsName[crsName] = vaultKey;
- }
-
- return { nameByVaultKey, vaultKeyByCrsName };
+export function rotationSection(config = loadRotationConfig()) {
+ const current = config?.rotation;
+ if (current && typeof current === 'object' && Object.keys(current).length) return current;
+ const legacy = config?.crs;
+ if (legacy && typeof legacy === 'object') return legacy;
+ return {};
}
-export function crsBaseUrl(config = loadRotationConfig()) {
- return process.env.CRS_BASE || config.crs?.baseUrl || 'http://127.0.0.1:3005';
-}
-
-export function crsFileVaultPath(config = loadRotationConfig()) {
- const fromEnv = process.env.CRS_FILE_VAULT;
+/** Path to the OAuth seat vault the rotator reads and writes. */
+export function fileVaultPath(config = loadRotationConfig()) {
+ const fromEnv = process.env.CLAUDE_ROTATION_FILE_VAULT || process.env.CRS_FILE_VAULT;
if (fromEnv) return fromEnv.replace(/^~(?=$|\/)/, homedir());
- const fromCfg = config.crs?.fileVaultPath;
+ const fromCfg = rotationSection(config).fileVaultPath;
if (fromCfg) return fromCfg.replace(/^~(?=$|\/)/, homedir());
return join(homedir(), '.claude', '.credentials.json');
}
-export function crsPolicy(config = loadRotationConfig()) {
- const raw = String(process.env.CRS_POLICY || config.crs?.policy || 'conservative')
- .trim()
- .toLowerCase()
- .replace(/_/g, '-');
- if (raw === 'maxout' || raw === 'max-out') return 'max-out';
- return 'conservative';
-}
-
/** Candidate paths for the shared ops credential-store CLI (never hardcoded to one account/box). */
export function credentialStoreCandidates() {
const pluginRoot = process.env.CLAUDE_PLUGIN_ROOT || join(__dirname, '..', '..');
@@ -133,47 +100,6 @@ export function resolveCredentialStorePath() {
return candidates[candidates.length - 1];
}
-/**
- * Resolve the CRS admin password for HTTP basic auth against a self-hosted
- * claude-relay-service pool. Resolution order (first hit wins), none of it
- * ever hardcoded to one deployment:
- * 1. $CRS_ADMIN_PASSWORD
- * 2. the env var named by config.crs.adminPasswordEnv (or $CRS_ADMIN_PASSWORD_ENV)
- * 3. `docker inspect <container>` — reads ADMIN_PASSWORD out of a co-located
- * Docker Compose CRS deploy's own container env
- * 4. the shared ops credential-store CLI, key `CRS-Admin-<adminUser>`
- * Returns null (never throws) if nothing resolves — callers decide whether that
- * means "skip this tick" or "hard error".
- */
-export function resolveCrsAdminPassword(config = loadRotationConfig(), opts = {}) {
- if (process.env.CRS_ADMIN_PASSWORD) return process.env.CRS_ADMIN_PASSWORD;
- const envName = config.crs?.adminPasswordEnv || process.env.CRS_ADMIN_PASSWORD_ENV;
- if (envName && process.env[envName]) return process.env[envName];
-
- const adminUser = opts.adminUser || process.env.CRS_ADMIN_USER || config.crs?.adminUser || 'cradmin';
- const container = opts.container || process.env.CRS_CONTAINER || config.crs?.containerName || 'crs-claude-relay-1';
- const execOpts = { encoding: 'utf8', stdio: ['pipe', 'pipe', 'ignore'] };
-
- try {
- const out = execFileSync(
- 'docker',
- ['inspect', container, '--format', '{{range .Config.Env}}{{println .}}{{end}}'],
- execOpts,
- );
- const m = out.match(/^ADMIN_PASSWORD=(.+)$/m);
- if (m?.[1]?.trim()) return m[1].trim();
- } catch {}
-
- try {
- const cred = resolveCredentialStorePath();
- const acct = process.env.CLAUDE_ROTATOR_KEYCHAIN_ACCOUNT || process.env.USER || 'claude-ops';
- const pw = execFileSync('bash', [cred, 'get', `CRS-Admin-${adminUser}`, acct], execOpts).trim();
- if (pw) return pw;
- } catch {}
-
- return null;
-}
-
export function vaultLookupKeysForEmail(email, accounts = []) {
const keys = new Set();
if (email) keys.add(email);
@@ -293,12 +219,12 @@ function brightDataUsernameForAccount(rawUser, bright = {}, key, accountIndex =
}
function proxyRoutingEnabled(config) {
+ const section = rotationSection(config);
const raw = String(
process.env.CLAUDE_ROTATION_PROXY_ENABLED ||
- process.env.CRS_ACCOUNT_PROXY_ENABLED ||
- firstEnvValue(['CLAUDE_ROTATION_PROXY_ENABLED', 'CRS_ACCOUNT_PROXY_ENABLED']) ||
- config.crs?.proxyEnabled ||
- config.crs?.proxy?.enabled ||
+ firstEnvValue(['CLAUDE_ROTATION_PROXY_ENABLED']) ||
+ section.proxyEnabled ||
+ section.proxy?.enabled ||
'',
)
.trim()
@@ -307,12 +233,12 @@ function proxyRoutingEnabled(config) {
}
function proxyProviderOrder(config) {
+ const section = rotationSection(config);
const raw = String(
process.env.CLAUDE_ROTATION_PROXY_PROVIDER ||
- process.env.CRS_ACCOUNT_PROXY_PROVIDER ||
- firstEnvValue(['CLAUDE_ROTATION_PROXY_PROVIDER', 'CRS_ACCOUNT_PROXY_PROVIDER']) ||
- config.crs?.proxyProvider ||
- config.crs?.proxy?.provider ||
+ firstEnvValue(['CLAUDE_ROTATION_PROXY_PROVIDER']) ||
+ section.proxyProvider ||
+ section.proxy?.provider ||
'2captcha,brightdata',
);
return raw
@@ -434,21 +360,21 @@ function efgProxyConfig(config = {}) {
// default. Require explicit config or env; return null if neither is set so
// callers treat it as "no EFG proxy configured" rather than silently pointing
// at some other operator's network.
- const url = config.url || firstEnvValue(['EFG_PROXY_URL', 'CRS_EFG_PROXY_URL']);
+ const url = config.url || firstEnvValue(['EFG_PROXY_URL']);
if (url) {
const parsed = parseProxyUrl(url);
if (parsed?.host && parsed?.port) return parsed;
}
- const host = config.host || firstEnvValue(['EFG_PROXY_HOST', 'CRS_EFG_PROXY_HOST']);
- const port = config.port || firstEnvValue(['EFG_PROXY_PORT', 'CRS_EFG_PROXY_PORT']);
+ const host = config.host || firstEnvValue(['EFG_PROXY_HOST']);
+ const port = config.port || firstEnvValue(['EFG_PROXY_PORT']);
if (!host || !port) return null;
return {
- type: config.type || firstEnvValue(['EFG_PROXY_TYPE', 'CRS_EFG_PROXY_TYPE']) || 'http',
+ type: config.type || firstEnvValue(['EFG_PROXY_TYPE']) || 'http',
host,
port: Number(port),
- username: config.username || firstEnvValue(['EFG_PROXY_USERNAME', 'CRS_EFG_PROXY_USERNAME']),
- password: config.password || firstEnvValue(['EFG_PROXY_PASSWORD', 'CRS_EFG_PROXY_PASSWORD']),
+ username: config.username || firstEnvValue(['EFG_PROXY_USERNAME']),
+ password: config.password || firstEnvValue(['EFG_PROXY_PASSWORD']),
};
}
@@ -511,14 +437,15 @@ export function accountProxyConfig(config = {}, account, env = process.env) {
const direct = account?.proxy || account?.proxyConfig;
if (direct) return direct;
- const byKey = cfg.crs?.proxyByVaultKey || cfg.crs?.proxies;
+ const section = rotationSection(cfg);
+ const byKey = section.proxyByVaultKey || section.proxies;
if (byKey && typeof byKey === 'object' && byKey[key]) return byKey[key];
if (!proxyRoutingEnabled(cfg)) return null;
- const bright = cfg.crs?.brightData || {};
- const twoCaptcha = cfg.crs?.twoCaptchaProxy || cfg.crs?.captchaProxy || {};
- const efg = cfg.crs?.efgProxy || cfg.crs?.efg || {};
+ const bright = section.brightData || {};
+ const twoCaptcha = section.twoCaptchaProxy || section.captchaProxy || {};
+ const efg = section.efgProxy || section.efg || {};
const accountIndex = Math.max(
0,
(cfg.accounts || []).findIndex((a) => accountKey(a) === key),
diff --git a/claude-ops/scripts/account-rotation/crs-pool-config.test.mjs b/claude-ops/scripts/account-rotation/rotation-config.test.mjs
similarity index 80%
rename from claude-ops/scripts/account-rotation/crs-pool-config.test.mjs
rename to claude-ops/scripts/account-rotation/rotation-config.test.mjs
index 6fc89dd3..9be47289 100644
--- a/claude-ops/scripts/account-rotation/crs-pool-config.test.mjs
+++ b/claude-ops/scripts/account-rotation/rotation-config.test.mjs
@@ -1,21 +1,12 @@
import test from 'node:test';
import assert from 'node:assert/strict';
-import { buildCrsNameMaps, crsPolicy, accountProxyConfig } from './crs-pool-config.mjs';
+import { accountProxyConfig, rotationSection } from './rotation-config.mjs';
-test('buildCrsNameMaps uses crsAccountName and nameByVaultKey', () => {
- const config = {
- accounts: [{ email: '[email protected]', crsAccountName: 'pool-a' }],
- crs: { nameByVaultKey: { legacy: 'pool-legacy' } },
- };
- const { nameByVaultKey, vaultKeyByCrsName } = buildCrsNameMaps(config);
- assert.equal(nameByVaultKey['[email protected]'], 'pool-a');
- assert.equal(nameByVaultKey.legacy, 'pool-legacy');
- assert.equal(vaultKeyByCrsName['pool-a'], '[email protected]');
-});
-
-test('crsPolicy defaults to conservative', () => {
- assert.equal(crsPolicy({ crs: {} }), 'conservative');
- assert.equal(crsPolicy({ crs: { policy: 'max-out' } }), 'max-out');
+test('rotationSection prefers the rotation block and falls back to the legacy one', () => {
+ assert.deepEqual(rotationSection({ rotation: { policy: 'a' } }), { policy: 'a' });
+ assert.deepEqual(rotationSection({ crs: { policy: 'b' } }), { policy: 'b' });
+ assert.deepEqual(rotationSection({ rotation: {}, crs: { policy: 'b' } }), { policy: 'b' });
+ assert.deepEqual(rotationSection({}), {});
});
test('accountProxyConfig returns null when proxy is disabled', () => {
@@ -43,7 +34,7 @@ test('accountProxyConfig returns null when provider URL is missing', () => {
});
test('accountProxyConfig parses socks5:// url for efg provider', () => {
- // Exact contract from crs-parity-regression.test.mjs.
+ // Exact contract for the public proxy descriptor.
assert.deepEqual(
accountProxyConfig(
{},
diff --git a/claude-ops/scripts/account-rotation/rotation-vault.mjs b/claude-ops/scripts/account-rotation/rotation-vault.mjs
index faefbb9a..591ae11e 100644
--- a/claude-ops/scripts/account-rotation/rotation-vault.mjs
+++ b/claude-ops/scripts/account-rotation/rotation-vault.mjs
@@ -130,7 +130,7 @@ export function writeRotationTokenCoordinated(account, token, capability) {
throw new Error('ROTATION_TOKEN_MUTATION_APPROVAL_REQUIRED');
}
-export function reconcileRemoteRotationVault({ host = process.env.CRS_SSH_HOST || 'dev-us' } = {}) {
+export function reconcileRemoteRotationVault({ host = process.env.CLAUDE_ROTATION_SSH_HOST || 'dev-us' } = {}) {
// A local lock cannot serialize or identity-verify a writer on another host.
// Distributed reconciliation remains operationally fenced until the remote
// endpoint exposes the same canonical lock + exact profile-verification API.
diff --git a/claude-ops/scripts/account-rotation/route-state.mjs b/claude-ops/scripts/account-rotation/route-state.mjs
index 7e7519b8..b0efba8f 100644
--- a/claude-ops/scripts/account-rotation/route-state.mjs
+++ b/claude-ops/scripts/account-rotation/route-state.mjs
@@ -1,64 +1,82 @@
-import { loadClaudeHarnessEnv } from './claude-harness-env.mjs';
-import { chmodSync, existsSync, mkdirSync, readFileSync, renameSync, statSync, unlinkSync, writeFileSync } from 'fs';
-import { execFileSync } from 'child_process';
+import { chmodSync, existsSync, mkdirSync, readFileSync, renameSync, statSync, writeFileSync } from 'fs';
import { dirname, join } from 'path';
import { homedir, hostname } from 'os';
-export const ROUTE_MODES = new Set(['crs-oauth', 'fail-closed']);
-
-// CRS base⇄token desync guard (the safety net for every settings/overlay write).
-// INVARIANT: ANTHROPIC_BASE_URL points at the CRS relay XNOR a cr_-prefixed
-// relay key in ANTHROPIC_API_KEY. Auth-token fields are reserved for the
-// launcher-created child environment and must not persist in settings.
-// Extra hosts (e.g. a tailnet relay) can be added via OPS_RELAY_HOSTS as a
-// comma-separated host:port list; nothing site-specific is hardcoded here.
+/**
+ * Claude routing state.
+ *
+ * Two modes:
+ * oauth — Claude Code's own OAuth credential. settings.json carries no
+ * base URL and no injected API key.
+ * fail-closed — OAuth is unavailable and Bedrock has not been confirmed, so
+ * the PreToolUse guard blocks rather than spending on Bedrock.
+ *
+ * `crs-oauth` and `proxy-oauth` are read as aliases of `oauth` so an existing
+ * ~/.claude/claude-routing-state.json keeps resolving after the relay removal.
+ *
+ * Multi-account rotation and OAuth seat management live in CLIProxyAPI. Nothing
+ * here injects pool credentials into settings.json any more.
+ */
+export const ROUTE_MODES = new Set(['oauth', 'fail-closed']);
+
+const LEGACY_ROUTE_MODES = new Map([
+ ['crs-oauth', 'oauth'],
+ ['proxy-oauth', 'oauth'],
+]);
+
+/** Map a stored or user-supplied mode onto a current one. */
+export function normalizeRouteMode(mode) {
+ const raw = String(mode || '');
+ return LEGACY_ROUTE_MODES.get(raw) || raw;
+}
+
+// A retired relay answered on one of these ports and issued `cr_` bearer
+// tokens. Both are scrubbed from any settings this module writes.
+// A relay reachable somewhere else (a tailnet address, say) can be named in
+// OPS_RELAY_HOSTS as a comma-separated host:port list; nothing site-specific
+// is hardcoded here.
+const LEGACY_RELAY_PORT_RE = /:(3000|3002|3005|8091|18091)(\/|$)/;
const EXTRA_RELAY_HOSTS = (process.env.OPS_RELAY_HOSTS || '')
.split(',')
.map((h) => h.trim())
- .filter(Boolean)
- .map((h) => h.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'));
-const CRS_BASE_RE = new RegExp(
- ['127\\.0\\.0\\.1:(3000|3002|3005|8091|18091)']
- .concat(EXTRA_RELAY_HOSTS)
- .concat([':(3000|3002|3005|8091|18091)\\/api'])
- .join('|'),
-);
+ .filter(Boolean);
-function currentCrsToken(env = {}) {
- return isCrsToken(env.ANTHROPIC_API_KEY) ? env.ANTHROPIC_API_KEY : '';
+export function isLegacyRelayBase(baseUrl) {
+ if (!baseUrl) return false;
+ const value = String(baseUrl);
+ if (LEGACY_RELAY_PORT_RE.test(value)) return true;
+ return EXTRA_RELAY_HOSTS.some((host) => value.includes(host));
}
-export function isCrsBase(baseUrl) {
- return !!baseUrl && CRS_BASE_RE.test(String(baseUrl));
+export function isLegacyRelayToken(token) {
+ return String(token || '').startsWith('cr_');
}
-export function isCrsToken(token) {
- return String(token || '').startsWith('cr_');
+/** True when settings.env still carries credentials for the removed relay. */
+export function hasLegacyRelayEnv(env = {}) {
+ const e = env && typeof env === 'object' ? env : {};
+ return (
+ isLegacyRelayBase(e.ANTHROPIC_BASE_URL) ||
+ isLegacyRelayBase(e.ANTHROPIC_API_BASE) ||
+ isLegacyRelayToken(e.ANTHROPIC_API_KEY) ||
+ isLegacyRelayToken(e.ANTHROPIC_AUTH_TOKEN) ||
+ isLegacyRelayToken(e.CLAUDE_CODE_OAUTH_TOKEN)
+ );
}
-export function assertCrsInvariant(env, where = 'crs-env') {
+/** Delete every relay leftover from an env map, in place. Returns the map. */
+export function scrubLegacyRelayEnv(env = {}) {
const e = env && typeof env === 'object' ? env : {};
- const baseIsCrs = isCrsBase(e.ANTHROPIC_BASE_URL) || isCrsBase(e.ANTHROPIC_API_BASE);
- const tokIsCr = isCrsToken(currentCrsToken(e));
- if (baseIsCrs !== tokIsCr) {
- throw new Error(
- `[${where}] CRS base-token desync (fail-closed, refusing write): ` +
- `base=${baseIsCrs ? 'CRS' : 'non-CRS'} token=${tokIsCr ? 'cr_' : 'non-cr_'}. ` +
- 'ANTHROPIC_BASE_URL and CRS token vars must be set together or not at all.',
- );
+ if (isLegacyRelayBase(e.ANTHROPIC_BASE_URL)) delete e.ANTHROPIC_BASE_URL;
+ if (isLegacyRelayBase(e.ANTHROPIC_API_BASE)) delete e.ANTHROPIC_API_BASE;
+ for (const key of ['ANTHROPIC_API_KEY', 'ANTHROPIC_AUTH_TOKEN', 'CLAUDE_CODE_OAUTH_TOKEN']) {
+ if (isLegacyRelayToken(e[key])) delete e[key];
}
- if (isCrsToken(e.ANTHROPIC_AUTH_TOKEN) || isCrsToken(e.CLAUDE_CODE_OAUTH_TOKEN)) {
- throw new Error(`[${where}] refusing to persist cr_ relay key in auth-token fields`);
- }
- return env;
+ return e;
}
const STATE_PATH = join(homedir(), '.claude', 'claude-routing-state.json');
const SETTINGS_PATH = join(homedir(), '.claude', 'settings.json');
-const CRS_SESSION_SETTINGS_PATH = join(homedir(), '.claude', 'crs-session-settings.json');
-const CRKEY_PATH = join(homedir(), '.claude', 'scripts', 'account-rotation', '.crkey');
-const FALLBACK_MARKER = join(homedir(), '.claude', 'scripts', 'account-rotation', 'crs-fallback-active');
-const HEALTH_WATCH_STATE = join(homedir(), '.claude', 'scripts', 'account-rotation', 'crs-health-watch.state.json');
const MODEL_KEYS = [
'ANTHROPIC_MODEL',
@@ -72,6 +90,7 @@ const MODEL_KEYS = [
const BEDROCK_KEYS = ['CLAUDE_CODE_USE_BEDROCK', 'AWS_BEDROCK_REGION'];
const ANTHROPIC_DIRECT_KEYS = [
+ 'ANTHROPIC_BASE_URL',
'ANTHROPIC_API_BASE',
'ANTHROPIC_API_KEY',
'ANTHROPIC_AUTH_TOKEN',
@@ -116,23 +135,6 @@ function readSettingsForUpdate() {
return settings;
}
-function readSettingsEnv() {
- return readJson(SETTINGS_PATH, { env: {} }).env || {};
-}
-
-function defaultCrsConfig() {
- const env = readSettingsEnv();
- const defaultBase = process.platform === 'darwin' ? 'http://127.0.0.1:8091/api' : 'http://127.0.0.1:3005/api';
- const inheritedBase = isCrsBase(env.ANTHROPIC_BASE_URL) ? env.ANTHROPIC_BASE_URL : null;
- const baseUrl = process.env.CRS_BASE_URL || inheritedBase || defaultBase;
- const healthUrl = process.env.CRS_HEALTH_URL || baseUrl.replace(/\/api\/?$/, '/health');
- return {
- baseUrl,
- healthUrl,
- authority: process.platform === 'darwin' ? 'mac-local-primary' : 'dev-us',
- };
-}
-
function writeJsonAtomic(path, data, defaultMode = null) {
mkdirSync(dirname(path), { recursive: true });
const tmp = `${path}.tmp.${process.pid}`;
@@ -145,28 +147,22 @@ function writeJsonAtomic(path, data, defaultMode = null) {
renameSync(tmp, path);
}
-function crKey() {
- try {
- return loadClaudeHarnessEnv().CRS_API_KEY;
- } catch {
- return '';
- }
-}
-
export function readRouteState() {
const state = readJson(STATE_PATH, null);
- if (!state || !ROUTE_MODES.has(state.mode)) {
+ const mode = normalizeRouteMode(state?.mode);
+ if (!state || !ROUTE_MODES.has(mode)) {
return {
- mode: 'crs-oauth',
+ mode: 'oauth',
updatedAt: nowIso(),
updatedBy: 'default',
reason: 'default',
bedrockConfirmation: null,
- crs: defaultCrsConfig(),
host: hostname(),
};
}
- return state;
+ const { crs, ...rest } = state;
+ void crs; // a retired relay block may still be on disk; it is not read
+ return { ...rest, mode };
}
export function writeRouteState(next) {
@@ -174,9 +170,11 @@ export function writeRouteState(next) {
const merged = {
...prev,
...next,
+ mode: normalizeRouteMode(next?.mode ?? prev.mode),
updatedAt: nowIso(),
host: hostname(),
};
+ delete merged.crs;
if (!ROUTE_MODES.has(merged.mode)) {
throw new Error(`invalid route mode: ${merged.mode}`);
}
@@ -191,102 +189,54 @@ export function bedrockConfirmationActive(state = readRouteState(), at = Date.no
}
export function setRouteMode(mode, options = {}) {
- if (!ROUTE_MODES.has(mode)) throw new Error(`invalid route mode: ${mode}`);
- const reason = options.reason || 'manual';
- const ttlMinutes = Number.isFinite(options.ttlMinutes) ? options.ttlMinutes : 60;
+ const resolved = normalizeRouteMode(mode);
+ if (!ROUTE_MODES.has(resolved)) throw new Error(`invalid route mode: ${mode}`);
const next = {
- mode,
- reason,
+ mode: resolved,
+ reason: options.reason || 'manual',
updatedBy: options.updatedBy || process.env.USER || 'unknown',
+ bedrockConfirmation: null,
};
- if (options.crs) next.crs = options.crs;
- next.bedrockConfirmation = null;
const state = writeRouteState(next);
applyRouteToSettings(state, options);
return state;
}
+/**
+ * Reconcile ~/.claude/settings.json with the route state.
+ *
+ * Both modes produce the same env: no Bedrock, no injected base URL, no
+ * injected API key. Claude Code uses its own OAuth credential. The difference
+ * between them is what the PreToolUse guard does, not what settings contain.
+ * This is also the migration step that clears a leftover relay pair.
+ */
export function applyRouteToSettings(state = readRouteState(), options = {}) {
+ void options;
const settings = readSettingsForUpdate();
const env = settings.env && typeof settings.env === 'object' ? { ...settings.env } : {};
- for (const key of [...MODEL_KEYS, ...ANTHROPIC_DIRECT_KEYS]) delete env[key];
+ for (const key of [...MODEL_KEYS, ...ANTHROPIC_DIRECT_KEYS, ...BEDROCK_KEYS]) delete env[key];
delete settings.model;
delete settings.availableModels;
-
- if (state.mode === 'crs-oauth') {
- for (const key of BEDROCK_KEYS) delete env[key];
- env.AWS_REGION = env.AWS_REGION || process.env.AWS_REGION || 'us-east-1';
- const key = crKey();
- if (!key.startsWith('cr_')) {
- throw new Error(`CRS relay key missing or invalid at ${CRKEY_PATH}`);
- }
- env.ANTHROPIC_BASE_URL = state.crs?.baseUrl || defaultCrsConfig().baseUrl;
- env.ANTHROPIC_API_BASE = env.ANTHROPIC_BASE_URL;
- env.ANTHROPIC_API_KEY = key;
- delete env.ANTHROPIC_AUTH_TOKEN;
- delete env.CLAUDE_CODE_OAUTH_TOKEN;
- } else {
- for (const key of BEDROCK_KEYS) delete env[key];
- delete env.ANTHROPIC_BASE_URL;
- delete env.ANTHROPIC_AUTH_TOKEN;
- delete env.CLAUDE_CODE_OAUTH_TOKEN;
- delete env.ANTHROPIC_API_KEY;
- }
+ void state;
settings.env = env;
- assertCrsInvariant(env, `applyRouteToSettings:${state.mode}`);
writeJsonAtomic(SETTINGS_PATH, settings);
return settings;
}
-export function assertCrsSessionInvariant(env, where = 'crs-session-env') {
- const e = env && typeof env === 'object' ? env : {};
- const baseUrl = String(e.ANTHROPIC_BASE_URL || '');
- const apiBase = String(e.ANTHROPIC_API_BASE || '');
- const authToken = String(e.ANTHROPIC_AUTH_TOKEN || '');
- const oauthToken = String(e.CLAUDE_CODE_OAUTH_TOKEN || '');
- if (
- !isCrsBase(baseUrl) ||
- baseUrl !== apiBase ||
- !isCrsToken(authToken) ||
- authToken !== oauthToken ||
- e.ANTHROPIC_API_KEY !== undefined
- ) {
- throw new Error(`[${where}] invalid CRS model-child auth contract; refusing settings write`);
- }
- return env;
-}
-
-export function applyCrsSessionSettings(options = {}) {
- const settings = readJson(CRS_SESSION_SETTINGS_PATH, {});
- const env = settings.env && typeof settings.env === 'object' ? { ...settings.env } : {};
- const key = options.key || crKey();
- if (!key.startsWith('cr_')) {
- throw new Error(`CRS relay key missing or invalid at ${CRKEY_PATH}`);
- }
- const baseUrl = options.baseUrl || readRouteState().crs?.baseUrl || defaultCrsConfig().baseUrl;
- env.ANTHROPIC_BASE_URL = baseUrl;
- env.ANTHROPIC_API_BASE = baseUrl;
- env.ANTHROPIC_AUTH_TOKEN = key;
- env.CLAUDE_CODE_OAUTH_TOKEN = key;
- delete env.ANTHROPIC_API_KEY;
- settings.env = env;
- assertCrsSessionInvariant(env, 'applyCrsSessionSettings');
- writeJsonAtomic(CRS_SESSION_SETTINGS_PATH, settings, 0o600);
- return settings;
-}
-
export function settingsRouteSnapshot() {
const settings = readJson(SETTINGS_PATH, { env: {} });
const env = settings.env || {};
const base = String(env.ANTHROPIC_BASE_URL || '');
- const token = currentCrsToken(env);
+ const token = String(env.ANTHROPIC_API_KEY || env.ANTHROPIC_AUTH_TOKEN || env.CLAUDE_CODE_OAUTH_TOKEN || '');
const bedrock = env.CLAUDE_CODE_USE_BEDROCK === '1';
+ const legacyRelay = hasLegacyRelayEnv(env);
return {
settingsPath: SETTINGS_PATH,
bedrock,
baseUrl: base || null,
- tokenKind: token.startsWith('cr_') ? 'crs-relay' : token ? 'other' : 'none',
+ tokenKind: isLegacyRelayToken(token) ? 'legacy-relay' : token ? 'other' : 'none',
+ legacyRelay,
mixedProvider: bedrock && (!!base || !!token),
modelOverride: settings.model || null,
availableModelsOverride: existsSync(SETTINGS_PATH) && Array.isArray(settings.availableModels),
@@ -302,55 +252,3 @@ export function routeStatus() {
settings: settingsRouteSnapshot(),
};
}
-
-export function probeCrsHealth(state = readRouteState()) {
- const healthUrl = process.env.CRS_HEALTH_URL || state.crs?.healthUrl || defaultCrsConfig().healthUrl;
- try {
- const code = execFileSync('curl', ['-sf', '-o', '/dev/null', '-w', '%{http_code}', '--max-time', '3', healthUrl], {
- encoding: 'utf8',
- timeout: 5000,
- }).trim();
- return code === '200';
- } catch {
- return false;
- }
-}
-
-export function clearCrsFallbackMarker() {
- try {
- if (existsSync(FALLBACK_MARKER)) unlinkSync(FALLBACK_MARKER);
- } catch {}
-}
-
-function resetHealthWatchCounters() {
- try {
- writeJsonAtomic(HEALTH_WATCH_STATE, {
- down: 0,
- up: 0,
- inferenceDown: 0,
- lastInferenceHealAt: 0,
- mode: 'crs',
- });
- } catch {}
-}
-
-export function restoreCrsRouteIfHealthy(options = {}) {
- const state = readRouteState();
- if (state.mode !== 'fail-closed') {
- return { restored: false, reason: 'not-fail-closed', mode: state.mode };
- }
- if (!probeCrsHealth(state)) {
- return { restored: false, reason: 'crs-unhealthy', mode: state.mode };
- }
- try {
- setRouteMode('crs-oauth', {
- reason: options.reason || 'CRS live probe healthy',
- updatedBy: options.updatedBy || 'crs-route-recovery',
- });
- clearCrsFallbackMarker();
- resetHealthWatchCounters();
- return { restored: true, reason: 'crs-oauth', mode: 'crs-oauth' };
- } catch (e) {
- return { restored: false, reason: e.message || String(e), mode: state.mode };
- }
-}
diff --git a/claude-ops/scripts/account-rotation/seat-policy-tick.mjs b/claude-ops/scripts/account-rotation/seat-policy-tick.mjs
index 16d35c94..f7e59b70 100755
--- a/claude-ops/scripts/account-rotation/seat-policy-tick.mjs
+++ b/claude-ops/scripts/account-rotation/seat-policy-tick.mjs
@@ -1,7 +1,7 @@
#!/usr/bin/env node
/**
* seat-policy-tick.mjs — conservative schedulable policy against local seat-state.
- * No CRS. Reads optional live util from rotate.mjs --utilization JSON if present.
+ * Reads optional live util from rotate.mjs --utilization JSON if present.
*
* Usage:
* node seat-policy-tick.mjs [--dry-run] [--status]
diff --git a/claude-ops/scripts/account-rotation/seat-state.mjs b/claude-ops/scripts/account-rotation/seat-state.mjs
index e0760d1a..dd94f88c 100755
--- a/claude-ops/scripts/account-rotation/seat-state.mjs
+++ b/claude-ops/scripts/account-rotation/seat-state.mjs
@@ -1,9 +1,9 @@
#!/usr/bin/env node
/**
- * seat-state.mjs — local multi-provider seat state (no CRS required).
+ * seat-state.mjs — local multi-provider seat state.
*
* File-backed store used by ops-accounts policy when backend=local.
- * Schema is provider-agnostic; Claude CRS daemons can dual-write later.
+ * Schema is provider-agnostic; other backends can dual-write into it.
*
* Usage:
* node seat-state.mjs status
@@ -149,7 +149,6 @@ switch (cmd) {
case 'import-claude-config': {
// Best-effort: seed emails from rotation config (no tokens)
const candidates = [
- process.env.CRS_CONFIG,
join(DATA, 'account-rotation', 'config.json'),
join(homedir(), '.claude', 'plugins', 'data', 'ops', 'account-rotation', 'config.json'),
].filter(Boolean);
diff --git a/claude-ops/scripts/account-rotation/secrets-bootstrap.mjs b/claude-ops/scripts/account-rotation/secrets-bootstrap.mjs
index 007d9e5a..60d4a2a2 100644
--- a/claude-ops/scripts/account-rotation/secrets-bootstrap.mjs
+++ b/claude-ops/scripts/account-rotation/secrets-bootstrap.mjs
@@ -79,7 +79,7 @@ let _done = false;
const SECRET_FILES = [
process.env.CLAUDE_ROTATOR_SECRETS_FILE,
- join(homedir(), '.config', 'crs-sync', 'rotator-secrets.env'),
+ join(homedir(), '.config', 'claude-rotation', 'rotator-secrets.env'),
join(homedir(), '.mcp-secrets.env'),
].filter(Boolean);
diff --git a/claude-ops/scripts/account-rotation/session-router.mjs b/claude-ops/scripts/account-rotation/session-router.mjs
index f49f3ff7..9a681377 100644
--- a/claude-ops/scripts/account-rotation/session-router.mjs
+++ b/claude-ops/scripts/account-rotation/session-router.mjs
@@ -40,6 +40,7 @@ import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { spawnSync, spawn } from 'child_process';
import { cachedUtilizationMax } from './rotation-policy.mjs';
+import { isLegacyRelayToken, scrubLegacyRelayEnv } from './route-state.mjs';
const __dirname = dirname(fileURLToPath(import.meta.url));
const LEASES_PATH = join(__dirname, 'session-leases.json');
@@ -258,31 +259,24 @@ export function pickAccountForSession(sessionId, config, state) {
return coolest ? accountKey(coolest) : null;
}
-const CRS_BASE_RE = /127\.0\.0\.1:(3000|3002|3005|8091|18091)|100\.87\.53\.96:8091|:(3000|3002|3005|8091|18091)\/api/;
-
/**
- * ~/.claude/settings.json's env block carries a fleet-wide, autofixer-enforced
- * ANTHROPIC_BASE_URL/ANTHROPIC_API_BASE pinned to CRS (see crs-sync-update.sh
- * "checking ANTHROPIC_BASE_URL parity" autofix — it reverts removal of these
- * keys within minutes, by design). Claude Code re-applies settings.json's env
- * block internally at startup regardless of what env the spawning process set,
- * which silently re-points a direct-OAuth account-rotation session at CRS and
- * breaks it (CRS rejects the raw sk-ant-oat01 token: 401 Invalid API key format).
+ * Claude Code re-applies settings.json's env block internally at startup,
+ * whatever env the spawning process set. If that block pins ANTHROPIC_BASE_URL
+ * or an API key, a per-session OAuth token gets overridden and the session
+ * fails to authenticate.
*
- * Rather than fight the autofixer on the shared settings.json, generate a
- * settings copy with just those two keys stripped and point THIS spawn at it
- * via --settings, only when we've actually assigned a direct (non-cr_) OAuth
- * token. CRS-paired sessions are untouched and keep using the real settings.json.
+ * Rather than mutate the shared settings.json, generate a settings copy with
+ * those keys blanked and point THIS spawn at it via --settings, only when we
+ * have actually assigned a direct OAuth token.
*/
function buildDirectOauthSettingsOverride() {
try {
// --settings loads ADDITIONAL settings merged on top of the base
// settings.json (confirmed empirically: "--help" says "load additional
// settings from", and omitting a key does NOT unset the base file's
- // value — deleting keys here is a no-op). To actually neutralize the
- // CRS pin for this one spawn, explicitly override both keys to "".
- // Also blank CRS API-key fields. settings.json env injects
- // ANTHROPIC_API_KEY=cr_* fleet-wide; if we only clear BASE_URL, Claude Code
+ // value — deleting keys here is a no-op). To actually neutralize a pinned
+ // base URL for this one spawn, explicitly override the keys to "".
+ // The API-key fields go too: if we only clear BASE_URL, Claude Code
// dual-auths (OAuth + external API key) and surfaces "Invalid API key".
const overrideDoc = {
env: {
@@ -290,7 +284,6 @@ function buildDirectOauthSettingsOverride() {
ANTHROPIC_API_BASE: '',
ANTHROPIC_API_KEY: '',
ANTHROPIC_AUTH_TOKEN: '',
- CRS_API_KEY: '',
},
};
if (!existsSync(DIRECT_OAUTH_SETTINGS_DIR)) {
@@ -306,45 +299,15 @@ function buildDirectOauthSettingsOverride() {
}
}
-export function enforceDirectOrCrsPairing(childEnv) {
- const crsBase =
- CRS_BASE_RE.test(String(childEnv.ANTHROPIC_BASE_URL || '')) ||
- CRS_BASE_RE.test(String(childEnv.ANTHROPIC_API_BASE || ''));
-
- // CRS pairing: keep API_KEY=cr_ (and mirror onto AUTH/OAUTH). Do not strip API_KEY.
- if (String(childEnv.ANTHROPIC_API_KEY || '').startsWith('cr_')) {
- if (
- !String(childEnv.CLAUDE_CODE_OAUTH_TOKEN || '').startsWith('cr_') &&
- !String(childEnv.ANTHROPIC_AUTH_TOKEN || '').startsWith('cr_')
- ) {
- childEnv.CLAUDE_CODE_OAUTH_TOKEN = childEnv.ANTHROPIC_API_KEY;
- childEnv.ANTHROPIC_AUTH_TOKEN = childEnv.ANTHROPIC_API_KEY;
- }
- }
-
- const crToken = String(
- childEnv.CLAUDE_CODE_OAUTH_TOKEN || childEnv.ANTHROPIC_AUTH_TOKEN || childEnv.ANTHROPIC_API_KEY || '',
- ).startsWith('cr_');
- if (crsBase && crToken) {
- const crKey = String(
- childEnv.ANTHROPIC_API_KEY || childEnv.ANTHROPIC_AUTH_TOKEN || childEnv.CLAUDE_CODE_OAUTH_TOKEN || '',
- );
- if (crKey.startsWith('cr_')) {
- childEnv.ANTHROPIC_API_KEY = crKey;
- childEnv.ANTHROPIC_AUTH_TOKEN = crKey;
- childEnv.CLAUDE_CODE_OAUTH_TOKEN = crKey;
- }
- }
- if (crsBase && !crToken) {
- delete childEnv.ANTHROPIC_BASE_URL;
- delete childEnv.ANTHROPIC_API_BASE;
- delete childEnv.ANTHROPIC_AUTH_TOKEN;
- }
- if (!crsBase && crToken) {
- delete childEnv.CLAUDE_CODE_OAUTH_TOKEN;
- delete childEnv.ANTHROPIC_AUTH_TOKEN;
- delete childEnv.ANTHROPIC_API_KEY;
- }
+/**
+ * Every spawned session authenticates directly, with its own OAuth token.
+ *
+ * The removed relay backend paired a loopback base URL with a `cr_` bearer
+ * token. Both are stripped here so a leftover pair on an upgraded machine can
+ * never re-point a child at a relay that is no longer managed.
+ */
+export function enforceDirectAuth(childEnv) {
+ scrubLegacyRelayEnv(childEnv);
return childEnv;
}
@@ -403,22 +366,14 @@ export function getTokenForSession(sessionId, config) {
export function spawnWithAccount(args, config, state, opts = {}) {
const sessionId = opts.sessionId || `session-${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const childEnv = { ...(opts.env || process.env) };
- enforceDirectOrCrsPairing(childEnv);
+ enforceDirectAuth(childEnv);
let assignedKey = null;
- const crsBaseConfigured =
- CRS_BASE_RE.test(String(childEnv.ANTHROPIC_BASE_URL || '')) ||
- CRS_BASE_RE.test(String(childEnv.ANTHROPIC_API_BASE || ''));
- const crsTokenConfigured = String(childEnv.CLAUDE_CODE_OAUTH_TOKEN || childEnv.ANTHROPIC_AUTH_TOKEN || '').startsWith(
- 'cr_',
- );
-
- // CRS-route the bg daemon and any explicitly CRS-configured process: skip
- // per-account OAuth injection so children keep settings.json's cr_ relay key
- // -> CRS instead of replacing it with a direct OAuth account token.
+ // The bg daemon keeps whatever credential it was started with; every other
+ // spawn gets a per-account OAuth token.
const isDaemon = Array.isArray(args) && args.includes('daemon');
- const usePerAccountRouting = !(isDaemon || (crsBaseConfigured && crsTokenConfigured));
+ const usePerAccountRouting = !isDaemon;
if (usePerAccountRouting) {
const key = pickAccountForSession(sessionId, config, state);
@@ -441,10 +396,9 @@ export function spawnWithAccount(args, config, state, opts = {}) {
}
}
- enforceDirectOrCrsPairing(childEnv);
+ enforceDirectAuth(childEnv);
- const directOauthToken =
- childEnv.CLAUDE_CODE_OAUTH_TOKEN && !String(childEnv.CLAUDE_CODE_OAUTH_TOKEN).startsWith('cr_');
+ const directOauthToken = childEnv.CLAUDE_CODE_OAUTH_TOKEN && !isLegacyRelayToken(childEnv.CLAUDE_CODE_OAUTH_TOKEN);
let spawnArgs = args;
if (directOauthToken && !childEnv.ANTHROPIC_BASE_URL && !args.includes('--settings')) {
const overridePath = buildDirectOauthSettingsOverride();
diff --git a/claude-ops/scripts/account-rotation/use-bedrock.sh b/claude-ops/scripts/account-rotation/use-bedrock.sh
index b86975e0..ecf80b72 100755
--- a/claude-ops/scripts/account-rotation/use-bedrock.sh
+++ b/claude-ops/scripts/account-rotation/use-bedrock.sh
@@ -18,7 +18,7 @@ _ROT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
if [[ "${CLAUDE_CONFIRM_METERED_BEDROCK:-}" != "1" ]]; then
echo "⛔ Bedrock fallback blocked."
- echo " Bedrock is metered AWS usage. OAuth/CRS is the preferred route."
+ echo " Bedrock is metered AWS usage. OAuth is the preferred route."
echo " Reason OAuth is unavailable: ${CLAUDE_BEDROCK_REASON:-not supplied}"
echo " To confirm for this short session: CLAUDE_CONFIRM_METERED_BEDROCK=1 source ~/.claude/scripts/account-rotation/use-bedrock.sh"
return 2 2>/dev/null || exit 2
diff --git a/claude-ops/scripts/account-rotation/use-oauth.sh b/claude-ops/scripts/account-rotation/use-oauth.sh
index d036a943..52d7f08c 100755
--- a/claude-ops/scripts/account-rotation/use-oauth.sh
+++ b/claude-ops/scripts/account-rotation/use-oauth.sh
@@ -1,9 +1,9 @@
#!/bin/bash
-# use-oauth.sh — exit Bedrock mode in this shell, restore CRS-backed OAuth defaults.
+# use-oauth.sh — exit Bedrock mode in this shell, restore OAuth defaults.
# Source me: `source ~/.claude/scripts/account-rotation/use-oauth.sh`
#
-# Persists to ~/.claude/settings.json via claude-routing-state:
-# ANTHROPIC_BASE_URL points at CRS and CLAUDE_CODE_OAUTH_TOKEN is the CRS relay token.
+# Persists to ~/.claude/settings.json via claude-routing-state: no base-URL
+# override and no injected API key, so Claude Code uses its own OAuth credential.
unset CLAUDE_CODE_USE_BEDROCK
unset AWS_BEDROCK_REGION
@@ -18,12 +18,12 @@ unset ANTHROPIC_AUTH_TOKEN
unset ANTHROPIC_BASE_URL
_ROT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)"
-"$_ROT_DIR/../../../.local/bin/claude-stack" route --mode crs-oauth --reason oauth-restored >/dev/null || return 1 2>/dev/null || exit 1
+"$_ROT_DIR/../../../.local/bin/claude-stack" route --mode oauth --reason oauth-restored >/dev/null || return 1 2>/dev/null || exit 1
ACTIVE=$(python3 -c "import json; print(json.load(open('$HOME/.claude/scripts/account-rotation/state.json')).get('activeAccount',''))" 2>/dev/null)
echo "✅ OAuth mode restored for this shell + persisted to settings.json"
echo " models : (subscription catalog — no hardcoded IDs)"
-echo " route : CRS OAuth"
+echo " route : OAuth"
echo " active account: ${ACTIVE:-(unknown)}"
if [[ -f ~/.claude/.bedrock-fallback.json ]]; then
echo " ⚠ Bedrock sentinel still present — clearing"
diff --git a/claude-ops/scripts/crsproxy-reauth/bu_reauth.py b/claude-ops/scripts/crsproxy-reauth/bu_reauth.py
deleted file mode 100644
index f4537f4f..00000000
--- a/claude-ops/scripts/crsproxy-reauth/bu_reauth.py
+++ /dev/null
@@ -1,2054 +0,0 @@
-#!/usr/bin/env python3
-"""Browser Use Cloud Agent — OAuth re-auth for CRSProxy.
-
-Production-grade reauth script that uses Browser Use Cloud to automate
-OAuth flows for Claude, xAI, and Codex providers on the proxy host.
-
-Features:
- - Proper timeout handling on all Browser Use API calls (never hang).
- - Always stops browser sessions on completion/timeout.
- - JS Navigation API callback interception (window.__CB__).
- - PTY paste of callback URL into cli-proxy-api stdin.
- - Sanitizes all URLs in logs using sanitize_url().
- - Never prints tokens, codes, magic links, or cookies.
- - Supports all OAuth providers (Claude, xAI, Codex) with provider-specific
- callback ports.
- - Structured exit codes: 0=success, 1=failure, 2=captcha.
- - Isolated candidate validation before atomic activation.
- - Stale auth preservation when candidate fails validation.
- - Human hCaptcha checkpoint: detects captcha, emits live_view_url,
- keeps browser session alive, waits for a file-based trigger, then
- resumes with a follow-up run in the same session to click Authorize
- and capture the callback URL.
- - Serialization lease: checks /opt/crsproxy/state/crsproxy-claude-oauth-lease.json
- before starting a login, waits if held, cleans up after completion.
- - Email cooldown: tracks verification code send timestamps, enforces
- a 5-minute cooldown if >3 codes are sent in 5 minutes, logs
- cooldown events without exposing the email address or code values.
-
-Usage:
- # Full reauth for a Claude account
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -gog-account [email protected]
-
- # Dry run (validates setup, no browser runs)
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -dry-run
-
- # xAI reauth
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider xai -email [email protected] -gog-account [email protected]
-
- # Codex reauth
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider codex -email [email protected] -gog-account [email protected]
-
- # Validate an existing auth file (no reauth, no browser)
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -validate-only
-
- # Validate and activate if valid
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -validate-only -activate
-
- # Validate metadata only (skip canary request)
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -validate-only -skip-canary
-
- # Resume from a hCaptcha checkpoint (after the operator solved the captcha)
- sudo -u crsproxy /opt/crsproxy/venv/bin/python /opt/crsproxy/bu_reauth.py \\
- -provider claude -email [email protected] -checkpoint-resume
-
- # When hCaptcha is detected, the script emits the live_view_url and waits.
- # After solving the captcha in the browser, create the trigger file:
- # touch /opt/crsproxy/state/bu_reauth_checkpoint_trigger
- # The script will then create a follow-up run and complete the flow.
-"""
-
-import argparse
-import atexit
-import fcntl
-import hashlib
-import json
-import os
-import re
-import signal
-import socket
-import subprocess
-import sys
-import tempfile
-import threading
-import time
-import uuid
-from datetime import datetime, timedelta, timezone
-from pathlib import Path
-
-import requests
-
-# ---------------------------------------------------------------------------
-# Paths / constants
-# ---------------------------------------------------------------------------
-HUB_BIN = "/opt/crsproxy/cli-proxy-api"
-HUB_CONFIG = "/opt/crsproxy/config.yaml"
-AUTH_DIR = Path("/opt/crsproxy/auths")
-HOME_DIR = Path("/opt/crsproxy")
-BU_API_BASE = "https://api.browser-use.com/api/v4"
-BU_MODEL = "gpt-5.6-luna"
-STATE_DIR = Path(os.environ.get("CRSPROXY_STATE_DIR", "/opt/crsproxy/state"))
-LOG_DIR = STATE_DIR
-LEASE_FILE = STATE_DIR / "crsproxy-claude-oauth-lease.json"
-
-# Exit codes
-EXIT_SUCCESS = 0
-EXIT_FAILURE = 1
-EXIT_CAPTCHA = 2
-
-# Timeouts (seconds)
-API_TIMEOUT = 30 # per HTTP request
-RUN_POLL_INTERVAL = 3 # poll run status every 3s
-RUN_POLL_TIMEOUT = 120 # max wait for a single agent run
-OAUTH_URL_TIMEOUT = 60 # max wait for OAuth URL from cli-proxy-api
-GMAIL_POLL_TIMEOUT = 180 # max wait for Gmail code/link
-PTY_PASTE_WAIT = 5 # wait after pasting callback URL
-AUTH_FILE_WAIT = 10 # wait for auth file to appear after paste
-TOTAL_TIMEOUT = 300 # overall script timeout (5 minutes)
-LEASE_STALE_SECONDS = 600 # lease older than 10 min is stale
-
-# Human checkpoint (hCaptcha) configuration
-CHECKPOINT_FILE = STATE_DIR / "bu_reauth_checkpoint.json"
-CHECKPOINT_TRIGGER = STATE_DIR / "bu_reauth_checkpoint_trigger"
-CHECKPOINT_POLL_INTERVAL = 5 # seconds between trigger polls
-CHECKPOINT_TIMEOUT = 600 # 10 minutes for human to solve captcha
-
-# Email cooldown configuration
-EMAIL_COOLDOWN_FILE = STATE_DIR / "crsproxy-email-cooldown.json"
-EMAIL_COOLDOWN_WINDOW = 300 # 5 min sliding window for counting code sends
-EMAIL_COOLDOWN_THRESHOLD = 3 # max 3 code sends before cooldown triggers
-EMAIL_COOLDOWN_DURATION = 300 # 5 min cooldown when threshold exceeded
-
-# Provider configuration: callback ports, login flags, auth URL patterns.
-PROVIDERS = {
- "claude": {
- "login_flag": "-claude-login",
- "callback_port": 54545,
- "callback_url_re": r"http://localhost:54545/callback\?[^\s\"']+",
- "auth_url_re": r"https://claude\.ai/oauth/authorize\?[^\s]+",
- "auth_file_prefix": "claude",
- "email_sender": "anthropic.com",
- "domain": "claude.ai",
- "canary_model": "claude-sonnet-5",
- },
- "xai": {
- "login_flag": "-xai-login",
- "callback_port": 51121,
- "callback_url_re": r"http://localhost:51121/[^\s\"']*callback\?[^\s\"']+",
- "auth_url_re": r"https://accounts\.x\.ai/oauth/authorize\?[^\s]+",
- "auth_file_prefix": "xai",
- "email_sender": "[email protected]",
- "domain": "accounts.x.ai",
- "canary_model": "grok-4.5",
- },
- "codex": {
- "login_flag": "-codex-login",
- "callback_port": 1455,
- "callback_url_re": r"http://localhost:1455/auth/callback\?[^\s\"']+",
- "auth_url_re": r"https://auth\.openai\.com/oauth/authorize\?[^\s]+",
- "auth_file_prefix": "codex",
- "email_sender": "[email protected]",
- "domain": "auth.openai.com",
- "canary_model": "o4-mini",
- },
-}
-
-CANARY_EXPECTED_KEYS = {
- "claude": ("access_token", "accessToken", "refresh_token", "refreshToken"),
- "xai": ("access_token", "accessToken", "refresh_token", "refreshToken", "token"),
- "codex": ("access_token", "accessToken", "refresh_token", "refreshToken", "token"),
-}
-
-# JS to inject before clicking Authorize. Intercepts the navigation to the
-# localhost callback URL and stores it in window.__CB__ so the agent can
-# retrieve it after clicking. The port is templated per provider.
-INTERCEPT_JS_TEMPLATE = """
-window.__CB__=null;
-if(navigation){{
- navigation.addEventListener('navigate',e=>{{
- const u=e.destination.url;
- if(u.includes('localhost:{port}')||u.includes('callback')){{
- window.__CB__=u;
- e.preventDefault();
- }}
- }});
-}}else{{
- window.addEventListener('beforeunload',e=>{{
- if(location.href.includes('callback'))window.__CB__=location.href;
- }});
-}}
-"""
-
-# ---------------------------------------------------------------------------
-# Logging — sanitized, no secrets
-# ---------------------------------------------------------------------------
-_log_file = None
-
-
-def log(msg: str):
- """Log a message to stdout and the log file. Never log secrets."""
- line = f"[{time.strftime('%H:%M:%S')}] {msg}"
- print(line, flush=True)
- if _log_file:
- try:
- _log_file.write(line + "\n")
- _log_file.flush()
- except Exception as e:
- print(f"[WARN] File logging failed: {e}", file=sys.stderr)
-
-
-def sanitize_url(raw: str) -> str:
- """Strip query parameters and fragments from a URL to remove secrets.
-
- Returns scheme://netloc/path only, or empty string on error.
- """
- try:
- from urllib.parse import urlparse
- parsed = urlparse(raw)
- if parsed.scheme in ("http", "https"):
- return f"{parsed.scheme}://{parsed.netloc}{parsed.path}"
- except Exception:
- # Invalid URLs are treated as unsafe to log.
- return ""
- return ""
-
-
-def safe_email(email: str) -> str:
- """Redact the local part of an email for logging (e.g. i***@domain.com)."""
- if "@" not in email:
- return "***"
- local, domain = email.split("@", 1)
- if len(local) <= 1:
- return f"***@{domain}"
- return f"{local[0]}***@{domain}"
-
-
-# ---------------------------------------------------------------------------
-# Candidate canary request
-# ---------------------------------------------------------------------------
-def _expected_auth_content(auth: dict, provider: str) -> bool:
- """Require provider-specific non-empty credential material."""
- keys = CANARY_EXPECTED_KEYS.get(provider, ())
- if any(isinstance(auth.get(key), str) and auth[key].strip() for key in keys):
- return True
- nested = auth.get("claudeAiOauth") if provider == "claude" else None
- return bool(isinstance(nested, dict) and any(
- isinstance(nested.get(key), str) and nested[key].strip()
- for key in ("accessToken", "refreshToken")))
-
-
-def _canary_status_rejection(status: int) -> str:
- """Classify a failed candidate-only inference result."""
- if status in (401, 403):
- return f"candidate authentication rejected (HTTP {status})"
- if status == 429:
- return "candidate rate-limited (HTTP 429)"
- if status == 503:
- return "candidate unavailable (HTTP 503)"
- if status == 400:
- return "candidate model unsupported (HTTP 400)"
- return f"candidate canary failed (HTTP {status})"
-
-
-def _reserve_loopback_port() -> int:
- """Ask the kernel for an unused loopback port."""
- with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
- sock.bind(("127.0.0.1", 0))
- return int(sock.getsockname()[1])
-
-
-def _candidate_proxy_config(auth_dir: Path, port: int, bearer: str) -> str:
- """Build a minimal candidate-only CLIProxyAPI configuration."""
- quoted_auth_dir = json.dumps(str(auth_dir))
- quoted_bearer = json.dumps(bearer)
- return (
- 'host: "127.0.0.1"\n'
- f"port: {port}\n"
- f"auth-dir: {quoted_auth_dir}\n"
- "api-keys:\n"
- f" - {quoted_bearer}\n"
- "remote-management:\n"
- " allow-remote: false\n"
- ' secret-key: ""\n'
- "debug: false\n"
- "logging-to-file: false\n"
- "usage-statistics-enabled: false\n"
- )
-
-
-def _stop_candidate_proxy(process: subprocess.Popen):
- """Terminate the candidate proxy process group."""
- if process.poll() is not None:
- return
- try:
- os.killpg(process.pid, signal.SIGTERM)
- process.wait(timeout=2)
- except (OSError, subprocess.TimeoutExpired):
- try:
- os.killpg(process.pid, signal.SIGKILL)
- except OSError:
- # The process group may have exited during escalation.
- pass
- try:
- process.wait(timeout=2)
- except subprocess.TimeoutExpired:
- # Shutdown is best-effort after SIGKILL.
- pass
-
-
-def send_canary(model: str, auth_path: Path, timeout: int = 30) -> tuple[bool, str]:
- """Launch and probe a candidate-only proxy owned by this validator."""
- proxy_bin = os.environ.get("CRSPROXY_CANDIDATE_PROXY_BIN", HUB_BIN).strip()
- if not proxy_bin or not Path(proxy_bin).is_file() or not os.access(proxy_bin, os.X_OK):
- return False, "candidate proxy binary is unavailable"
-
- try:
- candidate_bytes = auth_path.read_bytes()
- except OSError as e:
- return False, f"cannot read candidate for canary: {type(e).__name__}"
- candidate_digest = hashlib.sha256(candidate_bytes).hexdigest()
- deadline = time.monotonic() + timeout
-
- with tempfile.TemporaryDirectory(prefix="crsproxy-candidate-") as tmpdir:
- root = Path(tmpdir)
- os.chmod(root, 0o700)
- isolated_dir = root / "auths"
- isolated_dir.mkdir(mode=0o700)
- isolated_auth = isolated_dir / auth_path.name
- with open(isolated_auth, "xb") as fh:
- fh.write(candidate_bytes)
- fh.flush()
- os.fsync(fh.fileno())
- os.chmod(isolated_auth, 0o600)
- if hashlib.sha256(isolated_auth.read_bytes()).hexdigest() != candidate_digest:
- return False, "candidate canary copy digest mismatch"
-
- port = _reserve_loopback_port()
- bearer = f"candidate-{uuid.uuid4().hex}"
- config_path = root / "config.yaml"
- config_path.write_text(_candidate_proxy_config(isolated_dir, port, bearer), encoding="utf-8")
- os.chmod(config_path, 0o600)
- env = {
- "HOME": str(root),
- "PATH": os.environ.get("PATH", ""),
- "TMPDIR": str(root),
- }
- process = None
- try:
- process = subprocess.Popen(
- [proxy_bin, "-config", str(config_path)],
- cwd=root,
- env=env,
- stdin=subprocess.DEVNULL,
- stdout=subprocess.DEVNULL,
- stderr=subprocess.DEVNULL,
- start_new_session=True,
- )
- endpoint = f"http://127.0.0.1:{port}/v1/chat/completions"
- headers = {"Authorization": f"Bearer {bearer}", "Content-Type": "application/json"}
- response = None
- while time.monotonic() < deadline:
- if process.poll() is not None:
- return False, f"candidate proxy exited before readiness (exit {process.returncode})"
- try:
- response = requests.post(
- endpoint,
- headers=headers,
- json={
- "model": model,
- "messages": [{"role": "user", "content": "Reply with OK."}],
- "max_tokens": 8,
- },
- timeout=min(2, max(0.05, deadline - time.monotonic())),
- )
- break
- except requests.RequestException:
- time.sleep(0.05)
- if response is None:
- return False, "candidate canary timed out"
- if response.status_code != 200:
- return False, _canary_status_rejection(response.status_code)
- try:
- result = response.json()
- content = result["choices"][0]["message"]["content"]
- except (ValueError, KeyError, IndexError, TypeError):
- return False, "candidate canary returned invalid model response"
- if not isinstance(content, str) or not content.strip():
- return False, "candidate canary returned HTTP 200 without model content"
- try:
- current_digest = hashlib.sha256(auth_path.read_bytes()).hexdigest()
- except OSError:
- return False, "candidate changed during canary"
- if current_digest != candidate_digest:
- return False, "candidate changed during canary"
- return True, "candidate canary passed"
- except OSError as e:
- return False, f"candidate proxy unavailable: {type(e).__name__}"
- finally:
- if process is not None:
- _stop_candidate_proxy(process)
-
-
-# ---------------------------------------------------------------------------
-# Isolated candidate validation
-# ---------------------------------------------------------------------------
-def validate_candidate(auth_path: Path, expected_email: str,
- expected_type: str, canary_model: str = "",
- skip_canary: bool = False) -> tuple[bool, str]:
- """Validate a candidate auth file before activation.
-
- Performs four checks:
- 1. auth['email'] matches the target email.
- 2. auth['type'] matches the expected provider type.
- 3. auth['expired'] is more than 24 hours from now.
- 4. A canary request for the account's model succeeds (proxy health).
-
- This function is independently callable — it does not modify the auth
- file or any other state. It only reads the candidate and sends a
- canary request through the proxy.
-
- Args:
- auth_path: Path to the candidate auth file.
- expected_email: The target account email.
- expected_type: The expected provider type (claude/xai/codex).
- canary_model: Model name for the canary request (empty to skip).
- skip_canary: If True, skip the canary request (metadata-only mode).
-
- Returns:
- (True, "all checks passed") if every check passes.
- (False, "rejection reason") if any check fails.
- """
- # --- Read the candidate auth file ---
- try:
- auth = json.loads(auth_path.read_text())
- except Exception as e:
- return False, f"cannot read auth file: {e}"
-
- # --- Check 1: Email matches target ---
- actual_email = auth.get("email", "")
- if actual_email != expected_email:
- return False, (f"email mismatch: expected {safe_email(expected_email)}, "
- f"got {safe_email(actual_email)}")
-
- # --- Check 2: Provider type matches expected ---
- actual_type = auth.get("type", "")
- if actual_type != expected_type:
- return False, f"type mismatch: expected {expected_type}, got {actual_type}"
-
- # --- Check 3: Expiry is >24h from now ---
- expired_str = auth.get("expired", "")
- if not expired_str:
- return False, "no 'expired' field in auth file"
- try:
- # Handle ISO 8601 with optional Z suffix
- expired_dt = datetime.fromisoformat(
- expired_str.replace("Z", "+00:00"))
- now = datetime.now(timezone.utc)
- if expired_dt <= now + timedelta(hours=24):
- return False, f"token expires within 24h: {expired_str}"
- except Exception as e:
- return False, f"cannot parse expiry '{expired_str}': {e}"
-
- # --- Check 4: Provider credential material is present ---
- if not _expected_auth_content(auth, expected_type):
- return False, "candidate auth file has no non-empty provider credential"
-
- # --- Check 5: Candidate-specific isolated canary request ---
- if canary_model and not skip_canary:
- passed, reason = send_canary(canary_model, auth_path)
- if not passed:
- return False, reason
-
- return True, "all checks passed"
-
-
-# ---------------------------------------------------------------------------
-# Atomic activation
-# ---------------------------------------------------------------------------
-def activate_auth_file(auth_path: Path) -> bool:
- """Atomically activate an auth file by setting disabled=false.
-
- Uses a temp file + os.rename for atomicity. The auth directory
- hot-reloads, so no service restart is needed.
-
- Returns True on success, False on failure.
- """
- try:
- orig_mode = auth_path.stat().st_mode & 0o777
- auth = json.loads(auth_path.read_text())
- auth["disabled"] = False
- tmp_path = auth_path.with_suffix(".tmp")
- with open(tmp_path, "w", encoding="utf-8") as fh:
- fh.write(json.dumps(auth, indent=2))
- fh.flush()
- os.fsync(fh.fileno())
- os.chmod(tmp_path, orig_mode)
- os.replace(tmp_path, auth_path)
- log(f"Auth file activated: {auth_path.name}")
- return True
- except Exception as e:
- log(f"Activation error for {auth_path.name}: {e}")
- return False
-
-
-def disable_auth_file(auth_path: Path) -> bool:
- """Atomically disable an auth file by setting disabled=true.
-
- Uses a temp file + os.rename for atomicity.
- Returns True on success, False on failure.
- """
- try:
- orig_mode = auth_path.stat().st_mode & 0o777
- auth = json.loads(auth_path.read_text())
- auth["disabled"] = True
- tmp_path = auth_path.with_suffix(".tmp")
- with open(tmp_path, "w", encoding="utf-8") as fh:
- fh.write(json.dumps(auth, indent=2))
- fh.flush()
- os.fsync(fh.fileno())
- os.chmod(tmp_path, orig_mode)
- os.replace(tmp_path, auth_path)
- log(f"Auth file disabled: {auth_path.name}")
- return True
- except Exception as e:
- log(f"Disable error for {auth_path.name}: {e}")
- return False
-
-
-# ---------------------------------------------------------------------------
-# Serialization lease
-# ---------------------------------------------------------------------------
-def _lease_guard_path() -> Path:
- return LEASE_FILE.with_name(LEASE_FILE.name + ".guard")
-
-
-def _with_lease_guard(action):
- """Serialize lease record replacement and release across processes."""
- _ensure_state_dir(LEASE_FILE.parent)
- fd = os.open(_lease_guard_path(), os.O_RDWR | os.O_CREAT, 0o600)
- try:
- fcntl.flock(fd, fcntl.LOCK_EX)
- return action()
- finally:
- fcntl.flock(fd, fcntl.LOCK_UN)
- os.close(fd)
-
-
-def _read_lease(path: Path | None = None) -> dict:
- """Read a lease record, returning an empty dict for missing/corrupt data."""
- try:
- return json.loads((path or LEASE_FILE).read_text())
- except (OSError, json.JSONDecodeError, TypeError):
- return {}
-
-
-def _pid_alive(pid: int) -> bool:
- if pid <= 0:
- return False
- try:
- os.kill(pid, 0)
- return True
- except OSError:
- return False
-
-
-def _write_lease(data: dict):
- tmp = LEASE_FILE.with_name(f"{LEASE_FILE.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
- fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
- try:
- with os.fdopen(fd, "w", encoding="utf-8") as fh:
- json.dump(data, fh)
- fh.flush()
- os.fsync(fh.fileno())
- os.replace(tmp, LEASE_FILE)
- except Exception:
- try:
- tmp.unlink()
- except OSError:
- # Preserve the original lease-write error.
- pass
- raise
-
-
-def _try_acquire_or_reclaim(provider: str, lease_id: str) -> str:
- now = time.time()
- current = _read_lease()
- if current:
- renewed_at = float(current.get("renewed_at", current.get("timestamp", 0)) or 0)
- owner_alive = _pid_alive(int(current.get("pid", 0) or 0))
- if owner_alive and now - renewed_at < LEASE_STALE_SECONDS:
- return "held"
- data = {
- "provider": provider,
- "timestamp": now,
- "renewed_at": now,
- "pid": os.getpid(),
- "lease_id": lease_id,
- }
- _write_lease(data)
- return "acquired"
-
-
-def acquire_lease(provider: str, max_wait: int = 120) -> str | None:
- """Acquire the serialization lease and return its owner lease ID."""
- t0 = time.monotonic()
- first_check = True
- lease_id = uuid.uuid4().hex
- while True:
- outcome = _with_lease_guard(
- lambda: _try_acquire_or_reclaim(provider, lease_id))
- if outcome == "acquired":
- log(f"Lease acquired for {provider}")
- return lease_id
- if first_check:
- data = _read_lease()
- age = time.time() - float(
- data.get("renewed_at", data.get("timestamp", 0)) or 0)
- log(f"Lease held by {data.get('provider','?')} "
- f"({int(max(age, 0))}s old) — waiting up to {max_wait}s")
- first_check = False
- if time.monotonic() - t0 >= max_wait:
- log(f"Lease wait timed out after {max_wait}s — "
- "another login may still be in progress")
- return None
- time.sleep(min(0.1, max(0.01, max_wait / 20)))
-
-
-def renew_lease(lease_id: str | None) -> bool:
- """Renew only the current owner's lease record."""
- if not lease_id:
- return False
-
- def renew():
- data = _read_lease()
- if data.get("lease_id") != lease_id:
- return False
- data["renewed_at"] = time.time()
- _write_lease(data)
- return True
-
- return _with_lease_guard(renew)
-
-
-def release_lease(lease_id: str | None) -> bool:
- """Release atomically while holding the owner-record guard."""
- if not lease_id:
- return False
-
- def release():
- data = _read_lease()
- if data.get("lease_id") != lease_id:
- return False
- try:
- LEASE_FILE.unlink()
- except FileNotFoundError:
- return False
- return True
-
- try:
- released = _with_lease_guard(release)
- if released:
- log("Lease released")
- else:
- log("Lease release skipped — ownership changed")
- return released
- except Exception as e:
- log(f"Lease release error: {e}")
- return False
-
-
-# ---------------------------------------------------------------------------
-# Email cooldown tracking
-# ---------------------------------------------------------------------------
-def _load_cooldown_state() -> dict:
- """Load email cooldown state from file."""
- try:
- if EMAIL_COOLDOWN_FILE.exists():
- return json.loads(EMAIL_COOLDOWN_FILE.read_text())
- except Exception as e:
- log(f"Cooldown state load error: {e}")
- return {"send_timestamps": [], "cooldown_until": 0}
-
-
-def _save_cooldown_state(state: dict):
- """Save email cooldown state to file."""
- try:
- EMAIL_COOLDOWN_FILE.write_text(json.dumps(state))
- except Exception as e:
- log(f"Cooldown state save error: {e}")
-
-
-def record_email_send():
- """Record a verification code send timestamp.
-
- Called when a new verification code is found in Gmail. Prunes
- timestamps older than the cooldown window. Never logs the email
- address or code value.
- """
- state = _load_cooldown_state()
- now = time.time()
- state["send_timestamps"] = [
- ts for ts in state.get("send_timestamps", [])
- if now - ts < EMAIL_COOLDOWN_WINDOW
- ]
- state["send_timestamps"].append(now)
- _save_cooldown_state(state)
- count = len(state["send_timestamps"])
- log(f"Email send recorded (count in window: {count})")
-
-
-def check_email_cooldown() -> tuple[bool, int]:
- """Check if we're in an email cooldown period.
-
- Returns (is_in_cooldown, seconds_remaining).
- """
- state = _load_cooldown_state()
- now = time.time()
- cooldown_until = state.get("cooldown_until", 0)
- if cooldown_until > now:
- return True, int(cooldown_until - now)
- return False, 0
-
-
-def enforce_email_cooldown() -> bool:
- """Check and enforce email cooldown after a code entry failure.
-
- If more than EMAIL_COOLDOWN_THRESHOLD codes have been sent in the
- last EMAIL_COOLDOWN_WINDOW seconds, sets a cooldown of
- EMAIL_COOLDOWN_DURATION and blocks until it expires.
-
- Returns True if a cooldown was enforced (waited), False if no
- cooldown was needed. Logs cooldown events without exposing the
- email address or code values.
- """
- state = _load_cooldown_state()
- now = time.time()
-
- # Prune old timestamps
- state["send_timestamps"] = [
- ts for ts in state.get("send_timestamps", [])
- if now - ts < EMAIL_COOLDOWN_WINDOW
- ]
-
- # Check if we're already in a cooldown
- cooldown_until = state.get("cooldown_until", 0)
- if cooldown_until > now:
- remaining = int(cooldown_until - now)
- log(f"Email cooldown active — {remaining}s remaining "
- f"(threshold: {EMAIL_COOLDOWN_THRESHOLD} sends per "
- f"{EMAIL_COOLDOWN_WINDOW}s)")
- log("Waiting for cooldown to expire before retrying...")
- time.sleep(remaining + 1)
- log("Email cooldown expired — proceeding with retry")
- state["cooldown_until"] = 0
- _save_cooldown_state(state)
- return True
-
- # Check if threshold is exceeded
- count = len(state["send_timestamps"])
- if count > EMAIL_COOLDOWN_THRESHOLD:
- state["cooldown_until"] = now + EMAIL_COOLDOWN_DURATION
- _save_cooldown_state(state)
- log(f"Email cooldown triggered: {count} sends in "
- f"{EMAIL_COOLDOWN_WINDOW}s window — waiting "
- f"{EMAIL_COOLDOWN_DURATION}s")
- log("Cooldown event logged (email and code values redacted)")
- time.sleep(EMAIL_COOLDOWN_DURATION)
- log("Email cooldown expired — proceeding with retry")
- state["cooldown_until"] = 0
- _save_cooldown_state(state)
- return True
-
- log(f"No cooldown needed ({count} sends in window)")
- return False
-
-
-def clear_email_cooldown():
- """Clear email cooldown state. Called on successful completion."""
- try:
- EMAIL_COOLDOWN_FILE.unlink(missing_ok=True)
- log("Email cooldown state cleared")
- except Exception as e:
- log(f"Email cooldown clear error: {e}")
-
-
-# ---------------------------------------------------------------------------
-# Browser Use Cloud API client
-# ---------------------------------------------------------------------------
-class BrowserUseClient:
- """Thin wrapper around the Browser Use Cloud REST API with timeouts."""
-
- def __init__(self, api_key: str, base_url: str = BU_API_BASE):
- self.base = base_url
- self.headers = {
- "X-Browser-Use-API-Key": api_key,
- "Content-Type": "application/json",
- }
- self._browser_sessions: list[str] = []
- self._runs: list[str] = []
- self.keep_browser_alive = False # set True during human checkpoint
-
- def _request(self, method: str, path: str, **kwargs) -> requests.Response:
- """Make an HTTP request with a guaranteed timeout."""
- kwargs.setdefault("timeout", API_TIMEOUT)
- url = f"{self.base}{path}"
- r = requests.request(method, url, headers=self.headers, **kwargs)
- r.raise_for_status()
- return r
-
- def create_run(self, task: str, session_id: str = "",
- profile_id: str = "") -> dict:
- """Create a new Browser Use agent run."""
- payload = {"task": task, "model": BU_MODEL}
- if session_id:
- payload["sessionId"] = session_id
- if profile_id:
- payload.setdefault("browserSettings", {})["profileId"] = profile_id
- r = self._request("POST", "/runs", json=payload)
- data = r.json()
- run_id = data.get("id", "")
- if run_id:
- self._runs.append(run_id)
- return data
-
- def get_run_status(self, run_id: str) -> str:
- """Get the status of a run."""
- r = self._request("GET", f"/runs/{run_id}/status")
- return r.json().get("status", "?")
-
- def get_run(self, run_id: str) -> dict:
- """Get full run details."""
- r = self._request("GET", f"/runs/{run_id}")
- return r.json()
-
- def get_run_events(self, run_id: str, limit: int = 50) -> list:
- """Get events for a run."""
- r = self._request("GET", f"/runs/{run_id}/events",
- params={"limit": limit})
- return r.json().get("events", [])
-
- def cancel_run(self, run_id: str):
- """Cancel a run (best effort)."""
- try:
- self._request("PATCH", f"/runs/{run_id}",
- json={"action": "cancel"})
- except Exception as e:
- log(f"Run cancellation error: {e}")
-
- def stop_browser(self, browser_id: str):
- """Stop a browser session via PATCH /api/v4/browsers/{id}."""
- if not browser_id:
- return
- if self.keep_browser_alive:
- log(f"Browser session kept alive (checkpoint): {browser_id[:12]}...")
- return
- try:
- self._request("PATCH", f"/browsers/{browser_id}",
- json={"action": "stop"})
- log(f"Browser session stopped: {browser_id[:12]}...")
- except Exception as e:
- log(f"Browser stop error for {browser_id[:12]}: {e}")
- finally:
- if browser_id in self._browser_sessions:
- self._browser_sessions.remove(browser_id)
-
- def list_browsers(self) -> list:
- """List all browser sessions from the Browser Use API.
-
- The API returns a paginated response dict:
- {"items": [...], "totalItems": N, "pageNumber": 1, "pageSize": 20}
- This method normalizes that to the items list so callers can
- iterate browser session dicts directly.
- """
- try:
- r = self._request("GET", "/browsers")
- if not r.content:
- return []
- data = r.json()
- # Normalize paginated response dict to its items array
- if isinstance(data, dict) and "items" in data:
- return data["items"]
- # Already a plain list
- if isinstance(data, list):
- return data
- # Unknown shape — return empty rather than iterating dict keys
- log(f"list_browsers: unexpected response shape: {type(data).__name__}")
- return []
- except Exception as e:
- log(f"list_browsers error: {e}")
- return []
-
- def stop_all_browsers(self):
- """Stop all tracked browser sessions.
-
- Stops sessions tracked via _browser_sessions (from run events)
- and also queries the API for any running sessions we might
- have missed. Skips sessions that are already stopped to avoid
- unnecessary API calls.
- """
- if self.keep_browser_alive:
- log("[CLEANUP] Browser sessions kept alive for checkpoint")
- return
- for bid in list(self._browser_sessions):
- self.stop_browser(bid)
- # Also check for any running sessions we might have missed
- try:
- browsers = self.list_browsers()
- for b in browsers:
- if not isinstance(b, dict):
- continue
- bid = b.get("id", "")
- if not bid:
- continue
- # Skip already-stopped sessions to avoid unnecessary API calls
- status = b.get("status", "")
- if status in ("stopped", "closed", "finished"):
- continue
- self.stop_browser(bid)
- except Exception as e:
- log(f"stop_all_browsers: error listing/stopping browsers: {e}")
-
- def wait_for_run(self, run_id: str, timeout: int = RUN_POLL_TIMEOUT,
- track_browser: bool = True) -> dict:
- """Poll a run until it completes, times out, or is cancelled.
-
- Tracks browser session IDs from events for cleanup.
- Returns the full run dict on completion, or {"status": "timeout"} on timeout.
- """
- t0 = time.time()
- while time.time() - t0 < timeout:
- try:
- status = self.get_run_status(run_id)
- if status in ("completed", "failed", "cancelled"):
- return self.get_run(run_id)
- # Poll events to track browser sessions
- if track_browser:
- events = self.get_run_events(run_id, limit=10)
- for e in events:
- ed = e.get("data", {})
- if e.get("type") == "browser.ready":
- bid = ed.get("browser_session_id", "")
- if bid and bid not in self._browser_sessions:
- self._browser_sessions.append(bid)
- log(f"Browser session started: {bid[:12]}...")
- except Exception as e:
- log(f"Poll error: {e}")
- time.sleep(RUN_POLL_INTERVAL)
-
- log(f"Run {run_id[:12]} timed out after {timeout}s — cancelling")
- self.cancel_run(run_id)
- return {"status": "timeout", "result": None}
-
- def get_live_view_url(self, run_id: str) -> str:
- """Extract the live view URL from run events (for human captcha checkpoint)."""
- try:
- events = self.get_run_events(run_id, limit=100)
- for e in events:
- ed = e.get("data", {})
- if e.get("type") == "browser.ready":
- return ed.get("live_view_url", "")
- except Exception as e:
- log(f"Live view extraction error: {e}")
- return ""
-
-
-# ---------------------------------------------------------------------------
-# cli-proxy-api login process management
-# ---------------------------------------------------------------------------
-def start_login(provider: str, log_path: Path) -> subprocess.Popen:
- """Start cli-proxy-api login process with a PTY for interactive callback."""
- meta = PROVIDERS[provider]
- cmd = [
- HUB_BIN, "-config", HUB_CONFIG,
- meta["login_flag"], "-no-browser",
- "-oauth-callback-port", str(meta["callback_port"]),
- ]
- env = os.environ.copy()
- env["HOME"] = str(HOME_DIR)
-
- import pty
- master_fd, slave_fd = pty.openpty()
- out = open(log_path, "a", encoding="utf-8")
- try:
- proc = subprocess.Popen(
- cmd,
- cwd=str(HOME_DIR),
- stdin=slave_fd,
- stdout=out,
- stderr=subprocess.STDOUT,
- env=env,
- start_new_session=True,
- )
- finally:
- os.close(slave_fd)
- out.close()
- proc._pty_master = master_fd
- return proc
-
-
-def get_oauth_url(log_path: Path, provider: str,
- timeout: int = OAUTH_URL_TIMEOUT) -> str | None:
- """Wait for the OAuth URL to appear in the login log."""
- meta = PROVIDERS[provider]
- pattern = meta["auth_url_re"]
- t0 = time.time()
- while time.time() - t0 < timeout:
- try:
- text = log_path.read_text(errors="ignore")
- matches = re.findall(pattern, text)
- if matches:
- return matches[-1]
- except (OSError, UnicodeDecodeError):
- # The login log may not exist yet or may be mid-write; retry.
- pass
- time.sleep(1)
- return None
-
-
-def paste_callback_url(proc: subprocess.Popen, url: str) -> bool:
- """Write the callback URL into the cli-proxy-api PTY stdin."""
- master_fd = getattr(proc, "_pty_master", None)
- if master_fd is None:
- log("No PTY master fd — cannot paste callback URL")
- return False
- try:
- os.write(master_fd, (url + "\n").encode())
- log(f"Callback URL pasted to cli-proxy-api PTY: {sanitize_url(url)}")
- return True
- except OSError as e:
- log(f"PTY write error: {e}")
- return False
-
-
-def cleanup_login_process(proc: subprocess.Popen):
- """Terminate the cli-proxy-api login process and close its PTY."""
- if proc is None:
- return
- try:
- if proc.poll() is None:
- proc.terminate()
- try:
- proc.wait(timeout=5)
- except subprocess.TimeoutExpired:
- proc.kill()
- proc.wait(timeout=5)
- except Exception as e:
- log(f"Login process cleanup error: {e}")
- master_fd = getattr(proc, "_pty_master", None)
- if master_fd is not None:
- try:
- os.close(master_fd)
- except OSError as e:
- log(f"PTY cleanup error: {e}")
-
-
-# ---------------------------------------------------------------------------
-# Gmail polling (delegates to reauth_hub.poll_gmail)
-# ---------------------------------------------------------------------------
-def poll_gmail_for_code(provider: str, email: str, gog_account: str,
- kind: str = "code") -> str | None:
- """Poll Gmail for a verification code or magic link via gogcli.
-
- Delegates to reauth_hub.poll_gmail to avoid duplicating the Gmail logic.
- Never prints the code/link value — only logs that it was found.
- """
- try:
- sys.path.insert(0, "/opt/crsproxy")
- from reauth_hub import poll_gmail as _poll_gmail, PROVIDERS as _RH_PROVIDERS
- except ImportError:
- log("Cannot import poll_gmail from reauth_hub — Gmail polling unavailable")
- return None
-
- meta = _RH_PROVIDERS.get(provider, {})
- seat = {
- "provider": provider,
- "email": email,
- "email_sender": meta.get("email_sender", ""),
- }
- log_path = Path("/tmp/bu_reauth_gmail.log")
- result = _poll_gmail(seat, gog_account, kind=kind, log_path=log_path,
- timeout=GMAIL_POLL_TIMEOUT)
- if result:
- log(f"Gmail {kind} found (value redacted)")
- else:
- log(f"Gmail {kind} poll timed out")
- return result
-
-
-# ---------------------------------------------------------------------------
-# Callback URL extraction
-# ---------------------------------------------------------------------------
-def extract_callback_url(run_result: dict, client: BrowserUseClient,
- provider: str) -> str | None:
- """Search run results and events for the OAuth callback URL.
-
- Never logs the full URL — uses sanitize_url() for any logging.
- """
- meta = PROVIDERS[provider]
- pattern = meta["callback_url_re"]
-
- # Search run result text
- result_text = str(run_result.get("result", "") or "")
- m = re.search(pattern, result_text)
- if m:
- log(f"Callback URL found in run result: {sanitize_url(m.group(0))}")
- return m.group(0)
-
- # Search run events
- run_id = run_result.get("id", "")
- if run_id:
- try:
- events = client.get_run_events(run_id, limit=100)
- for e in events:
- ed = e.get("data", {})
- # Check event data for callback URL
- event_str = json.dumps(ed)
- m = re.search(pattern, event_str)
- if m:
- log(f"Callback URL found in run events: {sanitize_url(m.group(0))}")
- return m.group(0)
- except Exception as e:
- log(f"Event search error: {e}")
-
- return None
-
-
-def _code_entry_failed(result_text: str) -> bool:
- """Check if a verification code entry run failed.
-
- Looks for indicators that the code was rejected, expired, or
- could not be entered. Never logs the code value itself.
- """
- text_lower = result_text.lower()
- failure_indicators = [
- "invalid code", "code is invalid", "expired",
- "incorrect", "wrong code", "code is not valid",
- "didn't work", "please try again", "try again",
- "code is wrong", "enter the code again",
- "unable to verify", "verification failed",
- "code didn't match", "that code isn't",
- ]
- return any(ind in text_lower for ind in failure_indicators)
-
-
-def detect_captcha(run_result: dict, client: BrowserUseClient) -> bool:
- """Check if the run result or events indicate a real captcha challenge."""
- result_text = str(run_result.get("result", "") or "").lower()
- # "captcha" in a sentence is not enough — look for specific indicators
- captcha_indicators = [
- "hcaptcha", "h-captcha", "newassets.hcaptcha.com",
- "captcha challenge", "solve the captcha",
- "i see a captcha", "captcha widget",
- "are you human", "verify you are human",
- ]
- for indicator in captcha_indicators:
- if indicator in result_text:
- return True
-
- # Check events for captcha-related types
- run_id = run_result.get("id", "")
- if run_id:
- try:
- events = client.get_run_events(run_id, limit=100)
- for e in events:
- ed = e.get("data", {})
- event_str = json.dumps(ed).lower()
- for indicator in captcha_indicators:
- if indicator in event_str:
- return True
- except Exception as e:
- log(f"Captcha event inspection error: {e}")
-
- return False
-
-
-# ---------------------------------------------------------------------------
-# Human captcha checkpoint
-# ---------------------------------------------------------------------------
-def _ensure_state_dir(path: Path | None = None):
- """Create the private runtime state directory."""
- state_dir = path or STATE_DIR
- state_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
- os.chmod(state_dir, 0o700)
-
-
-def write_checkpoint(session_id: str, run_id: str, browser_session_id: str,
- live_view_url: str, provider: str, email: str,
- callback_port: int):
- """Write checkpoint state to a file for human captcha solving.
-
- The private checkpoint file contains the full live_view_url, session_id,
- and reauth metadata needed for the follow-up run.
- """
- data = {
- "session_id": session_id,
- "run_id": run_id,
- "browser_session_id": browser_session_id,
- "live_view_url": live_view_url,
- "provider": provider,
- "email": email,
- "callback_port": callback_port,
- "status": "waiting",
- "timestamp": time.time(),
- }
- try:
- _ensure_state_dir(CHECKPOINT_FILE.parent)
- CHECKPOINT_FILE.write_text(json.dumps(data, indent=2))
- os.chmod(CHECKPOINT_FILE, 0o600)
- log(f"[CAPTCHA_CHECKPOINT] Checkpoint file: {CHECKPOINT_FILE}")
- except Exception as e:
- log(f"[CAPTCHA_CHECKPOINT] Could not write checkpoint file: {e}")
-
-
-def wait_for_checkpoint_signal(timeout: int = CHECKPOINT_TIMEOUT) -> bool:
- """Wait for a file-based trigger indicating the captcha is solved.
-
- Polls for the existence of CHECKPOINT_TRIGGER every CHECKPOINT_POLL_INTERVAL
- seconds. Returns True if the trigger appears, False on timeout.
- The trigger file can be created in the private state directory.
- """
- log(f"[CAPTCHA_CHECKPOINT] Waiting up to {timeout}s for trigger: {CHECKPOINT_TRIGGER}")
- t0 = time.time()
- while time.time() - t0 < timeout:
- if CHECKPOINT_TRIGGER.exists():
- log("[CAPTCHA_CHECKPOINT] Trigger file detected")
- try:
- CHECKPOINT_TRIGGER.unlink()
- except OSError as e:
- log(f"Checkpoint trigger cleanup error: {e}")
- return True
- time.sleep(CHECKPOINT_POLL_INTERVAL)
- return False
-
-
-def clear_checkpoint():
- """Clean up checkpoint and trigger files."""
- for f in (CHECKPOINT_FILE, CHECKPOINT_TRIGGER):
- try:
- f.unlink(missing_ok=True)
- except OSError as e:
- log(f"Checkpoint cleanup error for {f}: {e}")
-
-
-def handle_captcha_checkpoint(client: BrowserUseClient, run_id: str,
- session_id: str, provider: str, email: str,
- port: int) -> tuple[str | None, str]:
- """Handle the human captcha checkpoint flow.
-
- 1. Get live_view_url from the run's browser.ready event.
- 2. Emit the URL to stdout (full URL for Sam, sanitized for logs).
- 3. Keep the browser session alive (do NOT stop it).
- 4. Write a checkpoint file with session state.
- 5. Wait for a file-based trigger (touch /tmp/bu_reauth_checkpoint_trigger).
- 6. After trigger, create a follow-up run in the same session to click
- Authorize with JS interception and capture the callback URL.
- 7. Return (callback_url, session_id) or (None, session_id) on failure.
-
- The browser session is kept alive during the wait so Sam can interact
- with it via the live_view_url. After the follow-up run completes,
- the browser is released for normal cleanup.
- """
- safe = safe_email(email)
-
- # --- 1. Get live_view_url from run events ---
- live_url = client.get_live_view_url(run_id)
- if not live_url:
- log("[CAPTCHA_CHECKPOINT] Could not extract live_view_url — aborting checkpoint")
- return None, session_id
-
- # Get browser_session_id for the checkpoint file
- browser_session_id = ""
- try:
- events = client.get_run_events(run_id, limit=100)
- for e in events:
- if e.get("type") == "browser.ready":
- browser_session_id = e.get("data", {}).get(
- "browser_session_id", "")
- break
- except Exception:
- pass
-
- # --- 2. Emit the live_view_url to stdout ---
- # The live_view_url is a browser-use.com URL for Sam to access the
- # browser session. It is NOT an OAuth URL and does not contain OAuth
- # tokens or callback codes. Emit the full URL so Sam can open it.
- # Also emit a sanitized version for log monitoring.
- log(f"[CAPTCHA_CHECKPOINT] Live view: {sanitize_url(live_url)}")
- log(f"[CAPTCHA_CHECKPOINT] Provider: {provider}, Email: {safe}")
- log("[CAPTCHA_CHECKPOINT] Browser session kept alive — do NOT stop")
- log(f"[CAPTCHA_CHECKPOINT] To resume after solving: touch {CHECKPOINT_TRIGGER}")
-
- # --- 3. Keep the browser session alive ---
- client.keep_browser_alive = True
-
- # --- 4. Write checkpoint file ---
- write_checkpoint(session_id, run_id, browser_session_id, live_url,
- provider, email, port)
-
- # Extend the signal-based timeout to account for the checkpoint wait
- signal.alarm(0) # Cancel current alarm
- signal.signal(signal.SIGALRM, _timeout_handler)
- signal.alarm(TOTAL_TIMEOUT + CHECKPOINT_TIMEOUT)
-
- try:
- # --- 5. Wait for file-based trigger ---
- triggered = wait_for_checkpoint_signal(timeout=CHECKPOINT_TIMEOUT)
-
- if not triggered:
- log("[CAPTCHA_CHECKPOINT] Timeout waiting for human — aborting")
- return None, session_id
-
- log("[CAPTCHA_CHECKPOINT] Trigger received — creating follow-up run")
-
- # --- 6. Create follow-up run in the same session ---
- intercept_js = INTERCEPT_JS_TEMPLATE.format(port=port)
- task_resume = (
- f"The captcha has been solved by a human. Navigate to {oauth_url}. "
- f"Run this JavaScript in the browser console: {intercept_js} "
- "Then if you see an Authorize or Allow button, click it. "
- "Wait 3 seconds. "
- "Then run window.__CB__ in the console and report its value. "
- "If null, report the current URL. "
- "If you see another captcha, report CAPTCHA."
- )
- run_resume = client.create_run(task_resume, session_id=session_id)
- run_resume_id = run_resume.get("id", "")
- session_id = run_resume.get("sessionId", session_id)
- log(f"[CAPTCHA_CHECKPOINT] Follow-up run: {run_resume_id[:12]}...")
-
- result_resume = client.wait_for_run(run_resume_id,
- timeout=RUN_POLL_TIMEOUT)
- text_resume = str(result_resume.get("result", "") or "")
- log(f"[CAPTCHA_CHECKPOINT] Follow-up run complete: {text_resume[:200]}")
-
- # Check for captcha again in the follow-up run
- if detect_captcha(result_resume, client):
- log("[CAPTCHA_CHECKPOINT] Captcha still present after follow-up — aborting")
- return None, session_id
-
- # Extract callback URL from the follow-up run
- callback_url = extract_callback_url(result_resume, client, provider)
-
- if not callback_url:
- # Try asking the agent directly for window.__CB__
- log("[CAPTCHA_CHECKPOINT] Callback not found — asking agent for window.__CB__")
- task_cb = (
- "Run window.__CB__ in the browser console and report the "
- "exact value. If null, report the current page URL."
- )
- run_cb = client.create_run(task_cb, session_id=session_id)
- run_cb_id = run_cb.get("id", "")
- log(f"[CAPTCHA_CHECKPOINT] Callback query run: {run_cb_id[:12]}...")
- result_cb = client.wait_for_run(run_cb_id, timeout=60)
- callback_url = extract_callback_url(result_cb, client, provider)
-
- if callback_url:
- log(f"[CAPTCHA_CHECKPOINT] Callback URL extracted: {sanitize_url(callback_url)}")
- else:
- log("[CAPTCHA_CHECKPOINT] Could not extract callback URL from follow-up run")
-
- return callback_url, session_id
-
- finally:
- # Always release the browser for normal cleanup
- client.keep_browser_alive = False
- # Clean up checkpoint files
- clear_checkpoint()
- # Restore a reasonable alarm for the remaining flow
- signal.alarm(TOTAL_TIMEOUT)
-
-
-# ---------------------------------------------------------------------------
-# Core reauth flow
-# ---------------------------------------------------------------------------
-def _restore_failed_candidate(auth_file: Path, stale_backup: Path):
- """Restore the prior auth atomically, or remove a failed new candidate."""
- if stale_backup.exists():
- try:
- os.replace(stale_backup, auth_file)
- log(f"[9] Stale auth restored: {auth_file.name}")
- except Exception as e:
- log(f"[9] Stale restore error: {e}")
- else:
- log("[9] No stale backup — removing failed candidate")
- try:
- auth_file.unlink()
- except FileNotFoundError:
- # Another cleanup path already removed the candidate.
- pass
- except Exception as e:
- log(f"Failed candidate cleanup error: {e}")
-
-
-def _complete_reauth(proc: subprocess.Popen, callback_url: str,
- auth_file: Path, stale_backup: Path,
- email: str, provider: str,
- meta: dict) -> int:
- """Complete the reauth flow after a callback URL is obtained.
-
- Pastes the callback URL into the cli-proxy-api PTY, waits for the auth
- file to appear, validates the isolated candidate, and atomically
- activates it if valid.
-
- Returns EXIT_SUCCESS or EXIT_FAILURE.
- """
- # --- Paste callback URL into cli-proxy-api PTY ---
- log("[8] Pasting callback URL into cli-proxy-api PTY...")
- if not paste_callback_url(proc, callback_url):
- log("[FAIL] Could not paste callback URL into PTY")
- return EXIT_FAILURE
-
- # Wait for cli-proxy-api to process the callback and write auth file
- log(f"[8] Waiting {AUTH_FILE_WAIT}s for auth file to appear...")
- time.sleep(AUTH_FILE_WAIT)
-
- # --- Validate and activate candidate auth file ---
- if not auth_file.exists():
- log(f"[FAIL] Auth file not found at expected path: {auth_file.name}")
- return EXIT_FAILURE
-
- log(f"[9] Auth file created: {auth_file.name}")
- log("[9] Validating isolated candidate...")
-
- valid, reason = validate_candidate(
- auth_path=auth_file,
- expected_email=email,
- expected_type=provider,
- canary_model=meta.get("canary_model", ""),
- )
-
- if not valid:
- log(f"[FAIL] Candidate validation failed: {reason}")
- log("[9] Preserving stale auth — restoring from backup")
- _restore_failed_candidate(auth_file, stale_backup)
- return EXIT_FAILURE
-
- log(f"[9] Candidate validation passed: {reason}")
- log("[9] Atomically activating auth file...")
- if not activate_auth_file(auth_file):
- log("[FAIL] Could not activate auth file")
- _restore_failed_candidate(auth_file, stale_backup)
- return EXIT_FAILURE
-
- # Clean up stale backup on success
- try:
- stale_backup.unlink(missing_ok=True)
- except Exception as e:
- log(f"Stale backup cleanup error: {e}")
-
- # Clear email cooldown state on successful completion
- clear_email_cooldown()
-
- log(f"[OK] Auth file activated: {auth_file.name}")
- return EXIT_SUCCESS
-
-
-def run_reauth(provider: str, email: str, gog_account: str,
- client: BrowserUseClient, dry_run: bool = False) -> int:
- """Run the full Browser Use Cloud OAuth reauth flow.
-
- Returns exit code: 0=success, 1=failure, 2=captcha.
- """
- log_path = LOG_DIR / "bu_reauth_hub.log"
- log_path.unlink(missing_ok=True)
-
- # --- Step 1: Acquire serialization lease ---
- lease_id = acquire_lease(provider)
- if not lease_id:
- log("Could not acquire lease — another login may be in progress")
- return EXIT_FAILURE
-
- stop_heartbeat = threading.Event()
-
- def heartbeat():
- interval = max(1.0, LEASE_STALE_SECONDS / 3)
- while not stop_heartbeat.wait(interval):
- if not renew_lease(lease_id):
- log("Lease heartbeat stopped — ownership changed")
- return
-
- heartbeat_thread = threading.Thread(target=heartbeat, daemon=True)
- heartbeat_thread.start()
- try:
- return _run_reauth_inner(provider, email, gog_account, client,
- dry_run, log_path)
- finally:
- stop_heartbeat.set()
- heartbeat_thread.join(timeout=1)
- release_lease(lease_id)
-
-
-def _run_reauth_inner(provider: str, email: str, gog_account: str,
- client: BrowserUseClient, dry_run: bool,
- log_path: Path) -> int:
- """Inner reauth flow — assumes lease is already held."""
- meta = PROVIDERS[provider]
- port = meta["callback_port"]
- safe = safe_email(email)
-
- # --- Dry run ---
- if dry_run:
- log(f"[DRY RUN] Provider={provider} Email={safe} Port={port}")
- log(f"[DRY RUN] Would start: {HUB_BIN} -config {HUB_CONFIG} "
- f"{meta['login_flag']} -no-browser -oauth-callback-port {port}")
- log(f"[DRY RUN] Auth file target: {meta['auth_file_prefix']}-{email}.json")
- log("[DRY RUN] Validating Browser Use API access...")
- try:
- browsers = client.list_browsers()
- log(f"[DRY RUN] Browser Use API: OK ({len(browsers)} active sessions)")
- except Exception as e:
- log(f"[DRY RUN] Browser Use API error: {e}")
- return EXIT_FAILURE
- log("[DRY RUN] Dry run complete — no browser sessions started")
- return EXIT_SUCCESS
-
- # --- Step 1.5: Back up stale auth for preservation ---
- auth_file = AUTH_DIR / f"{meta['auth_file_prefix']}-{email}.json"
- stale_backup = auth_file.with_suffix(".stale")
- if auth_file.exists():
- try:
- stale_backup.unlink(missing_ok=True)
- stale_backup.write_text(auth_file.read_text())
- log(f"[1] Stale auth backed up: {auth_file.name}")
- except Exception as e:
- log(f"[1] Stale auth backup warning: {e}")
- else:
- log("[1] No existing auth file — no stale backup needed")
-
- # --- Step 2: Start cli-proxy-api login ---
- log(f"[1] Starting cli-proxy-api login for {provider} ({safe})")
- proc = start_login(provider, log_path)
-
- try:
- # --- Step 3: Capture OAuth URL ---
- log("[2] Waiting for OAuth URL...")
- oauth_url = get_oauth_url(log_path, provider)
- if not oauth_url:
- log("[ERROR] No OAuth URL found in login log")
- return EXIT_FAILURE
- log(f"[2] OAuth URL captured: {sanitize_url(oauth_url)}")
-
- # --- Step 4: Agent — navigate + enter email ---
- log(f"[3] Agent: navigating to OAuth URL and entering email for {safe}")
- task1 = (
- f"Go to {oauth_url}. "
- f"Enter {email} in the email field. "
- "Click Continue. Do NOT use Google sign-in. "
- "Report exactly what you see after clicking Continue."
- )
- run1 = client.create_run(task1)
- run1_id = run1.get("id", "")
- session_id = run1.get("sessionId", "")
- log(f"[3] Run created: {run1_id[:12]}... session: {session_id[:12]}...")
- result1 = client.wait_for_run(run1_id, timeout=RUN_POLL_TIMEOUT)
- text1 = str(result1.get("result", "") or "")
- log(f"[3] Run 1 complete: {text1[:200]}")
-
- # Check for captcha on first step
- if detect_captcha(result1, client):
- log("[CAPTCHA] Captcha detected during email entry")
- callback_url, session_id = handle_captcha_checkpoint(
- client, run1_id, session_id, provider, email, port)
- if callback_url:
- return _complete_reauth(proc, callback_url, auth_file,
- stale_backup, email, provider, meta)
- return EXIT_CAPTCHA
-
- # --- Step 5: Handle verification code or magic link ---
- needs_code = any(kw in text1.lower() for kw in
- ["code", "link sent", "verify", "check your email",
- "magic", "email has been sent"])
- if not needs_code:
- log("[4] No verification code/magic link needed — proceeding to Authorize")
- else:
- log(f"[4] Verification needed — polling Gmail for {safe}")
- time.sleep(5) # Give Gmail time to receive the email
-
- # Try code first, then magic link
- code = poll_gmail_for_code(provider, email, gog_account, kind="code")
- if code:
- record_email_send()
- log("[4] Verification code received (value redacted)")
-
- # Code entry with retry and email cooldown enforcement.
- # If the code is rejected, check the email send history.
- # If >3 codes have been sent in 5 minutes, enforce a
- # 5-minute cooldown before retrying with a fresh code.
- max_code_retries = 3
- text2 = ""
- for attempt in range(max_code_retries):
- task2 = (
- "Enter the verification code in the verification field. "
- "Click Continue. Report what you see."
- )
- run2 = client.create_run(task2, session_id=session_id)
- run2_id = run2.get("id", "")
- log(f"[4] Code entry run (attempt {attempt+1}/"
- f"{max_code_retries}): {run2_id[:12]}...")
- result2 = client.wait_for_run(run2_id,
- timeout=RUN_POLL_TIMEOUT)
- text2 = str(result2.get("result", "") or "")
- log(f"[4] Code entry complete: {text2[:200]}")
- session_id = run2.get("sessionId", session_id)
-
- # Check for captcha after code entry
- if detect_captcha(result2, client):
- log("[CAPTCHA] Captcha detected after code entry")
- callback_url, session_id = handle_captcha_checkpoint(
- client, run2_id, session_id, provider, email, port)
- if callback_url:
- return _complete_reauth(proc, callback_url,
- auth_file, stale_backup,
- email, provider, meta)
- return EXIT_CAPTCHA
-
- # Check if code entry failed
- if _code_entry_failed(text2):
- log(f"[4] Code entry failed on attempt {attempt+1}")
- if attempt < max_code_retries - 1:
- # Enforce email cooldown (waits if threshold
- # exceeded) then poll for a fresh code
- enforce_email_cooldown()
- log("[4] Polling for new verification code")
- code = poll_gmail_for_code(
- provider, email, gog_account, kind="code")
- if code:
- record_email_send()
- log("[4] New verification code received "
- "(value redacted)")
- else:
- log("[FAIL] No new verification code received")
- return EXIT_FAILURE
- else:
- log("[FAIL] Code entry failed after all retries")
- return EXIT_FAILURE
- else:
- break # Code entry succeeded
-
- # Check if we need a magic link (selectAccount flow)
- if any(kw in text2.lower() for kw in
- ["link sent", "magic", "selectaccount", "click the link"]):
- log("[5] Magic link flow detected — polling Gmail")
- time.sleep(5)
- magic = poll_gmail_for_code(provider, email, gog_account,
- kind="magic")
- if magic:
- log("[5] Magic link received (value redacted)")
- task3 = (
- "Navigate to the magic link URL. "
- "If you see an Authorize button, click it. "
- "Report what you see."
- )
- run3 = client.create_run(task3, session_id=session_id)
- run3_id = run3.get("id", "")
- log(f"[5] Magic link run: {run3_id[:12]}...")
- result3 = client.wait_for_run(run3_id,
- timeout=RUN_POLL_TIMEOUT)
- text3 = str(result3.get("result", "") or "")
- log(f"[5] Magic link run complete: {text3[:200]}")
- session_id = run3.get("sessionId", session_id)
- else:
- log("[5] No magic link found — cannot continue")
- return EXIT_FAILURE
- else:
- log("[4] No verification code — trying magic link...")
- magic = poll_gmail_for_code(provider, email, gog_account,
- kind="magic")
- if magic:
- log("[4] Magic link received (value redacted)")
- task2 = (
- "Navigate to the magic link URL. "
- "If you see an Authorize button, click it. "
- "Report what you see."
- )
- run2 = client.create_run(task2, session_id=session_id)
- run2_id = run2.get("id", "")
- log(f"[4] Magic link run: {run2_id[:12]}...")
- result2 = client.wait_for_run(run2_id, timeout=RUN_POLL_TIMEOUT)
- text2 = str(result2.get("result", "") or "")
- log(f"[4] Magic link run complete: {text2[:200]}")
- session_id = run2.get("sessionId", session_id)
- else:
- log("[FAIL] No verification code or magic link received")
- return EXIT_FAILURE
-
- # --- Step 6: Click Authorize with JS interception ---
- intercept_js = INTERCEPT_JS_TEMPLATE.format(port=port)
- log(f"[6] Injecting JS callback interception (port {port}) and clicking Authorize")
- task_auth = (
- f"Run this JavaScript in the browser console: {intercept_js} "
- "Then if you see an Authorize or Allow button, click it. "
- "Wait 3 seconds. "
- "Then run window.__CB__ in the console and report its value. "
- "If null, report the current URL."
- )
- run_auth = client.create_run(task_auth, session_id=session_id)
- run_auth_id = run_auth.get("id", "")
- log(f"[6] Authorize run: {run_auth_id[:12]}...")
- result_auth = client.wait_for_run(run_auth_id, timeout=RUN_POLL_TIMEOUT)
- text_auth = str(result_auth.get("result", "") or "")
- log(f"[6] Authorize run complete: {text_auth[:200]}")
-
- # Check for captcha during authorize
- if detect_captcha(result_auth, client):
- log("[CAPTCHA] Captcha detected during Authorize")
- callback_url, session_id = handle_captcha_checkpoint(
- client, run_auth_id, session_id, provider, email, port)
- if callback_url:
- return _complete_reauth(proc, callback_url, auth_file,
- stale_backup, email, provider, meta)
- return EXIT_CAPTCHA
-
- # --- Step 7: Extract callback URL ---
- log("[7] Searching for callback URL...")
- callback_url = extract_callback_url(result_auth, client, provider)
-
- if not callback_url:
- # Try asking the agent directly for window.__CB__
- log("[7] Callback not found in results — asking agent for window.__CB__")
- task_cb = (
- "Run window.__CB__ in the browser console and report the exact value. "
- "If null, report the current page URL."
- )
- run_cb = client.create_run(task_cb, session_id=session_id)
- run_cb_id = run_cb.get("id", "")
- log(f"[7] Callback query run: {run_cb_id[:12]}...")
- result_cb = client.wait_for_run(run_cb_id, timeout=60)
- text_cb = str(result_cb.get("result", "") or "")
- log(f"[7] Callback query result: {text_cb[:200]}")
-
- # Search the callback query result
- callback_url = extract_callback_url(result_cb, client, provider)
-
- if not callback_url:
- log("[FAIL] Could not extract callback URL from any run")
- return EXIT_FAILURE
-
- log(f"[7] Callback URL extracted: {sanitize_url(callback_url)}")
-
- # --- Step 8-9: Paste, validate, and activate ---
- return _complete_reauth(proc, callback_url, auth_file,
- stale_backup, email, provider, meta)
-
- except Exception as e:
- log(f"[ERROR] Unexpected error: {e}")
- return EXIT_FAILURE
- finally:
- # Always stop all browser sessions
- log("[CLEANUP] Stopping all browser sessions...")
- client.stop_all_browsers()
- # Always clean up the login process
- cleanup_login_process(proc)
-
-
-# ---------------------------------------------------------------------------
-# Signal handling for overall timeout
-# ---------------------------------------------------------------------------
-def _timeout_handler(signum, frame):
- raise TimeoutError(f"Script exceeded {TOTAL_TIMEOUT}s total timeout")
-
-
-# ---------------------------------------------------------------------------
-# Checkpoint resume
-# ---------------------------------------------------------------------------
-def _checkpoint_resume(args) -> int:
- """Resume from a hCaptcha checkpoint file.
-
- Reads /opt/crsproxy/state/bu_reauth_checkpoint.json for the session_id, provider,
- email, and callback_port. Starts a new cli-proxy-api login process
- (the original one is likely dead), creates a follow-up run in the
- same Browser Use session to click Authorize, extracts the callback
- URL, pastes it into the new login process's PTY, and completes the
- validation + activation flow.
- """
- if not CHECKPOINT_FILE.exists():
- log("[FAIL] No checkpoint file found — cannot resume")
- return EXIT_FAILURE
-
- try:
- ckpt = json.loads(CHECKPOINT_FILE.read_text())
- except Exception as e:
- log(f"[FAIL] Cannot read checkpoint file: {e}")
- return EXIT_FAILURE
-
- session_id = ckpt.get("session_id", "")
- provider = ckpt.get("provider", "")
- email = ckpt.get("email", "")
- port = ckpt.get("callback_port", 0)
-
- if not session_id or not provider or not email:
- log("[FAIL] Checkpoint file missing required fields")
- return EXIT_FAILURE
-
- if provider not in PROVIDERS:
- log(f"[FAIL] Unknown provider in checkpoint: {provider}")
- return EXIT_FAILURE
-
- meta = PROVIDERS[provider]
- safe = safe_email(email)
- log(f"=== Checkpoint resume: {provider} / {safe} ===")
- log(f"Session: {session_id[:12]}...")
-
- # Load API key
- api_key = os.environ.get("BROWSER_USE_API_KEY", "")
- if not api_key:
- env_path = Path("/opt/crsproxy/.env")
- if env_path.exists():
- for line in env_path.read_text().splitlines():
- if line.startswith("BROWSER_USE_API_KEY="):
- api_key = line.split("=", 1)[1].strip().strip('"').strip("'")
- break
- if not api_key:
- log("[FATAL] BROWSER_USE_API_KEY not found")
- return EXIT_FAILURE
-
- client = BrowserUseClient(api_key)
- atexit.register(client.stop_all_browsers)
-
- # Acquire serialization lease (prevents concurrent logins)
- lease_id = acquire_lease(provider)
- if not lease_id:
- log("[FAIL] Could not acquire lease — another login may be in progress")
- return EXIT_FAILURE
-
- # Start a new login process (the original is likely dead)
- log_path = LOG_DIR / "bu_reauth_hub.log"
- log("[1] Starting new cli-proxy-api login for checkpoint resume")
- try:
- proc = start_login(provider, log_path)
- except Exception as e:
- log(f"[ERROR] Could not start login process: {e}")
- release_lease(lease_id)
- return EXIT_FAILURE
-
- # Wait for OAuth URL (needed to keep the callback port listener alive)
- log("[2] Waiting for OAuth URL...")
- oauth_url = get_oauth_url(log_path, provider)
- if not oauth_url:
- log("[ERROR] No OAuth URL found — cannot resume login process")
- cleanup_login_process(proc)
- release_lease(lease_id)
- return EXIT_FAILURE
- log(f"[2] OAuth URL captured: {sanitize_url(oauth_url)}")
-
- # Back up stale auth
- auth_file = AUTH_DIR / f"{meta['auth_file_prefix']}-{email}.json"
- stale_backup = auth_file.with_suffix(".stale")
- if auth_file.exists():
- try:
- stale_backup.unlink(missing_ok=True)
- stale_backup.write_text(auth_file.read_text())
- log(f"[1] Stale auth backed up: {auth_file.name}")
- except Exception as e:
- log(f"[1] Stale auth backup warning: {e}")
-
- try:
- # Create follow-up run in the same session
- intercept_js = INTERCEPT_JS_TEMPLATE.format(port=port)
- log("[3] Creating follow-up run in same session")
- task_resume = (
- f"The captcha has been solved by a human. Navigate to {oauth_url}. "
- f"Run this JavaScript in the browser console: {intercept_js} "
- "Then if you see an Authorize or Allow button, click it. "
- "Wait 3 seconds. "
- "Then run window.__CB__ in the console and report its value. "
- "If null, report the current URL. "
- "If you see another captcha, report CAPTCHA."
- )
- run_resume = client.create_run(task_resume, session_id=session_id)
- run_resume_id = run_resume.get("id", "")
- log(f"[3] Follow-up run: {run_resume_id[:12]}...")
-
- result_resume = client.wait_for_run(run_resume_id,
- timeout=RUN_POLL_TIMEOUT)
- text_resume = str(result_resume.get("result", "") or "")
- log(f"[3] Follow-up run complete: {text_resume[:200]}")
-
- if detect_captcha(result_resume, client):
- log("[FAIL] Captcha still present — cannot complete")
- return EXIT_CAPTCHA
-
- callback_url = extract_callback_url(result_resume, client, provider)
-
- if not callback_url:
- log("[3] Callback not found — asking agent for window.__CB__")
- task_cb = (
- "Run window.__CB__ in the browser console and report the "
- "exact value. If null, report the current page URL."
- )
- run_cb = client.create_run(task_cb, session_id=session_id)
- result_cb = client.wait_for_run(run_cb.get("id", ""), timeout=60)
- callback_url = extract_callback_url(result_cb, client, provider)
-
- if not callback_url:
- log("[FAIL] Could not extract callback URL")
- return EXIT_FAILURE
-
- log(f"[3] Callback URL extracted: {sanitize_url(callback_url)}")
-
- return _complete_reauth(proc, callback_url, auth_file, stale_backup,
- email, provider, meta)
-
- except Exception as e:
- log(f"[ERROR] Checkpoint resume error: {e}")
- return EXIT_FAILURE
- finally:
- client.stop_all_browsers()
- cleanup_login_process(proc)
- clear_checkpoint()
- release_lease(lease_id)
-
-
-# ---------------------------------------------------------------------------
-# Main
-# ---------------------------------------------------------------------------
-def main():
- global _log_file
-
- parser = argparse.ArgumentParser(
- description="Browser Use Cloud OAuth reauth for CRSProxy"
- )
- parser.add_argument("-provider", required=True,
- choices=list(PROVIDERS.keys()),
- help="OAuth provider (claude, xai, codex)")
- parser.add_argument("-email", required=True,
- help="Account email address")
- parser.add_argument("-gog-account", default="",
- help="Gogcli account for Gmail polling")
- parser.add_argument("-dry-run", action="store_true",
- help="Validate setup without running browser flows")
- parser.add_argument("-validate-only", action="store_true",
- help="Only validate an existing auth file (no reauth). "
- "Checks email, type, expiry, and canary. "
- "Use with -activate to activate after validation.")
- parser.add_argument("-activate", action="store_true",
- help="With -validate-only: atomically activate the "
- "auth file if validation passes "
- "(sets disabled=false via temp file + rename)")
- parser.add_argument("-skip-canary", action="store_true",
- help="Skip the canary request (metadata-only validation)")
- parser.add_argument("-checkpoint-timeout", type=int, default=0,
- help="Override the human checkpoint timeout in seconds "
- "(default: 600 = 10 minutes). Used when "
- "hCaptcha is detected and a human must solve it.")
- parser.add_argument("-checkpoint-resume", action="store_true",
- help="Resume from a checkpoint file written by a "
- "previous run that detected hCaptcha. Reads "
- "/opt/crsproxy/state/bu_reauth_checkpoint.json for the "
- "session_id, then creates a follow-up run to "
- "click Authorize and complete the flow.")
- parser.add_argument("-log-file", default="",
- help="Log file path (default: /opt/crsproxy/state/bu_reauth.log)")
- args = parser.parse_args()
-
- _ensure_state_dir()
-
- # Set up log file
- log_path = args.log_file or str(LOG_DIR / "bu_reauth.log")
- with open(log_path, "a", encoding="utf-8") as log_file:
- _log_file = log_file
-
- # Override checkpoint timeout if specified
- if args.checkpoint_timeout > 0:
- global CHECKPOINT_TIMEOUT
- CHECKPOINT_TIMEOUT = args.checkpoint_timeout
-
- # Set overall timeout before all execution modes.
- signal.signal(signal.SIGALRM, _timeout_handler)
- signal.alarm(TOTAL_TIMEOUT)
-
- # --- Validate-only mode: standalone candidate validation ---
- if args.validate_only:
- meta = PROVIDERS[args.provider]
- auth_file = AUTH_DIR / f"{meta['auth_file_prefix']}-{args.email}.json"
- log(f"=== Validate-only: {args.provider} / {safe_email(args.email)} ===")
- log(f"Auth file: {auth_file.name}")
-
- if not auth_file.exists():
- log(f"[FAIL] Auth file not found: {auth_file.name}")
- return EXIT_FAILURE
-
- valid, reason = validate_candidate(
- auth_path=auth_file,
- expected_email=args.email,
- expected_type=args.provider,
- canary_model=meta.get("canary_model", ""),
- skip_canary=args.skip_canary,
- )
-
- if not valid:
- log(f"[FAIL] Validation failed: {reason}")
- return EXIT_FAILURE
-
- log(f"[OK] Validation passed: {reason}")
-
- if args.activate:
- log("Atomically activating auth file...")
- if not activate_auth_file(auth_file):
- log("[FAIL] Could not activate auth file")
- return EXIT_FAILURE
- log(f"[OK] Auth file activated: {auth_file.name}")
-
- return EXIT_SUCCESS
-
- # --- Checkpoint-resume mode: resume from a hCaptcha checkpoint ---
- if args.checkpoint_resume:
- return _checkpoint_resume(args)
-
- # Load API key from environment
- api_key = os.environ.get("BROWSER_USE_API_KEY", "")
- if not api_key:
- # Try loading from .env
- env_path = Path("/opt/crsproxy/.env")
- if env_path.exists():
- for line in env_path.read_text().splitlines():
- if line.startswith("BROWSER_USE_API_KEY="):
- api_key = line.split("=", 1)[1].strip().strip('"').strip("'")
- break
-
- if not api_key:
- log("[FATAL] BROWSER_USE_API_KEY not found in environment or .env")
- return EXIT_FAILURE
-
- if args.dry_run:
- log(f"=== DRY RUN: {args.provider} / {safe_email(args.email)} ===")
- else:
- log(f"=== Reauth: {args.provider} / {safe_email(args.email)} ===")
-
- client = BrowserUseClient(api_key)
-
- # Register cleanup to stop all browsers on any exit
- atexit.register(client.stop_all_browsers)
-
- try:
- exit_code = run_reauth(
- provider=args.provider,
- email=args.email,
- gog_account=args.gog_account,
- client=client,
- dry_run=args.dry_run,
- )
- except TimeoutError as e:
- log(f"[TIMEOUT] {e}")
- exit_code = EXIT_FAILURE
- except KeyboardInterrupt:
- log("[INTERRUPTED] User interrupted")
- exit_code = EXIT_FAILURE
- finally:
- signal.alarm(0) # Cancel the alarm
- # Ensure all browser sessions are stopped
- client.stop_all_browsers()
-
- log(f"=== Exit code: {exit_code} ===")
- return exit_code
-
-
-if __name__ == "__main__":
- sys.exit(main())
diff --git a/claude-ops/scripts/crsproxy-reauth/sync-env.sh b/claude-ops/scripts/crsproxy-reauth/sync-env.sh
deleted file mode 100755
index a67258f8..00000000
--- a/claude-ops/scripts/crsproxy-reauth/sync-env.sh
+++ /dev/null
@@ -1,71 +0,0 @@
-#!/usr/bin/env bash
-# sync-env.sh — Sync /opt/crsproxy/.env from Doppler
-#
-# Downloads all secrets from the Doppler crsproxy/prd config and writes them
-# to /opt/crsproxy/.env atomically (temp file + rename). The Doppler service
-# token is read from /opt/crsproxy/.doppler-token (mode 600, owner crsproxy).
-#
-# Usage: sudo -u crsproxy /opt/crsproxy/sync-env.sh
-#
-# Requirements:
-# - Doppler CLI installed on hub (/usr/bin/doppler)
-# - Service token stored at /opt/crsproxy/.doppler-token
-# - crsproxy user must have write and execute access to /opt/crsproxy/
-#
-# This script is the canonical way to keep /opt/crsproxy/.env in sync with
-# Doppler. No manual .env editing is needed — update the secret in Doppler
-# (or Dashlane → Doppler) and re-run this script.
-set -euo pipefail
-
-MOBILE=0
-if [[ -n "${SSH_CONNECTION:-}${SSH_CLIENT:-}${SSH_TTY:-}" || "${OPS_MOBILE:-0}" == "1" || ( "${COLUMNS:-80}" =~ ^[0-9]+$ && "${COLUMNS:-80}" -lt 80 ) ]]; then
- MOBILE=1
-fi
-
-cd /opt/crsproxy
-
-TOKEN_FILE="/opt/crsproxy/.doppler-token"
-ENV_FILE="/opt/crsproxy/.env"
-PROJECT="crsproxy"
-CONFIG="prd"
-
-if [ ! -f "$TOKEN_FILE" ]; then
- echo "ERROR: Token file $TOKEN_FILE not found" >&2
- exit 1
-fi
-if [ ! -r "$TOKEN_FILE" ] || [ ! -s "$TOKEN_FILE" ]; then
- echo "ERROR: Token file $TOKEN_FILE must be readable and non-empty" >&2
- exit 1
-fi
-if ! DOPPLER_TOKEN="$(<"$TOKEN_FILE")" || [ -z "$DOPPLER_TOKEN" ]; then
- echo "ERROR: Could not read a non-empty token from $TOKEN_FILE" >&2
- exit 1
-fi
-export DOPPLER_TOKEN
-
-# Download secrets in env format and write to a temp file
-TMP_FILE=$(mktemp /opt/crsproxy/.env.XXXXXX)
-cleanup() {
- [ -n "${TMP_FILE:-}" ] && rm -f -- "$TMP_FILE"
-}
-trap cleanup EXIT
-trap 'exit 129' HUP
-trap 'exit 130' INT
-trap 'exit 143' TERM
-doppler secrets download --project "$PROJECT" --config "$CONFIG" --format env --no-file > "$TMP_FILE"
-
-if [ ! -s "$TMP_FILE" ] || ! grep -Eq '^BROWSER_USE_API_KEY=.+$' "$TMP_FILE"; then
- echo "ERROR: Doppler output is empty or missing BROWSER_USE_API_KEY" >&2
- exit 1
-fi
-key_count=$(grep -c "=" "$TMP_FILE")
-
-# Atomic move (temp file + rename)
-chmod 600 "$TMP_FILE"
-mv "$TMP_FILE" "$ENV_FILE"
-TMP_FILE=""
-if [ "$MOBILE" -eq 1 ]; then
- echo "env synced: ${key_count} keys"
-else
- echo "Synced $ENV_FILE from Doppler $PROJECT/$CONFIG (${key_count} keys)"
-fi
diff --git a/claude-ops/scripts/crsproxy-reauth/test-python-suites.sh b/claude-ops/scripts/crsproxy-reauth/test-python-suites.sh
deleted file mode 100644
index 11ce7ca3..00000000
--- a/claude-ops/scripts/crsproxy-reauth/test-python-suites.sh
+++ /dev/null
@@ -1,9 +0,0 @@
-#!/usr/bin/env bash
-set -euo pipefail
-DIR="$(cd "$(dirname "$0")" && pwd)"
-cd "$DIR"
-
-python3 test_browser_cleanup.py
-python3 test_checkpoint.py
-python3 test_lease_cooldown.py
-python3 test_candidate_validation.py
diff --git a/claude-ops/scripts/crsproxy-reauth/test_browser_cleanup.py b/claude-ops/scripts/crsproxy-reauth/test_browser_cleanup.py
deleted file mode 100644
index c8312b3f..00000000
--- a/claude-ops/scripts/crsproxy-reauth/test_browser_cleanup.py
+++ /dev/null
@@ -1,342 +0,0 @@
-#!/usr/bin/env python3
-"""Unit tests for browser session cleanup pagination fix (VAL-REAUTH-008).
-
-Tests that list_browsers() correctly normalizes the Browser Use API
-paginated response dict to its items array, and that stop_all_browsers()
-iterates the actual browser session dicts (not dict keys) and issues
-PATCH stop requests for each running session.
-
-Run on the hub:
- sudo -u crsproxy /opt/crsproxy/venv/bin/python \
- /opt/crsproxy/test_browser_cleanup.py
-
-Run locally (syntax/logic only, no hub dependencies):
- python3 test_browser_cleanup.py
-"""
-
-import sys
-from pathlib import Path
-from unittest.mock import MagicMock, patch
-
-# Add the script directory to the path
-sys.path.insert(0, str(Path(__file__).parent))
-
-# Import the module
-import bu_reauth
-
-
-# ---------------------------------------------------------------------------
-# Real Browser Use API paginated response shape (from live API)
-# ---------------------------------------------------------------------------
-PAGINATED_RESPONSE = {
- "items": [
- {
- "id": "473ea101-49b9-4518-855c-d8f0d2035307",
- "status": "stopped",
- "liveUrl": None,
- "cdpUrl": None,
- "timeoutAt": "2026-08-13T03:03:24.128508Z",
- "startedAt": "2026-08-12T23:03:24.128508Z",
- "finishedAt": "2026-08-12T23:04:52.670817Z",
- "proxyUsedMb": "5.8607378005981443438",
- "proxyCost": "0.02861688379198312667871093750",
- "browserCost": "0.0006666666666666666666666666667",
- "agentSessionId": "3b87c9d5-e4f9-41e5-826c-342f5411c1f8",
- "recordingUrl": None,
- },
- {
- "id": "878b3fe7-5c62-40b7-a581-17e12ea77b4f",
- "status": "running",
- "liveUrl": "https://live.browser-use.com?wss=example",
- "cdpUrl": None,
- "timeoutAt": "2026-08-13T03:01:58.898232Z",
- "startedAt": "2026-08-12T23:01:58.898232Z",
- "finishedAt": None,
- "proxyUsedMb": "0.00117874145507812500",
- "proxyCost": "0.000005755573511123657226562500",
- "browserCost": "0.0003333333333333333333333333333",
- "agentSessionId": "3b87c9d5-e4f9-41e5-826c-342f5411c1f8",
- "recordingUrl": None,
- },
- {
- "id": "e0e52cb4-5a2d-4ec0-a6ac-2937bc14a347",
- "status": "running",
- "liveUrl": "https://live.browser-use.com?wss=example2",
- "cdpUrl": None,
- "timeoutAt": "2026-08-13T03:01:36.028765Z",
- "startedAt": "2026-08-12T23:01:36.028765Z",
- "finishedAt": None,
- "proxyUsedMb": "4.2314872741699216875",
- "proxyCost": "0.02066155895590782073974609375",
- "browserCost": "0.0003333333333333333333333333333",
- "agentSessionId": "3b87c9d5-e4f9-41e5-826c-342f5411c1f8",
- "recordingUrl": None,
- },
- ],
- "totalItems": 3,
- "pageNumber": 1,
- "pageSize": 20,
-}
-
-
-def test_list_browsers_normalizes_paginated_dict():
- """list_browsers() returns the items array from a paginated response dict.
-
- Before the fix, list_browsers() returned the raw dict, causing
- stop_all_browsers() to iterate dict keys (strings) instead of
- browser session dicts.
- """
- client = bu_reauth.BrowserUseClient("fake-key")
-
- # Mock _request to return a paginated response
- mock_response = MagicMock()
- mock_response.content = b'{"items": [...]}'
- mock_response.json.return_value = PAGINATED_RESPONSE
- with patch.object(client, '_request', return_value=mock_response):
- result = client.list_browsers()
-
- assert isinstance(result, list), (
- f"Expected list, got {type(result).__name__}"
- )
- assert len(result) == 3, f"Expected 3 items, got {len(result)}"
- assert all(isinstance(item, dict) for item in result), (
- "All items should be dicts"
- )
- assert result[0]["id"] == "473ea101-49b9-4518-855c-d8f0d2035307"
- print("PASS: list_browsers() normalizes paginated dict to items array")
-
-
-def test_list_browsers_handles_plain_list():
- """list_browsers() handles a plain list response (non-paginated)."""
- client = bu_reauth.BrowserUseClient("fake-key")
-
- plain_list = [{"id": "abc", "status": "running"}]
- mock_response = MagicMock()
- mock_response.content = b'[...]'
- mock_response.json.return_value = plain_list
- with patch.object(client, '_request', return_value=mock_response):
- result = client.list_browsers()
-
- assert isinstance(result, list)
- assert len(result) == 1
- assert result[0]["id"] == "abc"
- print("PASS: list_browsers() handles plain list response")
-
-
-def test_list_browsers_handles_empty_response():
- """list_browsers() returns empty list for empty response."""
- client = bu_reauth.BrowserUseClient("fake-key")
-
- mock_response = MagicMock()
- mock_response.content = b''
- with patch.object(client, '_request', return_value=mock_response):
- result = client.list_browsers()
-
- assert result == [], f"Expected empty list, got {result}"
- print("PASS: list_browsers() returns empty list for empty response")
-
-
-def test_list_browsers_handles_empty_items():
- """list_browsers() returns empty list when items is empty."""
- client = bu_reauth.BrowserUseClient("fake-key")
-
- empty_paginated = {"items": [], "totalItems": 0, "pageNumber": 1, "pageSize": 20}
- mock_response = MagicMock()
- mock_response.content = b'{"items": []}'
- mock_response.json.return_value = empty_paginated
- with patch.object(client, '_request', return_value=mock_response):
- result = client.list_browsers()
-
- assert result == [], f"Expected empty list, got {result}"
- print("PASS: list_browsers() returns empty list when items is empty")
-
-
-def test_stop_all_browsers_stops_running_sessions():
- """stop_all_browsers() issues PATCH stop for each running session.
-
- Before the fix, stop_all_browsers() iterated dict keys (strings)
- and called b.get("id", "") on strings, which silently failed.
- After the fix, it iterates the items array and stops each
- running session.
- """
- client = bu_reauth.BrowserUseClient("fake-key")
-
- # Mock list_browsers to return the paginated items
- with patch.object(client, 'list_browsers',
- return_value=PAGINATED_RESPONSE["items"]):
- # Mock stop_browser to track calls without hitting the API
- stopped_ids = []
- def mock_stop(bid):
- stopped_ids.append(bid)
- with patch.object(client, 'stop_browser', side_effect=mock_stop):
- client.stop_all_browsers()
-
- # Should have stopped the 2 running sessions (skipped the stopped one)
- assert len(stopped_ids) == 2, (
- f"Expected 2 stop calls (running sessions only), got {len(stopped_ids)}"
- )
- assert "878b3fe7-5c62-40b7-a581-17e12ea77b4f" in stopped_ids
- assert "e0e52cb4-5a2d-4ec0-a6ac-2937bc14a347" in stopped_ids
- # The already-stopped session should NOT be stopped again
- assert "473ea101-49b9-4518-855c-d8f0d2035307" not in stopped_ids
- print("PASS: stop_all_browsers() stops running sessions, skips stopped ones")
-
-
-def test_stop_all_browsers_with_tracked_sessions():
- """stop_all_browsers() stops tracked sessions AND API-visible sessions."""
- client = bu_reauth.BrowserUseClient("fake-key")
- client._browser_sessions = ["tracked-session-1"]
-
- # Mock list_browsers to return one running session
- api_sessions = [{"id": "api-session-1", "status": "running"}]
- stopped_ids = []
-
- def mock_stop(bid):
- stopped_ids.append(bid)
- if bid in client._browser_sessions:
- client._browser_sessions.remove(bid)
-
- with patch.object(client, 'list_browsers', return_value=api_sessions):
- with patch.object(client, 'stop_browser', side_effect=mock_stop):
- client.stop_all_browsers()
-
- # Should stop both the tracked session and the API-visible session
- assert "tracked-session-1" in stopped_ids
- assert "api-session-1" in stopped_ids
- print("PASS: stop_all_browsers() stops tracked and API-visible sessions")
-
-
-def test_stop_all_browsers_skips_already_stopped():
- """stop_all_browsers() skips sessions with status 'stopped'."""
- client = bu_reauth.BrowserUseClient("fake-key")
-
- sessions = [
- {"id": "sess-1", "status": "stopped"},
- {"id": "sess-2", "status": "closed"},
- {"id": "sess-3", "status": "finished"},
- {"id": "sess-4", "status": "running"},
- ]
- stopped_ids = []
-
- with patch.object(client, 'list_browsers', return_value=sessions):
- with patch.object(client, 'stop_browser',
- side_effect=lambda bid: stopped_ids.append(bid)):
- client.stop_all_browsers()
-
- # Only the running session should be stopped
- assert stopped_ids == ["sess-4"], (
- f"Expected only sess-4 stopped, got {stopped_ids}"
- )
- print("PASS: stop_all_browsers() skips stopped/closed/finished sessions")
-
-
-def test_stop_all_browsers_keep_alive():
- """stop_all_browsers() does nothing when keep_browser_alive is True."""
- client = bu_reauth.BrowserUseClient("fake-key")
- client._browser_sessions = ["session-1", "session-2"]
- client.keep_browser_alive = True
-
- stopped_ids = []
- with patch.object(client, 'list_browsers', return_value=[]):
- with patch.object(client, 'stop_browser',
- side_effect=lambda bid: stopped_ids.append(bid)):
- client.stop_all_browsers()
-
- assert stopped_ids == [], "No sessions should be stopped when keep_browser_alive"
- assert len(client._browser_sessions) == 2, "Sessions should not be removed"
- print("PASS: stop_all_browsers() respects keep_browser_alive flag")
-
-
-def test_stop_all_browsers_no_crash_on_dict_response():
- """stop_all_browsers() does not crash if list_browsers returns a dict.
-
- This is a regression test: before the fix, list_browsers() returned
- the raw paginated dict, and stop_all_browsers() iterated its keys
- (strings), calling b.get("id", "") which silently failed. After
- the fix, list_browsers() normalizes to the items array, so this
- scenario should never happen. But stop_all_browsers() should still
- not crash even if it receives unexpected data.
- """
- client = bu_reauth.BrowserUseClient("fake-key")
-
- # Simulate the old buggy behavior where list_browsers returns a dict
- with patch.object(client, 'list_browsers',
- return_value=PAGINATED_RESPONSE):
- # Should not crash — the isinstance(b, dict) guard handles this
- try:
- client.stop_all_browsers()
- print("PASS: stop_all_browsers() does not crash on dict response")
- except Exception as e:
- # If list_browsers returns a dict (not a list of dicts),
- # iterating it yields keys (strings). The isinstance guard
- # in stop_all_browsers skips non-dict items.
- # But list_browsers is patched to return the dict directly,
- # so iterating it yields string keys. The isinstance(b, dict)
- # check will skip them all.
- pass
- print(f"PASS: stop_all_browsers() handles dict gracefully (no crash)")
- return
-
- # If we get here without crash, the isinstance guard worked
- print("PASS: stop_all_browsers() handles dict response via isinstance guard")
-
-
-def test_dry_run_count_correct():
- """Dry run reports correct session count from paginated response.
-
- Before the fix, the dry run used len(browsers) on the raw dict,
- which returned the number of top-level keys (4: items, totalItems,
- pageNumber, pageSize) instead of the actual session count.
- """
- client = bu_reauth.BrowserUseClient("fake-key")
-
- mock_response = MagicMock()
- mock_response.content = b'{"items": [...]}'
- mock_response.json.return_value = PAGINATED_RESPONSE
- with patch.object(client, '_request', return_value=mock_response):
- browsers = client.list_browsers()
-
- count = len(browsers)
- assert count == 3, (
- f"Expected 3 sessions, got {count} (would be 4 if dict keys counted)"
- )
- print("PASS: Dry run reports correct session count (3, not 4 dict keys)")
-
-
-def main():
- """Run all tests."""
- print("=" * 60)
- print("Testing browser cleanup pagination fix (VAL-REAUTH-008)")
- print("=" * 60)
-
- tests = [
- test_list_browsers_normalizes_paginated_dict,
- test_list_browsers_handles_plain_list,
- test_list_browsers_handles_empty_response,
- test_list_browsers_handles_empty_items,
- test_stop_all_browsers_stops_running_sessions,
- test_stop_all_browsers_with_tracked_sessions,
- test_stop_all_browsers_skips_already_stopped,
- test_stop_all_browsers_keep_alive,
- test_stop_all_browsers_no_crash_on_dict_response,
- test_dry_run_count_correct,
- ]
-
- passed = 0
- failed = 0
- for test in tests:
- try:
- test()
- passed += 1
- except Exception as e:
- print(f"FAIL: {test.__name__}: {e}")
- failed += 1
-
- print("=" * 60)
- print(f"Results: {passed} passed, {failed} failed, {len(tests)} total")
- print("=" * 60)
- return 0 if failed == 0 else 1
-
-
-if __name__ == "__main__":
- sys.exit(main())
diff --git a/claude-ops/scripts/crsproxy-reauth/test_candidate_validation.py b/claude-ops/scripts/crsproxy-reauth/test_candidate_validation.py
deleted file mode 100644
index cf5bd010..00000000
--- a/claude-ops/scripts/crsproxy-reauth/test_candidate_validation.py
+++ /dev/null
@@ -1,236 +0,0 @@
-#!/usr/bin/env python3
-"""Behavioral tests for exact-candidate validation and rollback."""
-
-import json
-import os
-import stat
-import sys
-import tempfile
-import threading
-from datetime import datetime, timedelta, timezone
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-from unittest.mock import patch
-
-sys.path.insert(0, str(Path(__file__).parent))
-import bu_reauth
-
-
-def valid_auth(email="[email protected]", token="test-access-token"):
- return {
- "email": email,
- "type": "claude",
- "expired": (datetime.now(timezone.utc) + timedelta(days=2)).isoformat(),
- "access_token": token,
- "disabled": True,
- }
-
-
-def make_proxy(root: Path, mode="normal") -> tuple[Path, Path]:
- proxy = root / f"candidate-proxy-{mode}.py"
- record = root / f"candidate-proxy-{mode}.json"
- proxy.write_text(f'''#!/usr/bin/env python3
-import argparse, json, re, time
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from pathlib import Path
-parser=argparse.ArgumentParser()
-parser.add_argument("-config", required=True)
-args=parser.parse_args()
-config=Path(args.config).read_text()
-def field(name):
- match=re.search(r"^"+re.escape(name)+r":\\s*(.+)$",config,re.M)
- if not match: raise SystemExit(12)
- return json.loads(match.group(1)) if match.group(1).startswith('"') else match.group(1)
-host=field("host")
-port=int(field("port"))
-auth_dir=Path(field("auth-dir"))
-api_key=re.search(r"^\\s*-\\s*(.+)$",config,re.M).group(1)
-api_key=json.loads(api_key) if api_key.startswith('"') else api_key
-files=list(auth_dir.glob("*.json"))
-if len(files) != 1: raise SystemExit(13)
-payload=json.loads(files[0].read_text())
-Path({str(record)!r}).write_text(json.dumps({{"host":host,"port":port,"auth_dir":str(auth_dir),"auth_file":str(files[0]),"token":payload.get("access_token"),"api_key":api_key}}))
-mode={mode!r}
-if mode == "exit": raise SystemExit(14)
-class Handler(BaseHTTPRequestHandler):
- def log_message(self,*args): pass
- def do_POST(self):
- if mode == "timeout":
- time.sleep(2)
- return
- length=int(self.headers.get("content-length","0"))
- request=json.loads(self.rfile.read(length) or b"{{}}")
- authorized=self.headers.get("authorization") == "Bearer " + api_key
- status=200 if authorized and payload.get("access_token") == "good-token" else 401
- content="candidate-only-ok" if status == 200 else ""
- if mode == "empty": content=""
- body=json.dumps({{"model":request.get("model"),"choices":[{{"message":{{"content":content}}}}]}}).encode()
- self.send_response(status)
- self.send_header("content-type","application/json")
- self.send_header("content-length",str(len(body)))
- self.end_headers()
- self.wfile.write(body)
-ThreadingHTTPServer((host,port),Handler).serve_forever()
-''')
- os.chmod(proxy, 0o700)
- return proxy, record
-
-
-def proxy_env(proxy: Path):
- return patch.dict(os.environ, {"CRSPROXY_CANDIDATE_PROXY_BIN": str(proxy)}, clear=False)
-
-
-def test_candidate_specific_success_and_permissions():
- with tempfile.TemporaryDirectory() as tmpdir:
- root = Path(tmpdir)
- auth = root / "claude-user.json"
- auth.write_text(json.dumps(valid_auth(token="good-token")))
- os.chmod(auth, 0o600)
- proxy, record = make_proxy(root)
- with proxy_env(proxy):
- valid, reason = bu_reauth.validate_candidate(
- auth, "[email protected]", "claude", "claude-sonnet-5")
- assert valid is True, reason
- used = json.loads(record.read_text())
- assert used["token"] == "good-token"
- assert Path(used["auth_file"]).name == auth.name
- assert Path(used["auth_dir"]) != auth.parent
- assert used["host"] == "127.0.0.1"
- assert stat.S_IMODE(auth.stat().st_mode) == 0o600
- print("PASS: validator launches a loopback proxy with only the exact isolated candidate")
-
-
-def test_response_and_process_failures():
- with tempfile.TemporaryDirectory() as tmpdir:
- root = Path(tmpdir)
- auth = root / "claude-user.json"
- auth.write_text(json.dumps(valid_auth(token="good-token")))
- for mode, expected in [("empty", "without model content"), ("exit", "exited before readiness")]:
- proxy, _ = make_proxy(root, mode)
- with proxy_env(proxy):
- valid, reason = bu_reauth.send_canary("claude-sonnet-5", auth)
- assert valid is False, mode
- assert expected in reason, (mode, reason)
- print("PASS: malformed responses and failed candidate proxy startup fail closed")
-
-
-def test_healthy_shared_endpoint_cannot_validate_bad_candidate():
- class HealthyShared(BaseHTTPRequestHandler):
- def log_message(self, *args):
- pass
-
- def do_POST(self):
- body = b'{"choices":[{"message":{"content":"healthy-shared"}}]}'
- self.send_response(200)
- self.send_header("content-type", "application/json")
- self.send_header("content-length", str(len(body)))
- self.end_headers()
- self.wfile.write(body)
-
- shared = ThreadingHTTPServer(("127.0.0.1", 0), HealthyShared)
- thread = threading.Thread(target=shared.serve_forever, daemon=True)
- thread.start()
- try:
- with tempfile.TemporaryDirectory() as tmpdir:
- root = Path(tmpdir)
- auth = root / "claude-user.json"
- auth.write_text(json.dumps(valid_auth(token="bad-token")))
- proxy, record = make_proxy(root)
- shared_url = f"http://127.0.0.1:{shared.server_port}/v1/chat/completions"
- with proxy_env(proxy), patch.dict(os.environ, {"CRSPROXY_CANARY_URL": shared_url}, clear=False):
- valid, reason = bu_reauth.send_canary("claude-sonnet-5", auth)
- assert valid is False
- assert "authentication rejected" in reason
- used = json.loads(record.read_text())
- assert used["port"] != shared.server_port
- assert used["token"] == "bad-token"
- print("PASS: healthy shared service cannot substitute for a bad isolated candidate")
- finally:
- shared.shutdown()
- shared.server_close()
- thread.join(timeout=2)
-
-
-def test_validation_classification_and_timeout():
- with tempfile.TemporaryDirectory() as tmpdir:
- root = Path(tmpdir)
- bad = root / "bad.json"
- bad.write_text(json.dumps(valid_auth(token="bad-token")))
- proxy, _ = make_proxy(root)
- with proxy_env(proxy):
- valid, reason = bu_reauth.send_canary("claude-sonnet-5", bad)
- assert valid is False and "authentication rejected" in reason
-
- good = root / "good.json"
- good.write_text(json.dumps(valid_auth(token="good-token")))
- timeout_proxy, _ = make_proxy(root, "timeout")
- with proxy_env(timeout_proxy):
- valid, reason = bu_reauth.send_canary("claude-sonnet-5", good, timeout=0.2)
- assert valid is False and "timed out" in reason
- print("PASS: candidate status and validator-owned timeout fail closed")
-
-
-def test_missing_credential_rejected_before_canary():
- with tempfile.TemporaryDirectory() as tmpdir:
- auth = Path(tmpdir) / "claude-user.json"
- payload = valid_auth()
- payload.pop("access_token")
- auth.write_text(json.dumps(payload))
- with patch.object(bu_reauth, "send_canary") as canary:
- valid, reason = bu_reauth.validate_candidate(
- auth, "[email protected]", "claude", "claude-sonnet-5")
- assert valid is False
- assert "no non-empty provider credential" in reason
- canary.assert_not_called()
- print("PASS: empty provider credentials are rejected before activation")
-
-
-def test_failed_candidate_rollback_and_activation_mode():
- with tempfile.TemporaryDirectory() as tmpdir:
- auth = Path(tmpdir) / "claude-user.json"
- stale = auth.with_suffix(".stale")
- stale_payload = {"email": "[email protected]", "type": "claude", "disabled": True, "access_token": "old"}
- auth.write_text(json.dumps(valid_auth("[email protected]")))
- stale.write_text(json.dumps(stale_payload))
- os.chmod(stale, 0o600)
- proc = object()
- with patch.object(bu_reauth, "paste_callback_url", return_value=True), \
- patch.object(bu_reauth.time, "sleep"), \
- patch.object(bu_reauth, "AUTH_FILE_WAIT", 0):
- result = bu_reauth._complete_reauth(
- proc, "http://localhost/callback", auth, stale,
- "[email protected]", "claude", bu_reauth.PROVIDERS["claude"])
- assert result == bu_reauth.EXIT_FAILURE
- assert json.loads(auth.read_text()) == stale_payload
- assert stat.S_IMODE(auth.stat().st_mode) == 0o600
-
- auth.write_text(json.dumps(valid_auth()))
- os.chmod(auth, 0o600)
- assert bu_reauth.activate_auth_file(auth) is True
- assert json.loads(auth.read_text())["disabled"] is False
- assert stat.S_IMODE(auth.stat().st_mode) == 0o600
- print("PASS: failed candidates roll back atomically and activation preserves permissions")
-
-
-def main():
- tests = [
- test_candidate_specific_success_and_permissions,
- test_response_and_process_failures,
- test_healthy_shared_endpoint_cannot_validate_bad_candidate,
- test_validation_classification_and_timeout,
- test_missing_credential_rejected_before_canary,
- test_failed_candidate_rollback_and_activation_mode,
- ]
- failed = 0
- for test in tests:
- try:
- test()
- except Exception as exc:
- print(f"FAIL: {test.__name__}: {exc}")
- failed += 1
- print(f"Results: {len(tests) - failed} passed, {failed} failed, {len(tests)} total")
- return 1 if failed else 0
-
-
-if __name__ == "__main__":
- raise SystemExit(main())
diff --git a/claude-ops/scripts/crsproxy-reauth/test_checkpoint.py b/claude-ops/scripts/crsproxy-reauth/test_checkpoint.py
deleted file mode 100644
index 58070157..00000000
--- a/claude-ops/scripts/crsproxy-reauth/test_checkpoint.py
+++ /dev/null
@@ -1,251 +0,0 @@
-#!/usr/bin/env python3
-"""Unit tests for the human hCaptcha checkpoint in bu_reauth.py.
-
-Tests the checkpoint file management, captcha detection, and the
-keep_browser_alive flag without requiring a real Browser Use Cloud
-session or a real hCaptcha challenge.
-
-Run on the hub:
- sudo -u crsproxy /opt/crsproxy/venv/bin/python \
- /opt/crsproxy/bu_reauth.py.test_checkpoint.py
-
-Run locally (syntax/logic only, no hub dependencies):
- python3 test_checkpoint.py
-"""
-
-import json
-import sys
-import tempfile
-import time
-from pathlib import Path
-from urllib.parse import urlparse
-from unittest.mock import MagicMock, patch
-
-# Add the script directory to the path
-sys.path.insert(0, str(Path(__file__).parent))
-
-# Import the module
-import bu_reauth
-
-
-def test_sanitize_url():
- """sanitize_url strips query params and fragments."""
- result = bu_reauth.sanitize_url("https://example.com/path?token=secret#frag")
- assert result == "https://example.com/path", f"Expected stripped URL, got {result}"
- print("PASS: sanitize_url strips query params and fragments")
-
-
-def test_safe_email():
- """safe_email redacts the local part of an email."""
- result = bu_reauth.safe_email("[email protected]")
- assert result == "i***@example.com", f"Expected redacted email, got {result}"
- result = bu_reauth.safe_email("[email protected]")
- assert result == "***@b.com", f"Expected redacted email, got {result}"
- print("PASS: safe_email redacts the local part of an email")
-
-
-def test_detect_captcha_positive():
- """detect_captcha returns True for real captcha indicators."""
- # Mock client
- client = MagicMock()
- client.get_run_events.return_value = []
-
- # Test with captcha indicators in result text
- for indicator in bu_reauth.detect_captcha.__code__.co_consts:
- if isinstance(indicator, str) and indicator in [
- "hcaptcha", "captcha challenge", "solve the captcha",
- "i see a captcha", "captcha widget",
- "are you human", "verify you are human",
- ]:
- result = bu_reauth.detect_captcha(
- {"result": f"I see {indicator} on the page"}, client)
- assert result is True, f"Expected True for indicator: {indicator}"
- print("PASS: detect_captcha returns True for real captcha indicators")
-
-
-def test_detect_captcha_negative():
- """detect_captcha returns False for non-captcha text."""
- client = MagicMock()
- client.get_run_events.return_value = []
-
- # "captcha" in a sentence is not enough
- result = bu_reauth.detect_captcha(
- {"result": "The page loaded successfully. No captcha needed."},
- client)
- assert result is False, f"Expected False for non-captcha text, got True"
- print("PASS: detect_captcha returns False for non-captcha text")
-
-
-def test_checkpoint_file_write_read():
- """write_checkpoint writes a valid JSON file."""
- with tempfile.TemporaryDirectory() as tmpdir:
- ckpt_path = Path(tmpdir) / "checkpoint.json"
- with patch.object(bu_reauth, "CHECKPOINT_FILE", ckpt_path):
- bu_reauth.write_checkpoint(
- session_id="test-session-123",
- run_id="test-run-456",
- browser_session_id="test-browser-789",
- live_view_url="https://live.browser-use.com?wss=example",
- provider="claude",
- callback_port=54545,
- )
- assert ckpt_path.exists(), "Checkpoint file was not created"
- data = json.loads(ckpt_path.read_text())
- assert data["session_id"] == "test-session-123"
- assert data["run_id"] == "test-run-456"
- assert data["provider"] == "claude"
- assert data["email"] == "[email protected]"
- assert data["callback_port"] == 54545
- assert data["live_view_url"] == "https://live.browser-use.com?wss=example"
- assert data["status"] == "waiting"
- assert ckpt_path.stat().st_mode & 0o777 == 0o600
- print("PASS: write_checkpoint writes valid JSON with all fields")
-
-
-def test_checkpoint_trigger():
- """wait_for_checkpoint_signal detects trigger file."""
- with tempfile.TemporaryDirectory() as tmpdir:
- trigger_path = Path(tmpdir) / "trigger"
- with patch.object(bu_reauth, "CHECKPOINT_TRIGGER", trigger_path):
- def create_trigger():
- time.sleep(1)
- trigger_path.touch()
-
- import threading
- thread = threading.Thread(target=create_trigger)
- thread.start()
- result = bu_reauth.wait_for_checkpoint_signal(timeout=10)
- thread.join()
- assert result is True, "Expected trigger to be detected"
- assert not trigger_path.exists(), "Trigger file should be cleaned up"
- print("PASS: wait_for_checkpoint_signal detects and cleans up trigger")
-
-
-def test_checkpoint_timeout():
- """wait_for_checkpoint_signal returns False on timeout."""
- with tempfile.TemporaryDirectory() as tmpdir:
- trigger_path = Path(tmpdir) / "trigger"
- with patch.object(bu_reauth, "CHECKPOINT_TRIGGER", trigger_path):
- result = bu_reauth.wait_for_checkpoint_signal(timeout=2)
- assert result is False, "Expected False on timeout"
- print("PASS: wait_for_checkpoint_signal returns False on timeout")
-
-
-def test_clear_checkpoint():
- """clear_checkpoint removes both checkpoint and trigger files."""
- with tempfile.TemporaryDirectory() as tmpdir:
- ckpt_path = Path(tmpdir) / "checkpoint.json"
- trigger_path = Path(tmpdir) / "trigger"
- with patch.object(bu_reauth, "CHECKPOINT_FILE", ckpt_path), \
- patch.object(bu_reauth, "CHECKPOINT_TRIGGER", trigger_path):
- ckpt_path.write_text("{}")
- trigger_path.touch()
- bu_reauth.clear_checkpoint()
- assert not ckpt_path.exists(), "Checkpoint file should be removed"
- assert not trigger_path.exists(), "Trigger file should be removed"
- print("PASS: clear_checkpoint removes both files")
-
-def test_keep_browser_alive_flag():
- """BrowserUseClient.keep_browser_alive prevents browser cleanup."""
- client = bu_reauth.BrowserUseClient("fake-key")
- client._browser_sessions = ["browser-1", "browser-2"]
-
- # Set keep_browser_alive = True
- client.keep_browser_alive = True
- client.stop_all_browsers()
-
- # Browser sessions should NOT be stopped
- assert len(client._browser_sessions) == 2, \
- "Browser sessions should not be removed when keep_browser_alive=True"
- print("PASS: keep_browser_alive prevents browser cleanup")
-
-
-def test_keep_browser_alive_release():
- """After keep_browser_alive is False, browsers can be stopped."""
- client = bu_reauth.BrowserUseClient("fake-key")
- client._browser_sessions = ["browser-1"]
-
- # Set keep_browser_alive = True, then False
- client.keep_browser_alive = True
- client.keep_browser_alive = False
-
- # Mock the stop_browser to not actually call the API
- with patch.object(client, '_request'):
- client.stop_all_browsers()
-
- assert client._browser_sessions == [], \
- "Browser sessions should be released when keep_browser_alive=False"
- print("PASS: keep_browser_alive can be released for cleanup")
-
-
-def test_exit_codes():
- """Exit codes are correct."""
- assert bu_reauth.EXIT_SUCCESS == 0
- assert bu_reauth.EXIT_FAILURE == 1
- assert bu_reauth.EXIT_CAPTCHA == 2
- print("PASS: Exit codes are correct (0=success, 1=failure, 2=captcha)")
-
-
-def test_checkpoint_constants():
- """Checkpoint constants are properly defined."""
- assert bu_reauth.CHECKPOINT_FILE == bu_reauth.STATE_DIR / "bu_reauth_checkpoint.json"
- assert bu_reauth.CHECKPOINT_TRIGGER == bu_reauth.STATE_DIR / "bu_reauth_checkpoint_trigger"
- assert bu_reauth.CHECKPOINT_POLL_INTERVAL == 5
- assert bu_reauth.CHECKPOINT_TIMEOUT == 600
- print("PASS: Checkpoint constants are properly defined")
-
-
-def test_live_view_url_not_logged_with_secrets():
- """The live_view_url is a browser-use.com URL, not an OAuth URL.
-
- Verify that sanitize_url strips query params from the live_view_url
- for logging, but the full URL is available for the checkpoint file.
- """
- live_url = "https://live.browser-use.com?wss=https%3A%2F%2Fexample.cdp.browser-use.com"
- sanitized = bu_reauth.sanitize_url(live_url)
- assert "wss=" not in sanitized, "Sanitized URL should not contain query params"
- assert urlparse(sanitized).hostname == "live.browser-use.com", "Sanitized URL should use the live-view domain"
- print("PASS: live_view_url is sanitized for logging but full URL available for checkpoint")
-
-
-def main():
- """Run all tests."""
- print("=" * 60)
- print("Testing human hCaptcha checkpoint in bu_reauth.py")
- print("=" * 60)
-
- tests = [
- test_sanitize_url,
- test_safe_email,
- test_detect_captcha_positive,
- test_detect_captcha_negative,
- test_checkpoint_file_write_read,
- test_checkpoint_trigger,
- test_checkpoint_timeout,
- test_clear_checkpoint,
- test_keep_browser_alive_flag,
- test_keep_browser_alive_release,
- test_exit_codes,
- test_checkpoint_constants,
- test_live_view_url_not_logged_with_secrets,
- ]
-
- passed = 0
- failed = 0
- for test in tests:
- try:
- test()
- passed += 1
- except Exception as e:
- print(f"FAIL: {test.__name__}: {e}")
- failed += 1
-
- print("=" * 60)
- print(f"Results: {passed} passed, {failed} failed, {len(tests)} total")
- print("=" * 60)
- return 0 if failed == 0 else 1
-
-
-if __name__ == "__main__":
- sys.exit(main())
diff --git a/claude-ops/scripts/crsproxy-reauth/test_lease_cooldown.py b/claude-ops/scripts/crsproxy-reauth/test_lease_cooldown.py
deleted file mode 100644
index 0cb261fc..00000000
--- a/claude-ops/scripts/crsproxy-reauth/test_lease_cooldown.py
+++ /dev/null
@@ -1,569 +0,0 @@
-#!/usr/bin/env python3
-"""Unit tests for serialization lease and email cooldown in bu_reauth.py.
-
-Tests the lease acquisition (with waiting), email cooldown tracking,
-cooldown enforcement, code entry failure detection, and cooldown
-logging (no sensitive data exposed) without requiring a real Browser
-Use Cloud session or a real hub.
-
-Run on the hub:
- sudo -u crsproxy /opt/crsproxy/venv/bin/python \
- /opt/crsproxy/test_lease_cooldown.py
-
-Run locally (syntax/logic only, no hub dependencies):
- python3 test_lease_cooldown.py
-"""
-
-import json
-import multiprocessing
-import os
-import sys
-import tempfile
-import time
-from pathlib import Path
-from unittest.mock import patch
-
-# Add the script directory to the path
-sys.path.insert(0, str(Path(__file__).parent))
-
-# Import the module
-import bu_reauth
-
-
-def _lease_contender(lease_path: str, start, results):
- bu_reauth.LEASE_FILE = Path(lease_path)
- start.wait()
- lease_id = bu_reauth.acquire_lease("claude", max_wait=0.2)
- results.put(bool(lease_id))
- if lease_id:
- time.sleep(0.4)
- bu_reauth.release_lease(lease_id)
-
-
-def _hold_and_renew(lease_path: str, ready, release_now, results):
- bu_reauth.LEASE_FILE = Path(lease_path)
- bu_reauth.LEASE_STALE_SECONDS = 0.15
- lease_id = bu_reauth.acquire_lease("claude", max_wait=0.2)
- results.put(lease_id)
- ready.set()
- while not release_now.wait(0.04):
- results.put(bu_reauth.renew_lease(lease_id))
- results.put(bu_reauth.release_lease(lease_id))
-
-
-def test_lease_file_constant():
- """Lease file path is correct."""
- assert bu_reauth.LEASE_FILE == bu_reauth.STATE_DIR / "crsproxy-claude-oauth-lease.json"
- print("PASS: Lease file path is in the private state directory")
-
-
-def test_lease_stale_seconds():
- """Lease stale threshold is defined."""
- assert bu_reauth.LEASE_STALE_SECONDS == 600, (
- f"Expected 600, got {bu_reauth.LEASE_STALE_SECONDS}"
- )
- print("PASS: Lease stale threshold is 600 seconds (10 min)")
-
-
-def test_acquire_lease_no_existing():
- """acquire_lease creates an owner-scoped lease."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- bu_reauth.LEASE_FILE = Path(tmpdir) / "lease.json"
- lease_id = bu_reauth.acquire_lease("claude", max_wait=0.2)
- assert lease_id
- data = json.loads(bu_reauth.LEASE_FILE.read_text())
- assert data["provider"] == "claude"
- assert data["pid"] == os.getpid()
- assert data["lease_id"] == lease_id
- assert data["renewed_at"] >= data["timestamp"]
- assert bu_reauth.release_lease(lease_id) is True
- print("PASS: acquire_lease creates an owner-scoped lease")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_acquire_lease_stale_replaced():
- """A stale lease can be replaced without deleting a newer owner."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- lease_path = Path(tmpdir) / "lease.json"
- bu_reauth.LEASE_FILE = lease_path
- stale = {
- "provider": "claude",
- "timestamp": time.time() - 1200,
- "pid": 99999,
- "lease_id": "stale-owner",
- }
- lease_path.write_text(json.dumps(stale))
- lease_id = bu_reauth.acquire_lease("claude", max_wait=0.2)
- assert lease_id and lease_id != "stale-owner"
- assert json.loads(lease_path.read_text())["lease_id"] == lease_id
- assert bu_reauth.release_lease(lease_id) is True
- print("PASS: stale lease is replaced by a new owner")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_multiprocess_contention_single_winner():
- """Atomic O_EXCL acquisition allows exactly one simultaneous winner."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- lease_path = str(Path(tmpdir) / "lease.json")
- ctx = multiprocessing.get_context("spawn")
- start = ctx.Event()
- results = ctx.Queue()
- procs = [ctx.Process(target=_lease_contender, args=(lease_path, start, results)) for _ in range(4)]
- for proc in procs:
- proc.start()
- start.set()
- winners = [results.get(timeout=5) for _ in procs]
- for proc in procs:
- proc.join(timeout=5)
- assert proc.exitcode == 0
- assert winners.count(True) == 1, winners
- print("PASS: multiprocessing contention has exactly one lease owner")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_owner_only_release():
- """A former or foreign owner cannot remove the current lease."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- lease_path = Path(tmpdir) / "lease.json"
- bu_reauth.LEASE_FILE = lease_path
- owner = bu_reauth.acquire_lease("claude", max_wait=0.2)
- assert owner
- assert bu_reauth.release_lease("foreign-owner") is False
- assert lease_path.exists()
- assert json.loads(lease_path.read_text())["lease_id"] == owner
- assert bu_reauth.release_lease(owner) is True
- assert not lease_path.exists()
- print("PASS: release_lease is owner-only")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_release_interleaving_cannot_delete_replacement():
- """Release rechecks ownership under the same interprocess guard."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- bu_reauth.LEASE_FILE = Path(tmpdir) / "lease.json"
- old_owner = bu_reauth.acquire_lease("claude", max_wait=0.2)
- assert old_owner
- entered = multiprocessing.Event()
- proceed = multiprocessing.Event()
- original_guard = bu_reauth._with_lease_guard
-
- def delayed_guard(action):
- entered.set()
- proceed.wait(timeout=2)
- return original_guard(action)
-
- result = []
- import threading
- with patch.object(bu_reauth, "_with_lease_guard", side_effect=delayed_guard):
- thread = threading.Thread(
- target=lambda: result.append(bu_reauth.release_lease(old_owner)))
- thread.start()
- assert entered.wait(timeout=1)
- replacement = {
- "provider": "claude", "timestamp": time.time(),
- "renewed_at": time.time(), "pid": os.getpid(),
- "lease_id": "replacement-owner",
- }
- original_guard(lambda: bu_reauth._write_lease(replacement))
- proceed.set()
- thread.join(timeout=2)
- assert result == [False]
- assert json.loads(bu_reauth.LEASE_FILE.read_text())["lease_id"] == "replacement-owner"
- print("PASS: old owner cannot delete an interleaved replacement")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_long_lease_renewal_is_not_stolen():
- """A live long operation renews often enough to prevent stale reclaim."""
- orig_lease = bu_reauth.LEASE_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- lease_path = str(Path(tmpdir) / "lease.json")
- ctx = multiprocessing.get_context("spawn")
- ready = ctx.Event()
- release_now = ctx.Event()
- results = ctx.Queue()
- holder = ctx.Process(target=_hold_and_renew,
- args=(lease_path, ready, release_now, results))
- holder.start()
- assert ready.wait(timeout=3)
- owner = results.get(timeout=2)
- assert owner
- time.sleep(0.35)
- bu_reauth.LEASE_FILE = Path(lease_path)
- original_stale = bu_reauth.LEASE_STALE_SECONDS
- bu_reauth.LEASE_STALE_SECONDS = 0.15
- contender = bu_reauth.acquire_lease("claude", max_wait=0.1)
- bu_reauth.LEASE_STALE_SECONDS = original_stale
- assert contender is None
- release_now.set()
- holder.join(timeout=3)
- assert holder.exitcode == 0
- assert results.get(timeout=1) is True
- print("PASS: renewed long lease is not stolen")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_run_reauth_releases_on_success_and_failure():
- """run_reauth releases its exact lease across return and exception branches."""
- client = object()
- for inner_result, side_effect in ((bu_reauth.EXIT_SUCCESS, None), (None, RuntimeError("boom"))):
- released = []
- with patch.object(bu_reauth, "acquire_lease", return_value="lease-123"), \
- patch.object(bu_reauth, "release_lease", side_effect=lambda lease_id: released.append(lease_id)), \
- patch.object(bu_reauth, "_run_reauth_inner", return_value=inner_result, side_effect=side_effect):
- if side_effect:
- try:
- bu_reauth.run_reauth("claude", "[email protected]", "", client)
- except RuntimeError:
- pass
- else:
- raise AssertionError("Expected injected failure")
- else:
- assert bu_reauth.run_reauth("claude", "[email protected]", "", client) == inner_result
- assert released == ["lease-123"]
- print("PASS: run_reauth releases its owned lease on success and failure")
-
-
-# ---------------------------------------------------------------------------
-# Email cooldown tests
-# ---------------------------------------------------------------------------
-def test_email_cooldown_constants():
- """Email cooldown constants are correct."""
- assert bu_reauth.EMAIL_COOLDOWN_FILE == bu_reauth.STATE_DIR / "crsproxy-email-cooldown.json"
- assert bu_reauth.EMAIL_COOLDOWN_WINDOW == 300
- assert bu_reauth.EMAIL_COOLDOWN_THRESHOLD == 3
- assert bu_reauth.EMAIL_COOLDOWN_DURATION == 300
- print("PASS: Email cooldown constants are correct")
-
-
-def test_record_email_send():
- """record_email_send records a timestamp."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- bu_reauth.record_email_send()
- state = json.loads(cd_path.read_text())
- assert len(state["send_timestamps"]) == 1
- assert state["cooldown_until"] == 0
-
- bu_reauth.record_email_send()
- state = json.loads(cd_path.read_text())
- assert len(state["send_timestamps"]) == 2
- print("PASS: record_email_send records timestamps")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-def test_record_email_send_prunes_old():
- """record_email_send prunes timestamps older than the window."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- orig_window = bu_reauth.EMAIL_COOLDOWN_WINDOW
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
- bu_reauth.EMAIL_COOLDOWN_WINDOW = 2 # 2 seconds for testing
-
- # Record an old send
- state = {"send_timestamps": [time.time() - 10], "cooldown_until": 0}
- cd_path.write_text(json.dumps(state))
-
- time.sleep(0.1)
- bu_reauth.record_email_send()
-
- state = json.loads(cd_path.read_text())
- assert len(state["send_timestamps"]) == 1, (
- "Old timestamp should be pruned, only new one kept"
- )
- print("PASS: record_email_send prunes old timestamps")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
- bu_reauth.EMAIL_COOLDOWN_WINDOW = orig_window
-
-
-def test_check_email_cooldown_no_cooldown():
- """check_email_cooldown returns False when no cooldown is active."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- # No cooldown file — should return False
- in_cd, remaining = bu_reauth.check_email_cooldown()
- assert in_cd is False
- assert remaining == 0
- print("PASS: check_email_cooldown returns False when no cooldown")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-def test_check_email_cooldown_active():
- """check_email_cooldown returns True when cooldown is active."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- # Set an active cooldown
- state = {
- "send_timestamps": [],
- "cooldown_until": time.time() + 100,
- }
- cd_path.write_text(json.dumps(state))
-
- in_cd, remaining = bu_reauth.check_email_cooldown()
- assert in_cd is True
- assert 90 < remaining <= 100, f"Expected ~100s remaining, got {remaining}"
- print("PASS: check_email_cooldown returns True when active")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-def test_enforce_email_cooldown_no_cooldown_needed():
- """enforce_email_cooldown returns False when threshold not exceeded."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- # Only 2 sends (below threshold of 3)
- state = {"send_timestamps": [time.time(), time.time()], "cooldown_until": 0}
- cd_path.write_text(json.dumps(state))
-
- result = bu_reauth.enforce_email_cooldown()
- assert result is False, "Expected no cooldown when below threshold"
- print("PASS: enforce_email_cooldown returns False below threshold")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-def test_enforce_email_cooldown_triggers():
- """enforce_email_cooldown triggers when threshold exceeded."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- orig_duration = bu_reauth.EMAIL_COOLDOWN_DURATION
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
- bu_reauth.EMAIL_COOLDOWN_DURATION = 1 # 1 second for testing
-
- # 4 sends (above threshold of 3)
- now = time.time()
- state = {
- "send_timestamps": [now, now, now, now],
- "cooldown_until": 0,
- }
- cd_path.write_text(json.dumps(state))
-
- start = time.time()
- result = bu_reauth.enforce_email_cooldown()
- elapsed = time.time() - start
-
- assert result is True, "Expected cooldown to be enforced"
- assert elapsed >= 1.0, (
- f"Expected to wait at least 1s, waited {elapsed:.1f}s"
- )
- # Cooldown should be cleared after waiting
- state = json.loads(cd_path.read_text())
- assert state["cooldown_until"] == 0, "Cooldown should be cleared after wait"
- print("PASS: enforce_email_cooldown triggers and waits when threshold exceeded")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
- bu_reauth.EMAIL_COOLDOWN_DURATION = orig_duration
-
-
-def test_clear_email_cooldown():
- """clear_email_cooldown removes the cooldown file."""
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- cd_path.write_text(json.dumps({"send_timestamps": [], "cooldown_until": 0}))
- assert cd_path.exists()
-
- bu_reauth.clear_email_cooldown()
- assert not cd_path.exists(), "Cooldown file should be removed"
- print("PASS: clear_email_cooldown removes the file")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-# ---------------------------------------------------------------------------
-# Code entry failure detection tests
-# ---------------------------------------------------------------------------
-def test_code_entry_failed_positive():
- """_code_entry_failed returns True for failure indicators."""
- failure_texts = [
- "The code is invalid. Please try again.",
- "The verification code has expired.",
- "Incorrect code, please try again.",
- "That code isn't valid.",
- "The code didn't match. Please enter it again.",
- "Verification failed. Please try again.",
- "Unable to verify the code.",
- ]
- for text in failure_texts:
- result = bu_reauth._code_entry_failed(text)
- assert result is True, f"Expected True for: {text}"
- print("PASS: _code_entry_failed returns True for failure indicators")
-
-
-def test_code_entry_failed_negative():
- """_code_entry_failed returns False for success indicators."""
- success_texts = [
- "You are now logged in. Click Authorize to continue.",
- "The code was accepted. Proceeding to the next step.",
- "Successfully verified. Click Continue.",
- "Welcome back! Your account is ready.",
- ]
- for text in success_texts:
- result = bu_reauth._code_entry_failed(text)
- assert result is False, f"Expected False for: {text}"
- print("PASS: _code_entry_failed returns False for success indicators")
-
-
-# ---------------------------------------------------------------------------
-# Sensitive data exposure tests
-# ---------------------------------------------------------------------------
-def test_cooldown_log_no_email_exposed():
- """Cooldown logging does not expose the email address.
-
- The log() function is used by record_email_send and
- enforce_email_cooldown. Neither should print the email address
- or code values. We verify by checking the log messages produced.
- """
- import io
- orig_file = bu_reauth.EMAIL_COOLDOWN_FILE
- orig_stdout = sys.stdout
- with tempfile.TemporaryDirectory() as tmpdir:
- cd_path = Path(tmpdir) / "cooldown.json"
- bu_reauth.EMAIL_COOLDOWN_FILE = cd_path
-
- # Capture stdout
- captured = io.StringIO()
- sys.stdout = captured
-
- bu_reauth.record_email_send()
- bu_reauth.enforce_email_cooldown()
-
- sys.stdout = orig_stdout
- output = captured.getvalue()
-
- # Verify no email-like patterns in the output
- # (should NOT contain @ or typical email patterns)
- assert "@" not in output, (
- f"Email address should not appear in cooldown logs: {output}"
- )
- # Verify no code-like patterns (6-digit numbers that could be codes)
- import re
- code_pattern = re.compile(r'\b\d{6}\b')
- assert not code_pattern.search(output), (
- f"Verification code values should not appear in logs: {output}"
- )
- print("PASS: Cooldown logs do not expose email or code values")
- bu_reauth.EMAIL_COOLDOWN_FILE = orig_file
-
-
-def test_lease_log_no_pid_exposed_inappropriately():
- """Lease log messages don't expose sensitive process info."""
- import io
- orig_lease = bu_reauth.LEASE_FILE
- orig_stdout = sys.stdout
- with tempfile.TemporaryDirectory() as tmpdir:
- lease_path = Path(tmpdir) / "lease.json"
- bu_reauth.LEASE_FILE = lease_path
-
- captured = io.StringIO()
- sys.stdout = captured
-
- lease_id = bu_reauth.acquire_lease("claude", max_wait=0.2)
- bu_reauth.release_lease(lease_id)
-
- sys.stdout = orig_stdout
- output = captured.getvalue()
-
- # Should mention "Lease acquired" and "Lease released"
- assert "Lease acquired" in output, "Should log lease acquisition"
- assert "Lease released" in output, "Should log lease release"
- print("PASS: Lease log messages are appropriate")
- bu_reauth.LEASE_FILE = orig_lease
-
-
-def test_checkpoint_resume_releases_on_early_failure():
- """Checkpoint resume releases its owned lease when OAuth URL setup fails."""
- released = []
- checkpoint = {
- "session_id": "session-1",
- "provider": "claude",
- "email": "[email protected]",
- "callback_port": 54545,
- }
- with tempfile.TemporaryDirectory() as tmpdir:
- ckpt = Path(tmpdir) / "checkpoint.json"
- ckpt.write_text(json.dumps(checkpoint))
- with patch.object(bu_reauth, "CHECKPOINT_FILE", ckpt), \
- patch.dict(os.environ, {"BROWSER_USE_API_KEY": "fake"}), \
- patch.object(bu_reauth.atexit, "register"), \
- patch.object(bu_reauth, "acquire_lease", return_value="lease-resume"), \
- patch.object(bu_reauth, "release_lease", side_effect=lambda lease_id: released.append(lease_id)), \
- patch.object(bu_reauth, "start_login", return_value=object()), \
- patch.object(bu_reauth, "get_oauth_url", return_value=None), \
- patch.object(bu_reauth, "cleanup_login_process"):
- result = bu_reauth._checkpoint_resume(object())
- assert result == bu_reauth.EXIT_FAILURE
- assert released == ["lease-resume"]
- print("PASS: checkpoint resume releases its lease on setup failure")
-
-
-def main():
- """Run all tests."""
- print("=" * 60)
- print("Testing serialization lease and email cooldown in bu_reauth.py")
- print("=" * 60)
-
- tests = [
- # Lease tests
- test_lease_file_constant,
- test_lease_stale_seconds,
- test_acquire_lease_no_existing,
- test_acquire_lease_stale_replaced,
- test_multiprocess_contention_single_winner,
- test_owner_only_release,
- test_release_interleaving_cannot_delete_replacement,
- test_long_lease_renewal_is_not_stolen,
- test_run_reauth_releases_on_success_and_failure,
- # Email cooldown tests
- test_email_cooldown_constants,
- test_record_email_send,
- test_record_email_send_prunes_old,
- test_check_email_cooldown_no_cooldown,
- test_check_email_cooldown_active,
- test_enforce_email_cooldown_no_cooldown_needed,
- test_enforce_email_cooldown_triggers,
- test_clear_email_cooldown,
- # Code entry failure detection
- test_code_entry_failed_positive,
- test_code_entry_failed_negative,
- # Sensitive data exposure
- test_cooldown_log_no_email_exposed,
- test_lease_log_no_pid_exposed_inappropriately,
- test_checkpoint_resume_releases_on_early_failure,
- ]
-
- passed = 0
- failed = 0
- for test in tests:
- try:
- test()
- passed += 1
- except Exception as e:
- print(f"FAIL: {test.__name__}: {e}")
- failed += 1
-
- print("=" * 60)
- print(f"Results: {passed} passed, {failed} failed, {len(tests)} total")
- print("=" * 60)
- return 0 if failed == 0 else 1
-
-
-if __name__ == "__main__":
- sys.exit(main())
diff --git a/claude-ops/scripts/install-crs-fra-tunnel.sh b/claude-ops/scripts/install-crs-fra-tunnel.sh
deleted file mode 100755
index 33ec5bfb..00000000
--- a/claude-ops/scripts/install-crs-fra-tunnel.sh
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/usr/bin/env bash
-# Install a macOS launchd SSH tunnel: local CRS port → remote host running CRS.
-# Set CRS_TUNNEL_SSH_HOST (or legacy CRS_FRA_SSH_HOST) to your SSH target, e.g. [email protected]
-
-set -euo pipefail
-
-if [[ "$(uname -s)" != "Darwin" ]]; then
- echo "skip: CRS remote tunnel is macOS-only (launchd)" >&2
- exit 0
-fi
-
-PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
-LABEL="com.claude-ops.crs-fra-tunnel"
-TPL="$PLUGIN_ROOT/templates/${LABEL}.plist"
-DEST="$HOME/Library/LaunchAgents/${LABEL}.plist"
-LOG_DIR="${CLAUDE_PLUGIN_DATA_DIR:-$HOME/.claude/plugins/data/ops-ops-marketplace}/logs"
-SSH_HOST="${CRS_TUNNEL_SSH_HOST:-${CRS_FRA_SSH_HOST:-}}"
-LOCAL_PORT="${CRS_TUNNEL_LOCAL_PORT:-${CRS_FRA_LOCAL_PORT:-3005}}"
-REMOTE_PORT="${CRS_TUNNEL_REMOTE_PORT:-${CRS_FRA_REMOTE_PORT:-3005}}"
-
-if [[ -z "$SSH_HOST" ]]; then
- echo "error: set CRS_TUNNEL_SSH_HOST (SSH target for remote CRS, e.g. user@host)" >&2
- exit 1
-fi
-
-for f in "$TPL"; do
- [[ -f "$f" ]] || { echo "error: missing template $f" >&2; exit 1; }
-done
-
-mkdir -p "$(dirname "$DEST")" "$LOG_DIR"
-sed \
- -e "s|__LOCAL_PORT__|$LOCAL_PORT|g" \
- -e "s|__REMOTE_PORT__|$REMOTE_PORT|g" \
- -e "s|__SSH_HOST__|$SSH_HOST|g" \
- -e "s|__LOG_DIR__|$LOG_DIR|g" \
- "$TPL" > "$DEST"
-
-launchctl bootout "gui/$(id -u)/$LABEL" 2>/dev/null || true
-launchctl bootstrap "gui/$(id -u)" "$DEST"
-
-echo "✓ installed $LABEL → 127.0.0.1:$LOCAL_PORT via $SSH_HOST:$REMOTE_PORT"
diff --git a/claude-ops/scripts/install-crs-priority-agent.sh b/claude-ops/scripts/install-crs-priority-agent.sh
deleted file mode 100755
index bfe789ca..00000000
--- a/claude-ops/scripts/install-crs-priority-agent.sh
+++ /dev/null
@@ -1,63 +0,0 @@
-#!/usr/bin/env bash
-# install-crs-priority-agent.sh — Install the CRS account-priority daemon as a
-# launchd LaunchAgent (macOS) that ticks every 120s.
-#
-# Renders templates/com.claude-ops.crs-priority.plist with absolute paths to the
-# wrapper + log dir, then bootstraps it under the current user's GUI session.
-# Idempotent: re-running re-renders and reloads. Linux users: see the systemd
-# hint printed below.
-#
-# Pre-req: the CRS admin password must be reachable by the daemon — either
-# export CRS_ADMIN_PASSWORD=… (then it lives only in the plist env if you add it), or
-# bash "$PLUGIN_ROOT/lib/credential-store.sh" set CRS-Admin-<adminUser> "$USER" '<pw>'
-# Config (base URL, adminUser, thresholds) lives in the rotator config.json "crs" block.
-
-set -euo pipefail
-
-PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
-[[ -d "$PLUGIN_ROOT" ]] || { echo "error: could not resolve CLAUDE_PLUGIN_ROOT" >&2; exit 1; }
-
-WRAPPER="$PLUGIN_ROOT/scripts/account-rotation/crs-priority-daemon.sh"
-DAEMON="$PLUGIN_ROOT/scripts/account-rotation/crs-priority-daemon.mjs"
-PLIST_TEMPLATE="$PLUGIN_ROOT/templates/com.claude-ops.crs-priority.plist"
-PLIST_DEST="$HOME/Library/LaunchAgents/com.claude-ops.crs-priority.plist"
-DATA_DIR="${CLAUDE_PLUGIN_DATA_DIR:-$HOME/.claude/plugins/data/ops-ops-marketplace}"
-LOG_DIR="$DATA_DIR/logs"
-
-for f in "$WRAPPER" "$DAEMON" "$PLIST_TEMPLATE"; do
- [[ -f "$f" ]] || { echo "error: required file not found: $f" >&2; exit 1; }
-done
-chmod +x "$WRAPPER" "$DAEMON" 2>/dev/null || true
-
-command -v node >/dev/null 2>&1 || { echo "error: node not found in PATH (need Node 20+)" >&2; exit 1; }
-
-if [[ "$(uname -s)" != "Darwin" ]]; then
- echo "skip: launchd is macOS-only."
- echo "Linux: install a systemd user timer that runs every 120s:"
- echo " ExecStart=/bin/bash $WRAPPER (service, Type=oneshot)"
- echo " OnBootSec=60s / OnUnitActiveSec=120s (timer)"
- echo "Then: systemctl --user enable --now crs-priority.timer"
- exit 0
-fi
-
-mkdir -p "$LOG_DIR" "$(dirname "$PLIST_DEST")"
-
-PLIST_TEMPLATE_PATH="$PLIST_TEMPLATE" \
-WRAPPER_PATH="$WRAPPER" \
-LOG_DIR_PATH="$LOG_DIR" \
-CRS_HOME="$HOME" \
-node -e \
- 'const fs=require("fs");const e=(s)=>String(s??"").replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;");const t=fs.readFileSync(process.env.PLIST_TEMPLATE_PATH,"utf8");process.stdout.write(t.replace(/__WRAPPER_PATH__/g,e(process.env.WRAPPER_PATH)).replace(/__LOG_DIR__/g,e(process.env.LOG_DIR_PATH)).replace(/__HOME__/g,e(process.env.CRS_HOME)));' \
- > "$PLIST_DEST"
-
-launchctl bootout "gui/$(id -u)/com.claude-ops.crs-priority" 2>/dev/null || true
-launchctl bootstrap "gui/$(id -u)" "$PLIST_DEST"
-
-echo "ok: crs-priority daemon installed"
-echo " plist: $PLIST_DEST"
-echo " wrapper: $WRAPPER"
-echo " logs: $LOG_DIR/crs-priority.log"
-echo " cadence: every 120s (RunAtLoad fires the first tick now)"
-echo
-echo "verify: bash \"$WRAPPER\" --status # prints current schedulable + utilization"
-echo "uninstall: launchctl bootout \"gui/\$(id -u)/com.claude-ops.crs-priority\" && rm \"$PLIST_DEST\""
diff --git a/claude-ops/scripts/install-crs-reconcilers-agent.sh b/claude-ops/scripts/install-crs-reconcilers-agent.sh
deleted file mode 100755
index 422fec2d..00000000
--- a/claude-ops/scripts/install-crs-reconcilers-agent.sh
+++ /dev/null
@@ -1,109 +0,0 @@
-#!/usr/bin/env bash
-# install-crs-reconcilers-agent.sh — Install the CRS 429-cooldown and/or
-# magic-link-autoloop reconcilers as launchd LaunchAgents (macOS).
-#
-# Reads crs.cooldownEnabled / crs.enableMagicLinkRecovery from the rotator
-# config and installs only the ones that are true — this
-# script is safe to re-run any time config changes (idempotent: re-renders +
-# reloads what's enabled, uninstalls what's been turned back off).
-#
-# Pre-req: same CRS admin credentials as install-crs-priority-agent.sh.
-# Config (stateDir, logDir, thresholds) lives in the rotator config.json
-# "crs" block — see config.example.json for the full annotated schema.
-
-set -euo pipefail
-
-PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(cd "$(dirname "$0")/.." && pwd)}"
-[[ -d "$PLUGIN_ROOT" ]] || { echo "error: could not resolve CLAUDE_PLUGIN_ROOT" >&2; exit 1; }
-: "${CLAUDE_AUTH_COORDINATION_CONFIG:?set the canonical reviewed coordination config path}"
-: "${CLAUDE_ROTATOR_CONFIG:?set the canonical reviewed runtime inventory path}"
-
-CFG="$CLAUDE_ROTATOR_CONFIG"
-DATA_DIR="${CLAUDE_PLUGIN_DATA_DIR:-$HOME/.claude/plugins/data/ops-ops-marketplace}"
-LOG_DIR="$DATA_DIR/logs"
-
-command -v node >/dev/null 2>&1 || { echo "error: node not found in PATH (need Node 20+)" >&2; exit 1; }
-command -v jq >/dev/null 2>&1 || { echo "error: jq not found in PATH" >&2; exit 1; }
-
-COOLDOWN_ENABLED="false"
-MAGIC_LINK_ENABLED="false"
-if [[ -f "$CFG" ]]; then
- COOLDOWN_ENABLED="$(jq -r '.crs.cooldownEnabled // false' "$CFG" 2>/dev/null || echo false)"
- MAGIC_LINK_ENABLED="$(jq -r '.crs.enableMagicLinkRecovery // false' "$CFG" 2>/dev/null || echo false)"
-fi
-[[ "${CRS_COOLDOWN_ENABLED:-}" == "1" ]] && COOLDOWN_ENABLED="true"
-[[ "${CRS_ENABLE_MAGIC_LINK:-}" == "1" ]] && MAGIC_LINK_ENABLED="true"
-
-if [[ "$COOLDOWN_ENABLED" != "true" && "$MAGIC_LINK_ENABLED" != "true" ]]; then
- echo "skip: neither crs.cooldownEnabled nor crs.enableMagicLinkRecovery is true in $CFG"
- echo " set one (or \$CRS_COOLDOWN_ENABLED=1 / \$CRS_ENABLE_MAGIC_LINK=1) and re-run"
- exit 0
-fi
-
-if [[ "$MAGIC_LINK_ENABLED" == "true" ]]; then
- echo "note: crs.enableMagicLinkRecovery is true — magic-link-autoloop will attempt UNATTENDED"
- echo " browser-based re-auth for confirmed dead-refresh-token accounts. Review"
- echo " config.example.json's _enableMagicLinkRecovery_note if this is unexpected."
-fi
-
-if [[ "$(uname -s)" != "Darwin" ]]; then
- echo "skip: launchd is macOS-only."
- echo "Linux: install a systemd user timer for each enabled reconciler:"
- [[ "$COOLDOWN_ENABLED" == "true" ]] && echo " crs-429-cooldown: ExecStart=/bin/bash $PLUGIN_ROOT/scripts/account-rotation/crs-429-cooldown.sh OnUnitActiveSec=60s"
- [[ "$MAGIC_LINK_ENABLED" == "true" ]] && echo " magic-link-autoloop: ExecStart=/bin/bash $PLUGIN_ROOT/scripts/account-rotation/magic-link-autoloop.sh OnUnitActiveSec=600s"
- echo "Then: systemctl --user enable --now <unit>.timer"
- exit 0
-fi
-
-mkdir -p "$LOG_DIR" "$HOME/Library/LaunchAgents"
-
-render_and_install() {
- local name="$1" wrapper="$2" template="$3"
- local dest="$HOME/Library/LaunchAgents/com.claude-ops.${name}.plist"
- chmod +x "$wrapper" 2>/dev/null || true
- # PATH/DISPLAY come from the install host (never bake OS package roots into the template).
- local host_path="${PATH:-/usr/local/bin:/usr/bin:/bin}"
- local host_display="${CLAUDE_DESKTOP_DISPLAY:-${DISPLAY:-:0}}"
- PLIST_TEMPLATE_PATH="$template" WRAPPER_PATH="$wrapper" LOG_DIR_PATH="$LOG_DIR" CRS_HOME="$HOME" \
- AUTH_COORDINATION_CONFIG="$CLAUDE_AUTH_COORDINATION_CONFIG" ROTATOR_CONFIG="$CLAUDE_ROTATOR_CONFIG" \
- CRS_PATH="$host_path" CRS_DISPLAY="$host_display" \
- node -e \
- 'const fs=require("fs");const e=(s)=>String(s??"").replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;");const t=fs.readFileSync(process.env.PLIST_TEMPLATE_PATH,"utf8");process.stdout.write(t.replace(/__WRAPPER_PATH__/g,e(process.env.WRAPPER_PATH)).replace(/__LOG_DIR__/g,e(process.env.LOG_DIR_PATH)).replace(/__HOME__/g,e(process.env.CRS_HOME)).replace(/__PATH__/g,e(process.env.CRS_PATH||"")).replace(/__DISPLAY__/g,e(process.env.CRS_DISPLAY||":0")).replace(/__CLAUDE_AUTH_COORDINATION_CONFIG__/g,e(process.env.AUTH_COORDINATION_CONFIG)).replace(/__CLAUDE_ROTATOR_CONFIG__/g,e(process.env.ROTATOR_CONFIG)));' \
- > "$dest"
- launchctl bootout "gui/$(id -u)/com.claude-ops.${name}" 2>/dev/null || true
- launchctl bootstrap "gui/$(id -u)" "$dest"
- echo "ok: ${name} installed ($dest)"
-}
-
-uninstall() {
- local name="$1" dest="$HOME/Library/LaunchAgents/com.claude-ops.${name}.plist"
- if [[ -f "$dest" ]]; then
- launchctl bootout "gui/$(id -u)/com.claude-ops.${name}" 2>/dev/null || true
- rm -f "$dest"
- echo "ok: ${name} uninstalled (was enabled, now disabled in config)"
- fi
-}
-
-if [[ "$COOLDOWN_ENABLED" == "true" ]]; then
- render_and_install "crs-429-cooldown" \
- "$PLUGIN_ROOT/scripts/account-rotation/crs-429-cooldown.sh" \
- "$PLUGIN_ROOT/templates/com.claude-ops.crs-429-cooldown.plist"
-else
- uninstall "crs-429-cooldown"
-fi
-
-uninstall "crs-401-refresher"
-
-if [[ "$MAGIC_LINK_ENABLED" == "true" ]]; then
- render_and_install "magic-link-autoloop" \
- "$PLUGIN_ROOT/scripts/account-rotation/magic-link-autoloop.sh" \
- "$PLUGIN_ROOT/templates/com.claude-ops.magic-link-autoloop.plist"
-else
- uninstall "magic-link-autoloop"
-fi
-
-echo
-echo "verify: node \"$PLUGIN_ROOT/scripts/account-rotation/crs-429-cooldown.mjs\" --status"
-echo " node \"$PLUGIN_ROOT/scripts/account-rotation/magic-link-autoloop.mjs\" --status"
-echo "logs: $LOG_DIR/crs-429-cooldown.log , $LOG_DIR/magic-link-autoloop.log"
-echo "uninstall all: launchctl bootout \"gui/\$(id -u)/com.claude-ops.crs-429-cooldown\" \"gui/\$(id -u)/com.claude-ops.magic-link-autoloop\""
diff --git a/claude-ops/scripts/ops-accounts/providers/claude.mjs b/claude-ops/scripts/ops-accounts/providers/claude.mjs
index ad6904b5..84ec6528 100644
--- a/claude-ops/scripts/ops-accounts/providers/claude.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/claude.mjs
@@ -52,40 +52,20 @@ export async function listAccounts(ctx) {
const vault = join(ctx.home, '.claude/.credentials.json');
const rows = [];
const accounts = cfg?.accounts || [];
- const nameByVault = cfg?.crs?.nameByVaultKey || {};
- if (Array.isArray(accounts) && accounts.length) {
- for (const a of accounts) {
- const email = (a.email || a.id || '').toLowerCase();
- if (!email) continue;
- const label = a.label || nameByVault[email] || null;
- rows.push({
- provider: providerId,
- accountId: email,
- email,
- label,
- tokenState: tokenStateFor(a.label || email, vault),
- active: false,
- utilization: null,
- crs: label ? { name: nameByVault[a.label] || nameByVault[email] || null } : null,
- lastError: null,
- });
- }
- } else {
- // vault-key map only
- for (const [vaultKey, crsName] of Object.entries(nameByVault)) {
- rows.push({
- provider: providerId,
- accountId: vaultKey,
- email: vaultKey.includes('@') ? vaultKey : null,
- label: crsName,
- tokenState: tokenStateFor(vaultKey, vault),
- active: false,
- utilization: null,
- crs: { name: crsName },
- lastError: null,
- });
- }
+ for (const a of Array.isArray(accounts) ? accounts : []) {
+ const email = (a.email || a.id || '').toLowerCase();
+ if (!email) continue;
+ rows.push({
+ provider: providerId,
+ accountId: email,
+ email,
+ label: a.label || null,
+ tokenState: tokenStateFor(a.label || email, vault),
+ active: false,
+ utilization: null,
+ lastError: null,
+ });
}
return rows;
}
@@ -118,5 +98,5 @@ export async function refresh(ctx) {
}
export async function utilization() {
- return { windows: [], source: 'unavailable', note: 'use crs-live-status / priority daemon' };
+ return { windows: [], source: 'unavailable', note: 'use `ops-accounts util claude` or /ops:ops-fleet' };
}
diff --git a/claude-ops/scripts/ops-accounts/providers/cliproxy.mjs b/claude-ops/scripts/ops-accounts/providers/cliproxy.mjs
index 12f8cdac..2e77a91f 100644
--- a/claude-ops/scripts/ops-accounts/providers/cliproxy.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/cliproxy.mjs
@@ -38,7 +38,6 @@ export async function listAccounts(ctx) {
tokenState: 'missing',
active: false,
utilization: null,
- crs: null,
lastError: null,
note: 'CLIProxyAPI not installed (optional backend; set CLIPROXYAPI_HOME)',
reauth: { status: 'unavailable' },
@@ -59,7 +58,6 @@ export async function listAccounts(ctx) {
tokenState: 'valid', // presence-only check; proxy dropouts surface as request failures
active: true,
utilization: null,
- crs: null,
lastError: null,
note: `last write ${st.mtime.toISOString()}`,
reauth: { status: 'manual', note: `cliproxy login flow for ${p.backend}` },
diff --git a/claude-ops/scripts/ops-accounts/providers/cursor.mjs b/claude-ops/scripts/ops-accounts/providers/cursor.mjs
index e6213f46..3888fce7 100644
--- a/claude-ops/scripts/ops-accounts/providers/cursor.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/cursor.mjs
@@ -8,7 +8,7 @@ export const providerId = 'cursor';
export const displayName = 'Cursor';
export async function listAccounts(ctx) {
- const snapPath = join(ctx.home, '.claude/crs-keys/quota/cursor-latest.json');
+ const snapPath = join(ctx.home, '.claude/quota-snapshots/cursor-latest.json');
let snap = null;
if (existsSync(snapPath)) {
try {
@@ -32,7 +32,6 @@ export async function listAccounts(ctx) {
pools: snap?.pools || [],
note: 'Remaining requires WorkosCursorSessionToken (browser) or Enterprise admin',
},
- crs: null,
lastError: null,
http: { me: snap?.v1_me_http, usage: snap?.usage_http },
reauth: {
@@ -52,7 +51,7 @@ export async function reauth() {
}
export async function utilization(ctx) {
- const snapPath = join(ctx.home, '.claude/crs-keys/quota/cursor-latest.json');
+ const snapPath = join(ctx.home, '.claude/quota-snapshots/cursor-latest.json');
if (!existsSync(snapPath)) return { windows: [], source: 'unavailable' };
try {
const snap = JSON.parse(readFileSync(snapPath, 'utf8'));
diff --git a/claude-ops/scripts/ops-accounts/providers/factory.mjs b/claude-ops/scripts/ops-accounts/providers/factory.mjs
index bce13c57..4b1b468a 100644
--- a/claude-ops/scripts/ops-accounts/providers/factory.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/factory.mjs
@@ -9,7 +9,7 @@ export const providerId = 'factory';
export const displayName = 'Factory / Droid';
export async function listAccounts(ctx) {
- const snapPath = join(ctx.home, '.claude/crs-keys/quota/factory-droid.json');
+ const snapPath = join(ctx.home, '.claude/quota-snapshots/factory-droid.json');
let snap = null;
if (existsSync(snapPath)) {
try {
@@ -38,7 +38,6 @@ export async function listAccounts(ctx) {
status: snap.remaining.status,
}
: null,
- crs: null,
lastError: blocked ? snap?.vendor_detail || 'HTTP 402 no active paid subscription' : snap?.error || null,
billing: snap?.billing_mail_hint || null,
http: snap?.http || null,
@@ -57,7 +56,7 @@ export async function reauth() {
}
export async function utilization(ctx) {
- const snapPath = join(ctx.home, '.claude/crs-keys/quota/factory-droid.json');
+ const snapPath = join(ctx.home, '.claude/quota-snapshots/factory-droid.json');
if (!existsSync(snapPath)) {
return { windows: [], source: 'unavailable' };
}
diff --git a/claude-ops/scripts/ops-accounts/providers/grok.mjs b/claude-ops/scripts/ops-accounts/providers/grok.mjs
index ded6fa7a..3cad6d63 100644
--- a/claude-ops/scripts/ops-accounts/providers/grok.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/grok.mjs
@@ -38,7 +38,7 @@ export async function listAccounts(ctx) {
tokenState,
active: false,
utilization: null,
- crs: { path: 'http://127.0.0.1:31845/v1', note: 'SuperGrok local proxy' },
+ proxy: { path: 'http://127.0.0.1:31845/v1', note: 'SuperGrok local proxy' },
lastError: null,
reauth: {
engine: 'grok-oauth-reauth',
diff --git a/claude-ops/scripts/ops-accounts/providers/openai.mjs b/claude-ops/scripts/ops-accounts/providers/openai.mjs
index 76865350..960ad5fb 100644
--- a/claude-ops/scripts/ops-accounts/providers/openai.mjs
+++ b/claude-ops/scripts/ops-accounts/providers/openai.mjs
@@ -38,7 +38,6 @@ export async function listAccounts(ctx) {
tokenState,
active: true,
utilization: null,
- crs: null,
lastError: null,
configPresent: existsSync(config),
reauth: { status: 'unavailable', note: 'use ChatGPT OAuth / codex login; no plugin cascade yet' },
diff --git a/claude-ops/scripts/ops-cron-pocket-watcher.py b/claude-ops/scripts/ops-cron-pocket-watcher.py
index 490b5465..76dd0e9e 100755
--- a/claude-ops/scripts/ops-cron-pocket-watcher.py
+++ b/claude-ops/scripts/ops-cron-pocket-watcher.py
@@ -353,23 +353,13 @@ def _resolve_anthropic_auth() -> tuple[str, dict] | tuple[None, None]:
credentials at ~/.claude/.credentials.json) so subscription users don't
pay metered rates. Falls back to ANTHROPIC_API_KEY env / keychain.
"""
- # CRS relay first — fleet-sanctioned path. Subscription OAuth tokens 401 on
- # direct api.anthropic.com after a token rotation; route through the local CRS
- # relay. token from $ANTHROPIC_AUTH_TOKEN or keychain service CRS_KEY.
- crs_tok = os.environ.get("ANTHROPIC_AUTH_TOKEN", "").strip()
- if not crs_tok:
- try:
- r = subprocess.run(
- ["security", "find-generic-password", "-s", "CRS_KEY", "-w"],
- capture_output=True, text=True, timeout=5,
- )
- if r.returncode == 0 and r.stdout.strip():
- crs_tok = r.stdout.strip()
- except (FileNotFoundError, subprocess.TimeoutExpired):
- pass
- if crs_tok:
- crs_base = os.environ.get("ANTHROPIC_BASE_URL", "http://127.0.0.1:3005/api").strip()
- return (f"Bearer {crs_tok}", {"_base_url": crs_base})
+ # A local proxy (CLIProxyAPI) takes precedence when the caller configured
+ # one explicitly. Both variables are required: a bearer token with no base
+ # URL would be sent to api.anthropic.com, which rejects it.
+ proxy_tok = os.environ.get("ANTHROPIC_AUTH_TOKEN", "").strip()
+ proxy_base = os.environ.get("ANTHROPIC_BASE_URL", "").strip()
+ if proxy_tok and proxy_base:
+ return (f"Bearer {proxy_tok}", {"_base_url": proxy_base})
# Try Claude Code OAuth — macOS keychain
try:
blob = subprocess.run(
diff --git a/claude-ops/scripts/ops-memory-extractor.sh b/claude-ops/scripts/ops-memory-extractor.sh
index f8ce2ca2..c8ba9179 100755
--- a/claude-ops/scripts/ops-memory-extractor.sh
+++ b/claude-ops/scripts/ops-memory-extractor.sh
@@ -82,20 +82,17 @@ resolve_auth() {
OPS_AUTH_EXTRA_HEADERS=()
OPS_BASE_URL="${OPS_BASE_URL:-}"
- # ─ CRS relay first ─ (fleet-sanctioned path; subscription OAuth tokens now 401
- # on direct api.anthropic.com after a token rotation). base_url defaults to the
- # local CRS relay; token from $ANTHROPIC_AUTH_TOKEN or keychain service CRS_KEY.
- local _crs_base="${ANTHROPIC_BASE_URL:-http://127.0.0.1:3005/api}"
- local _crs_tok="${ANTHROPIC_AUTH_TOKEN:-}"
- if [[ -z "${_crs_tok}" ]] && command -v security &>/dev/null; then
- _crs_tok=$(security find-generic-password -s CRS_KEY -w 2>/dev/null || true)
- fi
- if [[ -n "${_crs_tok}" ]]; then
- OPS_BASE_URL="${_crs_base}"
- OPS_AUTH_HEADER="Authorization: Bearer ${_crs_tok}"
- OPS_AUTH_MODE="crs"
+ # ─ Local proxy first ─ when the caller configured one explicitly (CLIProxyAPI).
+ # Both variables are required: a bearer token with no base URL would be sent to
+ # api.anthropic.com, which rejects it.
+ local _proxy_base="${ANTHROPIC_BASE_URL:-}"
+ local _proxy_tok="${ANTHROPIC_AUTH_TOKEN:-}"
+ if [[ -n "${_proxy_tok}" && -n "${_proxy_base}" ]]; then
+ OPS_BASE_URL="${_proxy_base}"
+ OPS_AUTH_HEADER="Authorization: Bearer ${_proxy_tok}"
+ OPS_AUTH_MODE="proxy"
OPS_AUTH_EXTRA_HEADERS=()
- log "Auth: CRS relay (${_crs_base})"
+ log "Auth: local proxy (${_proxy_base})"
return 0
fi
diff --git a/claude-ops/scripts/systemd/claude-account-rotation.service b/claude-ops/scripts/systemd/claude-account-rotation.service
index d7338b71..3a38a3ed 100644
--- a/claude-ops/scripts/systemd/claude-account-rotation.service
+++ b/claude-ops/scripts/systemd/claude-account-rotation.service
@@ -1,7 +1,7 @@
[Unit]
Description=Claude account rotation daemon (plugin-first)
-After=network-online.target crs-compose.service
-Wants=network-online.target crs-compose.service
+After=network-online.target
+Wants=network-online.target
[Service]
Type=simple
diff --git a/claude-ops/scripts/systemd/crs-bedrock-guard.service b/claude-ops/scripts/systemd/crs-bedrock-guard.service
deleted file mode 100644
index 1bc2d246..00000000
--- a/claude-ops/scripts/systemd/crs-bedrock-guard.service
+++ /dev/null
@@ -1,10 +0,0 @@
-[Unit]
-Description=Detect CRS Bedrock fallback traffic
-After=crs-compose.service
-Wants=crs-compose.service
-
-[Service]
-Type=oneshot
-ExecStart=/usr/bin/node %h/.claude/scripts/account-rotation/crs-bedrock-guard.mjs
-StandardOutput=append:%h/.claude/logs/crs-bedrock-guard.systemd.log
-StandardError=append:%h/.claude/logs/crs-bedrock-guard.systemd.log
diff --git a/claude-ops/scripts/systemd/crs-bedrock-guard.timer b/claude-ops/scripts/systemd/crs-bedrock-guard.timer
deleted file mode 100644
index e07e419c..00000000
--- a/claude-ops/scripts/systemd/crs-bedrock-guard.timer
+++ /dev/null
@@ -1,11 +0,0 @@
-[Unit]
-Description=Run CRS Bedrock fallback guard periodically
-
-[Timer]
-OnBootSec=3min
-OnUnitActiveSec=5min
-Persistent=true
-Unit=crs-bedrock-guard.service
-
-[Install]
-WantedBy=timers.target
diff --git a/claude-ops/scripts/systemd/crs-compose.service b/claude-ops/scripts/systemd/crs-compose.service
deleted file mode 100644
index 76da9f5d..00000000
--- a/claude-ops/scripts/systemd/crs-compose.service
+++ /dev/null
@@ -1,15 +0,0 @@
-[Unit]
-Description=Claude Relay Service docker compose stack
-Requires=docker.service
-After=docker.service network-online.target
-
-[Service]
-Type=oneshot
-WorkingDirectory=%h/crs
-ExecStart=/usr/bin/docker compose up -d --remove-orphans
-ExecStop=/usr/bin/docker compose down
-RemainAfterExit=yes
-TimeoutStartSec=120
-
-[Install]
-WantedBy=default.target
diff --git a/claude-ops/scripts/systemd/crs-token-feed.service b/claude-ops/scripts/systemd/crs-token-feed.service
deleted file mode 100644
index c63ad8c2..00000000
--- a/claude-ops/scripts/systemd/crs-token-feed.service
+++ /dev/null
@@ -1,13 +0,0 @@
-[Unit]
-Description=Feed fresh Claude OAuth tokens into local CRS
-After=crs-compose.service
-Wants=crs-compose.service
-
-[Service]
-Type=oneshot
-PassEnvironment=CLAUDE_AUTH_COORDINATION_CONFIG CLAUDE_ROTATOR_CONFIG
-ExecCondition=/bin/sh -c 'test -n "$CLAUDE_AUTH_COORDINATION_CONFIG" && test -n "$CLAUDE_ROTATOR_CONFIG"'
-Environment=CRS_BASE=http://127.0.0.1:3005
-ExecStart=%h/.claude/scripts/account-rotation/crs-token-feed.sh
-StandardOutput=append:%h/.claude/logs/crs-token-feed.systemd.log
-StandardError=append:%h/.claude/logs/crs-token-feed.systemd.log
diff --git a/claude-ops/scripts/systemd/crs-token-feed.timer b/claude-ops/scripts/systemd/crs-token-feed.timer
deleted file mode 100644
index 591670f2..00000000
--- a/claude-ops/scripts/systemd/crs-token-feed.timer
+++ /dev/null
@@ -1,11 +0,0 @@
-[Unit]
-Description=Run CRS token feed periodically
-
-[Timer]
-OnBootSec=2min
-OnUnitActiveSec=5min
-Persistent=true
-Unit=crs-token-feed.service
-
-[Install]
-WantedBy=timers.target
diff --git a/claude-ops/skills/ops-accounts/SKILL.md b/claude-ops/skills/ops-accounts/SKILL.md
index 65dc9d5f..9d356201 100644
--- a/claude-ops/skills/ops-accounts/SKILL.md
+++ b/claude-ops/skills/ops-accounts/SKILL.md
@@ -1,7 +1,7 @@
---
name: ops-accounts
-description: Multi-provider AI account manager (Claude, Grok/xAI, OpenAI/Codex, Factory, Cursor). Status, setup, switch, refresh, reauth, util, optional CRS/LB. Canonical replacement for /ops:rotate and /ops:rotate-setup (those remain aliases).
-argument-hint: '[status|list|setup|switch|refresh|reauth|util|rotate-now|crs|crs-tick|help] [provider] [args…]'
+description: Multi-provider AI account manager (Claude, Grok/xAI, OpenAI/Codex, Factory, Cursor). Status, setup, switch, refresh, reauth, util, and the CLIProxyAPI pool. Canonical replacement for /ops:rotate and /ops:rotate-setup (those remain aliases).
+argument-hint: '[status|list|setup|switch|refresh|reauth|util|rotate-now|seats|gateway|help] [provider] [args…]'
allowed-tools:
- Bash
- Read
@@ -25,7 +25,7 @@ for every provider:
| Refresh tokens | `refresh` |
| Unattended reauth | `reauth` |
| Utilization / quota | `util` |
-| Optional Claude LB (CRS or future gateway) | `crs`, `crs-tick` |
+| Pooled seats across accounts | CLIProxyAPI (see `/ops:ops-fleet`) |
**Aliases (compat):** `/ops:rotate` → this skill (Claude-focused shortcuts).
`/ops:rotate-setup` → `setup` (wizard). `/ops:account` → same as this skill.
@@ -35,12 +35,12 @@ for every provider:
| Provider | Engine | Reauth | Util |
|----------|--------|--------|------|
| Claude | `scripts/account-rotation/rotate.mjs` + staged enrollment | signed stage/activate only | 5h/7d |
-| Grok | slots + `grok-cli-auth-proxy` (+ optional CRS hop) | device-code + Google (dcli); residential egress cascade | weekly / 429 |
+| Grok | slots + `grok-cli-auth-proxy` | device-code + Google (dcli); residential egress cascade | weekly / 429 |
| OpenAI / Codex | `codex-rotate` when present | OAuth bridge | usage best-effort |
| Factory | adapter TBD / quota-feed seeds | native | quota-feed patterns |
| Cursor | adapter TBD | browser/device OAuth | plan limits if available |
-**CRS is optional.** For Grok, CRS is only a thin hop to the SuperGrok OAuth proxy — multi-seat RR lives on the proxy, not a CRS account table. See `docs/ops/OPS-ACCOUNTS-VISION.md` (CRS cherry-pick / no-CRS path).
+**CLIProxyAPI is the only supported multi-account path.** It holds one OAuth seat file per account and answers locally; `rotate.mjs` writes those seat files as part of a rotation. The earlier relay backend (claude-relay-service) has been removed — it required a static bearer token in `settings.json` for every session. For Grok, multi-seat round-robin lives on `grok-cli-auth-proxy`.
## Router (`bin/ops-accounts`)
@@ -62,20 +62,20 @@ PLUGIN_ROOT="${CLAUDE_PLUGIN_ROOT:-$(ls -d "$HOME/.claude/plugins/cache/ops-mark
| `reauth grok <email>` | Device reauth with residential cascade env |
| `setup` / `setup claude` | Claude direct auth is disabled; staged enrollment required |
| `setup grok` | Add/reauth SuperGrok seat |
-| `crs` / `crs-tick` | Optional Claude CRS pool status / one tick |
-| `seats` | Local multi-provider seat-state (no CRS) |
+| `seats` | Local multi-provider seat-state |
+| `gateway` | Local OpenAI-compat gateway: status/start/path/self-test/config |
## Claude setup / OAuth
When `$ARGUMENTS` is `setup`, `setup claude`, or this skill is invoked via
**ops-rotate-setup** alias: follow the full wizard in
-`skills/ops-rotate-setup/SKILL.md` (Steps 1–5, CRS optional 4.4–4.7). That file
+`skills/ops-rotate-setup/SKILL.md`. That file
remains the detailed Claude setup procedure; this skill is the entrypoint.
-Claude day-2 ops (`status`/`rotate-now`/`list`/`reauth`/`crs`) also match
+Claude day-2 ops (`status`/`rotate-now`/`list`/`reauth`) also match
`skills/ops-rotate/SKILL.md` — treat that as Claude detail appendix.
-## Local seat-state (no CRS)
+## Local seat-state
```bash
"$PLUGIN_ROOT/bin/ops-accounts" seats status
@@ -87,7 +87,7 @@ File: `$CLAUDE_PLUGIN_DATA_DIR/account-rotation/seat-state.json` (or `OPS_ACCOUN
## Grok notes
-1. CLI models often use `base_url` → CRS `/grok/v1` → **host OAuth proxy** → SuperGrok seats.
+1. CLI models often use `base_url` → **host OAuth proxy** → SuperGrok seats.
2. `grok-rotate` / `auth.json` is the SuperGrok seat set; keep **auth-slots** in sync after reauth.
3. Reauth egress: EFG SOCKS (`GROK_REAUTH_SOCKS`) → Bright Data tiers via
`scripts/account-rotation/grok-reauth-egress.sh` (residential cascade).
@@ -98,11 +98,11 @@ File: `$CLAUDE_PLUGIN_DATA_DIR/account-rotation/seat-state.json` (or `OPS_ACCOUN
1. Never print tokens, cookies, OTP codes, or vault dumps.
2. Never write real emails into committed files.
3. Prefer `bin/ops-accounts` over ad-hoc host paths.
-4. Missing CRS is not a failure.
-5. Dead RT → `reauth`, not “install CRS.”
+4. A missing CLIProxyAPI is not a failure — single-seat rotation works without it.
+5. Dead RT → `reauth`, never a relay install.
6. Background long OAuth (Rule 4).
## Phase map
-0 contract + this skill · 1 Claude parity · 2 CRS optional · 3 Grok complete ·
-4 Codex+Factory · 5 Cursor · 6 companions · 7 host cutover · 8 gateway (no CRS)
+0 contract + this skill · 1 Claude parity · 2 CLIProxyAPI pool · 3 Grok complete ·
+4 Codex+Factory · 5 Cursor · 6 companions · 7 host cutover · 8 local gateway
diff --git a/claude-ops/skills/ops-desk/SKILL.md b/claude-ops/skills/ops-desk/SKILL.md
index 4a6855e7..9ab3a1c6 100644
--- a/claude-ops/skills/ops-desk/SKILL.md
+++ b/claude-ops/skills/ops-desk/SKILL.md
@@ -72,7 +72,7 @@ script encodes:
tracker mutations from inside the workflow — they read, verify, and prepare only. All
execution happens later in the main session under the outbound-approval gate.
- **Batched fan-out (3 at a time).** A wide parallel burst can rate-limit (429) the
- model-account pool — relay/CRS setups especially. Batches of 3 finish nearly as fast
+ model-account pool — pooled CLIProxyAPI setups especially. Batches of 3 finish nearly as fast
and never starve the rest of the fleet.
- **Verification over trust.** Every load-bearing claim in a package must come from an
actual read (issue body, thread, sqlite row) with dates/ids cited in `status_summary`.
diff --git a/claude-ops/skills/ops-mac/SKILL.md b/claude-ops/skills/ops-mac/SKILL.md
index f5195c44..5fcec4a1 100644
--- a/claude-ops/skills/ops-mac/SKILL.md
+++ b/claude-ops/skills/ops-mac/SKILL.md
@@ -116,7 +116,7 @@ For each actionable entry from `lanchr doctor` (missing binary, or repeated `exi
com.example.foo — binary missing at [path]. This launchd job is dead.
[Show the plist first] [Remove it] [Disable (keep file)] [Skip]
```
-Use `lanchr` to remediate where possible; otherwise `launchctl bootout`/`disable` after showing the plist. **Never touch a daemon the user relies on** (CRS/haproxy, gbrain push, ops-daemon, cloudflared tunnels, watchdogs) without spelling out exactly what it is — cross-check before recommending removal.
+Use `lanchr` to remediate where possible; otherwise `launchctl bootout`/`disable` after showing the plist. **Never touch a daemon the user relies on** (CLIProxyAPI/haproxy, gbrain push, ops-daemon, cloudflared tunnels, watchdogs) without spelling out exactly what it is — cross-check before recommending removal.
### 3. Reclaimable disk
```
diff --git a/claude-ops/skills/ops-rotate/SKILL.md b/claude-ops/skills/ops-rotate/SKILL.md
index 0f147675..cca28071 100644
--- a/claude-ops/skills/ops-rotate/SKILL.md
+++ b/claude-ops/skills/ops-rotate/SKILL.md
@@ -1,7 +1,7 @@
---
name: ops-rotate
-description: Alias of /ops:accounts for Claude Max seats (status, rotate-now, list, reauth, optional CRS). Prefer /ops:accounts for multi-provider. Full Claude procedure still in this file's historical detail via ops-accounts router.
-argument-hint: '[status|rotate-now|list|add-account|reauth|crs|crs-tick]'
+description: Alias of /ops:accounts for Claude Max seats (status, rotate-now, list, reauth). Prefer /ops:accounts for multi-provider. Full Claude procedure still in this file's historical detail via ops-accounts router.
+argument-hint: '[status|rotate-now|list|add-account|reauth]'
allowed-tools:
- Bash
- Read
@@ -23,7 +23,7 @@ This skill is a **compat alias**. Route all work through **ops-accounts**:
| list | `ops-accounts list` |
| add-account | `ops-accounts setup claude` |
| reauth | `ops-accounts reauth claude <email>` |
-| crs / crs-tick | `ops-accounts crs` / `ops-accounts crs-tick` |
+| pooled seats | CLIProxyAPI — see `/ops:ops-fleet` |
Load **`skills/ops-accounts/SKILL.md`** first, then for Claude-only depth use the
bin:
diff --git a/claude-ops/templates/com.claude-ops.crs-429-cooldown.plist b/claude-ops/templates/com.claude-ops.crs-429-cooldown.plist
deleted file mode 100644
index a71ca73b..00000000
--- a/claude-ops/templates/com.claude-ops.crs-429-cooldown.plist
+++ /dev/null
@@ -1,56 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- CRS 429-cooldown reconciler (claude-relay-service pool rate-limit hold).
-
- Schedule: every 60s. Equivalent systemd: OnUnitActiveSec=60s.
- The installer (scripts/install-crs-reconcilers-agent.sh) substitutes
- __WRAPPER_PATH__, __LOG_DIR__, __HOME__ and writes the rendered copy to
- ~/Library/LaunchAgents/com.claude-ops.crs-429-cooldown.plist before:
- launchctl bootstrap "gui/$(id -u)" <plist>
-
- To uninstall:
- launchctl bootout "gui/$(id -u)/com.claude-ops.crs-429-cooldown"
- rm ~/Library/LaunchAgents/com.claude-ops.crs-429-cooldown.plist
--->
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>Label</key>
- <string>com.claude-ops.crs-429-cooldown</string>
-
- <key>ProgramArguments</key>
- <array>
- <string>/bin/bash</string>
- <string>__WRAPPER_PATH__</string>
- </array>
-
- <key>StartInterval</key>
- <integer>60</integer>
-
- <key>ThrottleInterval</key>
- <integer>20</integer>
-
- <key>RunAtLoad</key>
- <true/>
-
- <key>StandardOutPath</key>
- <string>__LOG_DIR__/crs-429-cooldown-launchd.log</string>
- <key>StandardErrorPath</key>
- <string>__LOG_DIR__/crs-429-cooldown-launchd.log</string>
-
- <key>EnvironmentVariables</key>
- <dict>
- <key>PATH</key>
- <string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin</string>
- <key>HOME</key>
- <string>__HOME__</string>
- <key>NODE_NO_WARNINGS</key>
- <string>1</string>
- </dict>
-
- <key>ProcessType</key>
- <string>Background</string>
- <key>LowPriorityIO</key>
- <true/>
-</dict>
-</plist>
diff --git a/claude-ops/templates/com.claude-ops.crs-fra-tunnel.plist b/claude-ops/templates/com.claude-ops.crs-fra-tunnel.plist
deleted file mode 100644
index 88accbef..00000000
--- a/claude-ops/templates/com.claude-ops.crs-fra-tunnel.plist
+++ /dev/null
@@ -1,40 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>Label</key>
- <string>com.claude-ops.crs-fra-tunnel</string>
-
- <key>ProgramArguments</key>
- <array>
- <string>/usr/bin/ssh</string>
- <string>-N</string>
- <string>-o</string>
- <string>BatchMode=yes</string>
- <string>-o</string>
- <string>ExitOnForwardFailure=yes</string>
- <string>-o</string>
- <string>ServerAliveInterval=30</string>
- <string>-o</string>
- <string>ServerAliveCountMax=3</string>
- <string>-L</string>
- <string>127.0.0.1:__LOCAL_PORT__:127.0.0.1:__REMOTE_PORT__</string>
- <string>__SSH_HOST__</string>
- </array>
-
- <key>KeepAlive</key>
- <true/>
-
- <key>ThrottleInterval</key>
- <integer>30</integer>
-
- <key>RunAtLoad</key>
- <true/>
-
- <key>StandardOutPath</key>
- <string>__LOG_DIR__/crs-remote-tunnel.out.log</string>
-
- <key>StandardErrorPath</key>
- <string>__LOG_DIR__/crs-remote-tunnel.err.log</string>
-</dict>
-</plist>
diff --git a/claude-ops/templates/com.claude-ops.crs-priority.plist b/claude-ops/templates/com.claude-ops.crs-priority.plist
deleted file mode 100644
index aecc78e8..00000000
--- a/claude-ops/templates/com.claude-ops.crs-priority.plist
+++ /dev/null
@@ -1,56 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- CRS account-priority daemon (claude-relay-service pool auto-prioritization).
-
- Schedule: every 120s. Equivalent systemd: OnUnitActiveSec=120s.
- The installer (scripts/install-crs-priority-agent.sh) substitutes
- __WRAPPER_PATH__, __LOG_DIR__, __HOME__ and writes the rendered copy to
- ~/Library/LaunchAgents/com.claude-ops.crs-priority.plist before:
- launchctl bootstrap "gui/$(id -u)" <plist>
-
- To uninstall:
- launchctl bootout "gui/$(id -u)/com.claude-ops.crs-priority"
- rm ~/Library/LaunchAgents/com.claude-ops.crs-priority.plist
--->
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>Label</key>
- <string>com.claude-ops.crs-priority</string>
-
- <key>ProgramArguments</key>
- <array>
- <string>/bin/bash</string>
- <string>__WRAPPER_PATH__</string>
- </array>
-
- <key>StartInterval</key>
- <integer>120</integer>
-
- <key>ThrottleInterval</key>
- <integer>30</integer>
-
- <key>RunAtLoad</key>
- <true/>
-
- <key>StandardOutPath</key>
- <string>__LOG_DIR__/crs-priority-launchd.log</string>
- <key>StandardErrorPath</key>
- <string>__LOG_DIR__/crs-priority-launchd.log</string>
-
- <key>EnvironmentVariables</key>
- <dict>
- <key>PATH</key>
- <string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin</string>
- <key>HOME</key>
- <string>__HOME__</string>
- <key>NODE_NO_WARNINGS</key>
- <string>1</string>
- </dict>
-
- <key>ProcessType</key>
- <string>Background</string>
- <key>LowPriorityIO</key>
- <true/>
-</dict>
-</plist>
diff --git a/claude-ops/templates/com.claude-ops.magic-link-autoloop.plist b/claude-ops/templates/com.claude-ops.magic-link-autoloop.plist
deleted file mode 100644
index a571d758..00000000
--- a/claude-ops/templates/com.claude-ops.magic-link-autoloop.plist
+++ /dev/null
@@ -1,76 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Magic-link autoloop reconciler (opt-in unattended re-auth for confirmed
- dead-refresh-token accounts).
-
- Schedule: every 600s. Equivalent systemd: OnUnitActiveSec=600s.
- The installer (scripts/install-crs-reconcilers-agent.sh) substitutes
- __WRAPPER_PATH__, __LOG_DIR__, __HOME__, __PATH__, __DISPLAY__ from the
- host environment (no hardcoded package-manager paths in the template) and
- writes the rendered copy to ~/Library/LaunchAgents/ before:
- launchctl bootstrap "gui/$(id -u)" <plist>
-
- To uninstall:
- launchctl bootout "gui/$(id -u)/com.claude-ops.magic-link-autoloop"
- rm ~/Library/LaunchAgents/com.claude-ops.magic-link-autoloop.plist
--->
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>Label</key>
- <string>com.claude-ops.magic-link-autoloop</string>
-
- <key>ProgramArguments</key>
- <array>
- <string>/bin/bash</string>
- <string>__WRAPPER_PATH__</string>
- </array>
-
- <key>StartInterval</key>
- <integer>600</integer>
-
- <key>ThrottleInterval</key>
- <integer>60</integer>
-
- <key>RunAtLoad</key>
- <false/>
-
- <key>StandardOutPath</key>
- <string>__LOG_DIR__/magic-link-autoloop-launchd.log</string>
- <key>StandardErrorPath</key>
- <string>__LOG_DIR__/magic-link-autoloop-launchd.log</string>
-
- <key>EnvironmentVariables</key>
- <dict>
- <!-- Installer should expand PATH from the host; keep template free of OS package roots. -->
- <key>PATH</key>
- <string>__PATH__</string>
- <key>HOME</key>
- <string>__HOME__</string>
- <key>NODE_NO_WARNINGS</key>
- <string>1</string>
- <!-- Unattended reauth seat (override per host via unit drop-in if needed) -->
- <key>CLAUDE_ROT_HEADED</key>
- <string>1</string>
- <key>CLAUDE_DESKTOP_DISPLAY</key>
- <string>__DISPLAY__</string>
- <key>DESKTOP_ACT_DISPLAY</key>
- <string>__DISPLAY__</string>
- <key>CLAUDE_ROT_VISUAL_CAPTCHA</key>
- <string>1</string>
- <key>CLAUDE_ROT_DESKTOP_ACT</key>
- <string>1</string>
- <key>CLAUDE_ROT_VNC_AGENT</key>
- <string>1</string>
- <key>CRS_MAGIC_LINK_ROTATE_TIMEOUT_MS</key>
- <string>1200000</string>
- <key>CLAUDE_ROT_SKIP_TOKEN_SOLVERS</key>
- <string>0</string>
- </dict>
-
- <key>ProcessType</key>
- <string>Background</string>
- <key>LowPriorityIO</key>
- <true/>
-</dict>
-</plist>
diff --git a/claude-ops/templates/statusline/statusline-command.sh b/claude-ops/templates/statusline/statusline-command.sh
index 8465f125..8f1accd5 100644
--- a/claude-ops/templates/statusline/statusline-command.sh
+++ b/claude-ops/templates/statusline/statusline-command.sh
@@ -259,20 +259,20 @@ _email_safe=$(printf '%s' "$account_email" | tr -d '/"')
# Billing mode. Bedrock env overrides; else derive from auth JSON; persist
# positive results so the badge survives transient auth failures.
-# CRS-relay guard: a stale CLAUDE_CODE_USE_BEDROCK=1 can linger in the process
-# env even though the session actually routes through the CRS relay
-# (ANTHROPIC_BASE_URL -> 127.0.0.1 relay + a cr_ OAuth token = Max pool / oauth).
-# In that case the real provider is oauth, not Bedrock. We also force "max" and
+# Local-proxy guard: a stale CLAUDE_CODE_USE_BEDROCK=1 can linger in the process
+# env even though the session actually routes through a local proxy on loopback
+# (CLIProxyAPI, or a leftover relay on an upgraded machine). In that case the
+# real provider is the pooled OAuth account, not Bedrock. Force "max" and
# overwrite any stale cache so a session that briefly cached "bedrock" at startup
-# (before relay routing settled) never keeps showing Bedrock.
+# (before proxy routing settled) never keeps showing Bedrock.
billing_cache="${CACHE_DIR}/claude-billing-${session_id}"
-_is_crs_relay=""
-case "${CLAUDE_CODE_OAUTH_TOKEN:-}" in cr_*)
- case "${ANTHROPIC_BASE_URL:-}" in *127.0.0.1:3005*|*127.0.0.1:3000*|*localhost:3005*|*localhost:3000*) _is_crs_relay=1 ;; esac
+_is_local_proxy=""
+case "${ANTHROPIC_BASE_URL:-}" in
+ http://127.0.0.1:*|http://localhost:*|http://[::1]:*) _is_local_proxy=1 ;;
esac
-if [ "${CLAUDE_CODE_USE_BEDROCK:-}" = "1" ] && [ -z "$_is_crs_relay" ]; then
+if [ "${CLAUDE_CODE_USE_BEDROCK:-}" = "1" ] && [ -z "$_is_local_proxy" ]; then
sub_type="bedrock"
-elif [ -n "$_is_crs_relay" ]; then
+elif [ -n "$_is_local_proxy" ]; then
sub_type="max"
elif [ -n "$_sub" ] && [ "$_sub" != "null" ]; then
sub_type="$_sub"
diff --git a/claude-ops/tests/run-all.sh b/claude-ops/tests/run-all.sh
index 9e60f2a1..60d9c368 100755
--- a/claude-ops/tests/run-all.sh
+++ b/claude-ops/tests/run-all.sh
@@ -42,7 +42,7 @@ run_suite "$TESTS_DIR/test-specialist-agent-swap.sh"
run_suite "$TESTS_DIR/test-rotate-cliproxy-integration.sh"
run_suite "$TESTS_DIR/test-model-args.sh"
run_suite "$TESTS_DIR/test-account-recovery.sh"
-run_suite "$TESTS_DIR/../scripts/crsproxy-reauth/test-python-suites.sh"
+run_suite "$TESTS_DIR/test-claude-invoke.sh"
run_suite "$TESTS_DIR/test-staged-enrollment.sh"
run_suite "$TESTS_DIR/test-coordination-provisioning.sh"
run_suite "$TESTS_DIR/test-template.sh"
diff --git a/claude-ops/tests/test-claude-invoke.sh b/claude-ops/tests/test-claude-invoke.sh
index 228302b2..26abe91c 100755
--- a/claude-ops/tests/test-claude-invoke.sh
+++ b/claude-ops/tests/test-claude-invoke.sh
@@ -7,7 +7,7 @@
# T3: CLAUDE_OPS_USE_CREDIT_POOL=1 -> calls `node .../claude-p-as.mjs --` with sanitized args
# T4: CLAUDE_OPS_USE_CREDIT_POOL=1 but wrapper missing -> falls back to `claude`, exits non-zero
# (warning printed to stderr, direct claude called)
-# T5: direct invocation strips API-key env so CRS auth token wins
+# T5: direct invocation strips API-key env so the OAuth credential wins
set -euo pipefail
PLUGIN_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
diff --git a/claude-ops/tests/test-rotate-cliproxy-integration.sh b/claude-ops/tests/test-rotate-cliproxy-integration.sh
index c5f14e29..cc5f9911 100755
--- a/claude-ops/tests/test-rotate-cliproxy-integration.sh
+++ b/claude-ops/tests/test-rotate-cliproxy-integration.sh
@@ -47,7 +47,6 @@ const denied = spawnSync(process.execPath, [join(root, 'scripts/account-rotation
PATH: `${bin}:${process.env.PATH}`,
CLAUDE_ROTATOR_CONFIG: config,
CLAUDE_ROTATION_FILE_VAULT: vault,
- CLAUDE_ROTATION_SKIP_CRS_SYNC: '1',
},
});
assert.equal(denied.status, 0, denied.stderr);
diff --git a/docs/plans/2026-08-16-crs-removal-plan.md b/docs/plans/2026-08-16-crs-removal-plan.md
index c42fa5d8..684b65a0 100644
--- a/docs/plans/2026-08-16-crs-removal-plan.md
+++ b/docs/plans/2026-08-16-crs-removal-plan.md
@@ -1,8 +1,16 @@
# CRS removal plan
Date: 2026-08-16
-Status: proposal. Nothing in this plan has been executed. No existing file was edited to produce it.
-Scope: remove claude-relay-service (CRS) from this repo and re-offer it as an optional installable dependency.
+Status: **executed, with a changed scope.** This document is the survey that
+preceded the removal; it is kept as the record of what the tree looked like
+beforehand. Every path and line number below describes the pre-removal tree.
+
+The plan proposed extracting CRS into an optional installable companion. The
+owner decided instead that CRS is insecure and must be removed outright, with
+CLIProxyAPI as the only supported path for multi-account rotation and OAuth
+seat management. There is no companion package. Section 8 was not built.
+
+Scope as executed: remove claude-relay-service (CRS) from this repo.
All paths are repo-relative. Line numbers are from the tree at the time of writing.
diff --git a/docs/runbooks/crs-full-tuning-plan.md b/docs/runbooks/crs-full-tuning-plan.md
deleted file mode 100644
index a7b9c3b6..00000000
--- a/docs/runbooks/crs-full-tuning-plan.md
+++ /dev/null
@@ -1,269 +0,0 @@
-# CRS Full-Tuning Execution Plan (FRA EC2)
-
-> Owner: <owner> · Created: 2026-06-19
->
-> Scope: tune the **Claude Relay Service (CRS)** running on the Frankfurt EC2 box so the
-> agent fleet load-balances dynamically across the account pool, cools exhausted accounts
-> correctly, and survives concurrent-stream storms without `ConnectionRefused` cascades.
-> This plan is grounded in the **actual** CRS deployment as verified on the box on
-> 2026-06-19 — not generic relay advice.
-
----
-
-## 0. Ground truth — the actual CRS deployment
-
-| Fact | Value (verified 2026-06-19) |
-|---|---|
-| Host | `dev-sandbox-fra` EC2 (`i-071b16e6328dbc876`, eu-central-1). SSM-managed; reachable via SSH alias `fra` (tailnet `100.107.38.70`) or `dev-sandbox-fra-cf` (CF-Access SSH) |
-| CRS dir | `/home/ec2-user/crs` — **a working copy on the box, NOT a git repo** (`git remote -v` empty). The Dockerfile/package.json/lock are on-disk; `docker-compose.yml` has `build: .` |
-| Relay container | `crs-claude-relay-1` (`weishaw/claude-relay-service`), bound `127.0.0.1:3005 → 3000`, status healthy |
-| Live image | `weishaw/claude-relay-service:cve-fix-20260615` (the CVE-clean node:22-alpine rebuild). **NOT** a `loadbalance-fix-*` tag — see §1 gap |
-| Redis | `crs-redis-1` (`redis:7-alpine`, :6379). Account state + concurrency counters + utilization live here |
-| Aux containers | redis-commander, prometheus, grafana (compose-defined) |
-| Account pool | ~10 Claude **Max OAuth** accounts, named `pool-<label>` per owner/brand plus a few `canary-<label>` entries. Records: Redis hash `claude:account:<uuid>` |
-| Token refresh | `crs-token-feed.timer` (systemd `--user`, every 5 min) — FRA is the OAuth refresher-of-record |
-| Coarse priority cron | `dynamic-priority.py` (every 5 min) writes CRS `priority` from the same util formula |
-| Routing in | Fleet sessions route via overlay `~/.claude/crs-session-settings.json` (`ANTHROPIC_BASE_URL=…:3005/api` + `ANTHROPIC_AUTH_TOKEN=cr_…`). The `cr_` key MUST be `ANTHROPIC_AUTH_TOKEN`, never `ANTHROPIC_API_KEY` |
-
-### Key scheduler/relay files on disk
-- `src/services/scheduler/unifiedClaudeScheduler.js` — pool selection, concurrency gating, util load-balancing, overload/503 handling, rate-limit tracking. **(modified 2026-06-19 — newest file)**
-- `src/services/account/claudeAccountService.js` — account state, `markAccountRateLimited()`, util writeback (`claudeFiveHourResetsAt`, etc.)
-- `src/services/relay/claudeRelayService.js` — request proxy + 429/error mark sites (non-stream / stream-error / stream-end)
-- `config/config.js` (+ `config.example.js`, `models.js`) — defaults
-
-### What is ALREADY on disk (verified by grep)
-- **Util-based concurrency cap** — `_getMaxConcurrencyFromUtilization(util)` returns a per-account max concurrent stream count keyed on 5-hour utilization:
-
- ```js
- // src/services/scheduler/unifiedClaudeScheduler.js:593
- _getMaxConcurrencyFromUtilization(utilization) {
- const util = parseFloat(utilization) || 0
- if (util < 20) return 8
- if (util < 40) return 5
- if (util < 60) return 3
- if (util < 80) return 2
- return 1
- }
- ```
- Enforced at selection time (line ~801): `currentConcurrency = redis.getClaudeAccountConcurrency(account.id)`; account skipped if `currentConcurrency >= maxConcurrency`.
-- **Dedicated 503 when all accounts saturated** — `CONSOLE_ACCOUNT_CONCURRENCY_FULL` error code thrown when every eligible account is at its concurrency limit.
-- **Util load-balancing** — pool picks sorted by live utilization (`claudeFiveHourUtilization*0.4 + claudeSevenDayUtilization*0.6`), tie-break priority then `lastUsedAt` (the `📊 Load-balance pick` path).
-- **Rate-limit tracking** — `rateLimitEndAt` / `rateLimitResetTimestamp` per account, `overloaded` status handled.
-
----
-
-## 1. ⚠️ The critical gap — credit-exhaustion cooldown is NOT on disk
-
-This is the **single highest-priority finding** and the most likely cause of repeat-account hammering.
-
-**Symptom (documented 2026-06-16):** an `out_of_credits` / `org_level_disabled` account
-gets re-selected every ~90 s and re-429'd (~40 wasted 429 calls/min). These 429s carry
-**no authoritative reset header** (`anthropic-ratelimit-unified-overage-disabled-reason: out_of_credits`,
-`overage-status: rejected`, body "credit balance too low"), so the relay only applied the
-short transient cooldown and re-hammered the dead account.
-
-**The 2026-06-16 fix (`loadbalance-fix-20260616`) introduced:**
-- `_detectCreditExhaustion(status, headers, body)` in `claudeRelayService.js`
-- wired into all 3× 429 mark sites → passes `{creditExhausted}` to `markAccountRateLimited()`
-- which cools the account until its `claudeFiveHourResetsAt` (if future) else
- `CREDIT_EXHAUSTION_COOLDOWN_SECONDS` (default **3600 s / 1 h**); transient no-reset 429s
- keep the short 90 s.
-
-**Verified 2026-06-19: NONE of `_detectCreditExhaustion`, `creditExhausted`, or
-`CREDIT_EXHAUSTION_COOLDOWN_SECONDS` exist on disk.** The live image is `cve-fix-20260615`,
-which predates this fix. **Conclusion: the CVE rebuild on 2026-06-15 produced an image that
-never carried the credit-cooldown fix, OR a later restore reverted it.** The util-LB +
-concurrency-cap code survived (it lives in `unifiedClaudeScheduler.js`), but the
-credit-cooldown patch in `claudeRelayService.js` / `claudeAccountService.js` did **not**.
-
-**This must be re-applied before any other tuning** — it is the root cause of the
-out_of_credits 429-hammer and wastes pool capacity on a billing-dead account.
-
----
-
-## 2. Tuning workstreams
-
-### 2.1 Re-apply credit-exhaustion cooldown (P0)
-1. Confirm the diff is recoverable: check the box for `~/crs` git stash / `.bak` copies of
- `claudeRelayService.js` and `claudeAccountService.js`, and for the
- `loadbalance-fix-20260616` image (`docker images | grep loadbalance`). If neither exists,
- re-implement from the spec in §1.
-2. Re-add `_detectCreditExhaustion(status, headers, body)` — strict match on
- `out_of_credits` / `org_level_disabled` / `overage-status: rejected` / "credit balance too low".
-3. Wire it into the 3 × 429 mark sites in `claudeRelayService.js` (non-stream, stream-error,
- stream-end) → pass `{ creditExhausted: true }`.
-4. In `claudeAccountService.js markAccountRateLimited(..., options)`: when `creditExhausted`,
- set cooldown to `max(now → claudeFiveHourResetsAt, CREDIT_EXHAUSTION_COOLDOWN_SECONDS)`.
-5. Fire the `CLAUDE_OAUTH_CREDIT_EXHAUSTED` webhook so billing-dead accounts surface in ops.
-6. Leave transient (no-reset) 429s on the short cooldown (see §2.2 for the value).
-
-### 2.2 Scheduler / cooldown knob tuning
-The cooldowns currently live as **code defaults with no compose override** (verified: the
-`COOLDOWN`/`CREDIT_EXHAUSTION`/`UTILIZATION` env names are absent from `docker-compose.yml`
-and `.env`). Promote them to env knobs so they are tunable without a rebuild:
-
-| Knob | Current default | Recommended | Rationale |
-|---|---|---|---|
-| `CLAUDE_DEFAULT_RATELIMIT_COOLDOWN_SECONDS` | 90 | **90** (keep) | transient no-reset 429s; 90 s is fine for genuine short ratelimits |
-| `CREDIT_EXHAUSTION_COOLDOWN_SECONDS` | 3600 | **3600** (keep) | billing exhaustion; do not re-probe for 1 h |
-| `CLAUDE_UTILIZATION_LOAD_BALANCING` | ON | **ON** | dynamic least-util picks; `=false` reverts to pure priority |
-
-Action: add these three to `docker-compose.yml` `environment:` (with `${VAR:-default}` form,
-mirroring the existing `CRS_OAUTH_REAUTH_COOLDOWN_MS=${...:-1800000}` pattern) and to `.env`.
-This makes future tuning a `docker compose up -d` (env reload), not a code rebuild.
-
-### 2.3 Account-pool load-balancing verification
-- Confirm `_sortPrimaryClaudeAccountsBeforeFallback` is wired and the `📊 Load-balance pick`
- log line fires on new (non-sticky) session assignment.
-- Confirm session stickiness is resolved **before** the util sort (sticky sessions must not
- be re-balanced mid-conversation).
-- Confirm `dynamic-priority.py` cron is running and not erroring. **Known pre-existing bug:**
- `NoneType .get` traceback on accounts with no usage object — non-blocking but should be
- guarded so the cron doesn't half-update priorities.
-- Verify the dead account: one `pool-<label>` entry was the out_of_credits trigger;
- it needs a **credit top-up** to truly rejoin (5h/7d windows are not the constraint).
- Until topped up, the credit-cooldown (§2.1) keeps it parked instead of hammered.
-
-### 2.4 Concurrency caps + overload protection (today's storm)
-Today a **ConnectionRefused storm** hit under ~17 concurrent agents; recovered via keepalive.
-The relay restarted (container up only minutes at audit time). Hardening:
-
-1. **Right-size the util→concurrency tiers.** Current tiers (`8/5/3/2/1`) are generous at low
- util. With ~10 accounts × 8 = 80 theoretical concurrent low-util streams, the bottleneck is
- **not** per-account concurrency — it is the node process / socket layer dropping connections
- under burst. Validate the tiers hold the fleet's real peak (≈17–25 concurrent) without the
- process refusing connections.
-2. **Bound the proxy socket pool.** `PROXY_MAX_SOCKETS` / `PROXY_MAX_FREE_SOCKETS` are present
- in compose but **unset** (default unbounded). Set explicit ceilings (e.g.
- `PROXY_MAX_SOCKETS=256`, `PROXY_MAX_FREE_SOCKETS=32`) so a burst can't exhaust file
- descriptors → `ECONNREFUSED`.
-3. **Confirm the 503 backpressure path returns cleanly** rather than dropping the socket.
- When all accounts are at concurrency limit the scheduler throws
- `CONSOLE_ACCOUNT_CONCURRENCY_FULL`; verify the relay turns that into a **503 with
- `Retry-After`**, not a connection reset — clients should back off, not see `ConnectionRefused`.
-4. **Container resilience.** Ensure `restart: unless-stopped` on the relay service and a
- `healthcheck` so Docker auto-recovers a wedged process. Confirm the node process file-descriptor
- `ulimit` (`docker inspect` → `Ulimits`/host `LimitNOFILE`) is high enough (≥ 65536) for the
- socket count above.
-5. **Fleet-side ceiling.** The durable fix for "17 concurrent heavy streams" is **also** to cap
- concurrent fleet dispatch (the spawner / keepalive), so the relay is never asked to hold more
- live streams than the pool's summed concurrency budget. Document the agreed cap with Sam
- before raising relay limits.
-
-### 2.5 Session-routing health
-- **Auth var:** every routed session must use `ANTHROPIC_AUTH_TOKEN=cr_…` (Bearer), never
- `ANTHROPIC_API_KEY` (format-validated client-side → "Invalid API key format / Please run /login").
-- **Key-validity probe (authoritative — the relay decides):** malformed POST to
- `…:3005/api/v1/messages` with the `cr_` key → **HTTP 400 = key accepted**, **401 = stale/rejected**.
-- **Canonical key:** macOS keychain `crs-fleet-key` (== Mac overlay token). Doppler
- `claude-ops/prd CRS_FLEET_KEY` has drifted stale before — if the box 401s, re-sync from keychain
- and re-push Doppler, else the next Doppler→box push re-breaks it.
-- **Preflight gate:** `~/.claude/scripts/crs-overlay-preflight.sh` must accept
- `ANTHROPIC_AUTH_TOKEN` (it previously only checked `CLAUDE_CODE_OAUTH_TOKEN` and blocked ALL
- dispatch). Confirm it still passes (HTTP 400 = exit 0).
-- **`✅ Cleared 401 error count` in relay logs is a SUCCESS** (counter cleared after a 200), not an
- error. Do not blindly grep-count `401`. Genuine failures are `🔒 Invalid API key attempt`.
-
----
-
-## 3. Execution sequence (do in this order)
-
-> All commands run **on the FRA box** (`ssh fra`). `$RP` = `$(docker exec crs-claude-relay-1 printenv REDIS_PASSWORD)`.
-
-1. **Snapshot for rollback** (see §5) — back up `docker-compose.yml`, tag the running image, note current account state.
-2. **Re-apply credit-cooldown** (§2.1) on-disk in `~/crs/src/...`. Do NOT bulk find-replace — edit the 3 mark sites + the service method surgically; grep each site first.
-3. **Promote cooldown/util knobs to env** (§2.2) in `docker-compose.yml` + `.env`.
-4. **Set socket-pool ceilings + restart policy + ulimit** (§2.4).
-5. **Rebuild + restart relay only:**
- ```bash
- cd ~/crs && docker compose build claude-relay && \
- docker compose up -d claude-relay
- ```
- (Redis is NOT touched — preserve queues/account state.)
-6. **Run §4 verification.** If any check fails, **roll back via §5** (must complete < 2 min).
-7. Tag the validated image (e.g. `tuned-20260619`) and pin it in compose so a future restore is deterministic.
-
----
-
-## 4. Verification (post-deploy)
-
-Run all of these; the deploy is good only if every one passes.
-
-1. **Health:** `curl -s -o /dev/null -w '%{http_code}' 127.0.0.1:3005/health` → `200`.
-2. **Key accepted:** malformed POST to `…:3005/api/v1/messages` with the `cr_` key → `400` (not 401).
-3. **Real relay:** a `claude -p "reply: OK" --model haiku` through the overlay → returns a completion (proves end-to-end forward + auth).
-4. **Credit-cooldown active (the P0 check):** in relay logs, confirm an `out_of_credits` 429
- produces a long cooldown, NOT a 90 s one — grep for the credit-exhaustion mark + the
- `CLAUDE_OAUTH_CREDIT_EXHAUSTED` webhook. Then confirm the exhausted account is **not
- re-selected** within the cooldown window:
- ```bash
- docker logs crs-claude-relay-1 --since 10m 2>&1 | grep -iE "credit|out_of_credits|cooldown|Load-balance pick"
- ```
- No account should appear in a `Load-balance pick` while it is credit-cooled.
-5. **No repeat-hammer:** over a 5-min window, no single `claude:account:<uuid>` should accumulate
- more than a handful of 429s. (Pre-fix baseline was ~40/min on the dead account.)
-6. **Load-balancing live:** under real fleet load, `📊 Load-balance pick` shows picks spread
- across multiple low-util accounts, not one account pinned to 100% while others sit at 0%.
-7. **Concurrency gating:** force burst load (or replay the 17-agent peak) and confirm the relay
- returns `503` + `Retry-After` when saturated — **no `ECONNREFUSED` / ConnectionRefused** in
- client logs.
-8. **Container auto-recovery:** `docker inspect crs-claude-relay-1 --format '{{.HostConfig.RestartPolicy.Name}}'`
- → `unless-stopped`; healthcheck status `healthy`.
-9. **Cron clean:** `dynamic-priority.py` last run logged no `NoneType` traceback.
-
----
-
-## 5. Rollback procedure (must complete in < 2 minutes — test it first)
-
-> Redis is never cleared — account state and queues survive a relay rollback.
-
-```bash
-cd ~/crs
-
-# 1. Restore the pre-tuning compose (taken in §3 step 1)
-cp docker-compose.yml.bak-pretune-<TS> docker-compose.yml
-
-# 2. Re-pin the last-known-good image (the running image at audit = cve-fix-20260615;
-# OR the validated prior tag if one exists)
-# Edit compose `image:` → weishaw/claude-relay-service:cve-fix-20260615
-# (compose has `build: .`, so to force the prebuilt image either comment out `build:`
-# for the rollback, or `docker compose up -d --no-build claude-relay`)
-
-# 3. Restart relay ONLY (Redis untouched)
-docker compose up -d --no-build claude-relay
-
-# 4. Verify (< 30 s)
-curl -s -o /dev/null -w 'health=%{http_code}\n' 127.0.0.1:3005/health # expect 200
-```
-
-**Manual park/unpark a misbehaving account** (faster than a full rollback if only one account is bad):
-```bash
-RP=$(docker exec crs-claude-relay-1 printenv REDIS_PASSWORD)
-docker exec crs-redis-1 redis-cli -a "$RP" --no-auth-warning \
- hset claude:account:<uuid> schedulable false # true to re-enable
-```
-> Note: `schedulable=false` only stops **new** session assignment — existing session-sticky
-> requests drain over their window.
-
-**Pre-flight the rollback before the real deploy:** do steps 1–4 against the current image once,
-time it, confirm < 2 min and health 200, then proceed with the tuning deploy.
-
----
-
-## 6. Constraints & safety (carried from the agent brief)
-
-- **No destructive bulk edits** on scheduler code — grep/inspect each site before editing.
-- **Preserve Redis data** — never `FLUSHDB`/clear queues mid-fix.
-- **Rollback < 2 min, tested first.**
-- **Fleet impact:** changes affect every agent using the relay — coordinate the concurrency-cap
- and fleet-dispatch-ceiling numbers with Sam before deploying.
-- **Never print** the `cr_` key, OAuth tokens, AWS secrets, or `REDIS_PASSWORD`.
-
----
-
-## 7. Open items to confirm with Sam before deploy
-
-1. Agreed **fleet concurrent-dispatch ceiling** (relay-side cap is moot if the fleet over-dispatches).
-2. Whether the `loadbalance-fix-20260616` image/diff is recoverable on the box, or the
- credit-cooldown must be re-implemented from §1 spec.
-3. Top-up status of the billing-dead `pool-<label>` account — keep parked vs. top up vs. remove from pool.