Skip to content

armgit / xerj

Public Imported from GitHub
0

AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.

feat(bench): neural-bulk A/B harness and frozen Okta corpus for ingest changes (#2448)

Christian Todiecb990c63de7e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 04026c4ca..883132225 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -49,6 +49,13 @@ continues in #2427 and #1122, which stay open.
### Added
+- `scripts/bench/neural-bulk-ab.sh CONTROL_SHA CANDIDATE_SHA [N]` is an
+ interleaved A/B harness for neural-embedding `_bulk` ingest. It builds each
+ SHA in a worktree, runs the real `xerj autoindex` client against a frozen
+ Okta corpus on a CPU-capped throwaway node, and reports median, p95 and the
+ control arm's spread per metric. It measures the harness only; no
+ performance claim is made. See `benchmarks/ingest-neural-bulk/README.md`.
+
- `xerj sessions import --format claude-code --input FILE --output NEW.jsonl`
prepares an explicit Claude Code JSONL transcript for ordinary `autoindex`.
It emits text-bearing user/assistant records through the existing native
diff --git a/benchmarks/ingest-neural-bulk/README.md b/benchmarks/ingest-neural-bulk/README.md
new file mode 100644
index 000000000..de92b5e14
--- /dev/null
+++ b/benchmarks/ingest-neural-bulk/README.md
@@ -0,0 +1,84 @@
+# ingest-neural-bulk
+
+A/B harness for neural-embedding `_bulk` ingest. It answers one question: does
+a change to the server make the same `xerj autoindex` run faster, slower, or
+noisier, with neural embeddings on and the CPU held fixed?
+
+## Run
+
+```
+scripts/bench/neural-bulk-ab.sh [--dry-run] CONTROL_SHA CANDIDATE_SHA [N=5]
+```
+
+`--dry-run` prints every command and writes nothing. Without it the script:
+
+1. Verifies `corpus/okta-bulk.tar.zst` against `corpus/SHA256SUMS`, extracts
+ it, and verifies each extracted file against `corpus/MANIFEST.tsv`.
+2. Builds each SHA once, in its own `git worktree`, with
+ `cargo build --release -j 2 -p xerj-server`. The same SHA twice builds once.
+ `CONTROL_BIN` / `CANDIDATE_BIN` skip a build.
+3. Runs control, candidate, control, candidate, ... N times each. Every run
+ gets a fresh data dir and a fresh client state dir.
+4. Starts the server on the first free port from 9560 (never 9200) under
+ `systemd-run --user --scope -p CPUQuota=400% -p MemoryMax=16G`, falling
+ back to `taskset -c 0-3` with a warning. Env: `XERJ_NUM_CPUS=4`,
+ `XERJ_PROF=1`, `XERJ_EMBED_MODE=neural`, `HF_HOME` (default
+ `~/.cache/huggingface`; warm it first or run 1 includes the model download).
+5. Drives it with
+ `xerj autoindex <corpus> --prefix bench --bulk-mb 8 --workers 4 --progress json --approve proceed`.
+6. Stops the server on exit, including on Ctrl-C, and removes the arm worktrees.
+
+Output goes to `benchmarks/ingest-neural-bulk/results/<date>/`: `meta.json`,
+`report.md`, and per run `run-<arm>-<i>/{server.log,client.log,summary.json}`.
+Set `RESULTS_DATE` to avoid clobbering an earlier run on the same day.
+
+## Control-vs-control precondition
+
+Before any candidate number is published, run the harness with the same SHA
+for both arms. The two arms' median `bulk total_ms p95` must differ by less
+than 10%. `report.md` prints PASS or FAIL for this. If it fails, the host is
+too noisy for the effect being measured.
+
+## Corpus
+
+`corpus/okta-bulk.tar.zst` is 203 files, 11,257,358 bytes raw, 461 KB
+compressed, stored directly in git (the repo has no `.gitattributes` and no
+binary-size policy). It holds the three Okta Management OpenAPI YAML copies
+(golang, nodejs, python SDKs) and the first 200 `okta-sdk-golang/okta/docs`
+files by sorted name. `MANIFEST.tsv` gives upstream repo, commit, source path,
+in-tarball path, bytes and sha256 for each file. The golang spec lives under
+`.generator/` upstream; `autoindex` skips hidden directories, so it is stored
+as `generator/` and `source_path` records the original.
+
+## Columns in `summary.json` and `report.md`
+
+| field | source |
+|---|---|
+| `wall_s` | `wall_s` in the client's `xerj-done` JSON object (plain `wall=` as fallback) |
+| `wall_wrapper_s` | shell-measured client wall time, for cross-checking |
+| `records` | `records` in the same `xerj-done` object |
+| `records_per_s` | `records / wall_s` |
+| `bulk_total_ms_p50`, `_p95` | nearest-rank percentiles of `total_ms=` on server `XERJ_PROF bulk` lines (`engine/crates/xerj-engine/src/bulk.rs`) |
+| `embed_ms_p50`, `_p95` | `embed_ms=` anywhere on a server `XERJ_PROF` line. Null on builds that do not emit it. |
+| `retries_window_failed` | server log count of `semantic embedding window failed; retrying per item` |
+| `retries_embed` | server log count of `embed retry` |
+| `client_bulk_retries` | `bulk_retries` in the `xerj-done` object (items re-sent after a 429); absent means 0 |
+| `retries` | sum of the three above |
+| `http_429` | log lines in server and client logs containing the whole word `429`. The server does not log requests, so this is indicative; `client_bulk_retries` is the authoritative backpressure count. |
+| `peak_rss_kb` | `VmHWM` from `/proc/<server pid>/status`, read after the client exits and before the server stops |
+| `cpu_s` | `utime + stime` from `/proc/<server pid>/stat`, divided by `CLK_TCK` |
+| `ok` | client exit 0 and `xerj-done` reported `ok=true`. Failed runs are listed in the report and excluded from statistics. |
+
+`report.md` shows, per metric, each arm's median and p95 across ok runs (with
+N below 20 the nearest-rank p95 is the max), the control arm's max-min spread,
+and the candidate-minus-control median delta. A delta is marked `EXCEEDS
+SPREAD` only when its magnitude is larger than the control spread. A spread of
+zero (for example N=1) flags any difference, so read it with N of at least 5.
+
+## Not covered
+
+- The client is not confined to the CPU quota; only the server is.
+- `passages/s` is not computed: the server does not report a passage count
+ that the harness can read.
+- Throughput is end-to-end with a cold index, per `docs/CONTRIBUTION_REVIEW.md`
+ matched-build guidance. Do not compare it with warm or partial-corpus numbers.
diff --git a/benchmarks/ingest-neural-bulk/corpus/MANIFEST.tsv b/benchmarks/ingest-neural-bulk/corpus/MANIFEST.tsv
new file mode 100644
index 000000000..d1682c134
--- /dev/null
+++ b/benchmarks/ingest-neural-bulk/corpus/MANIFEST.tsv
@@ -0,0 +1,204 @@
+upstream_repo commit source_path path bytes sha256
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db .generator/okta-management-APIs-oasv3-noEnums-inheritance.yaml okta-bulk/okta-sdk-golang/generator/okta-management-APIs-oasv3-noEnums-inheritance.yaml 3556393 0423a06b46f37ec8eda61fd0fd6934d2cba321946b68f0e5850f70a600306048
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AAGUIDAuthenticatorCharacteristics.md okta-bulk/okta-sdk-golang/okta/docs/AAGUIDAuthenticatorCharacteristics.md 4036 c6bacf320cc19c7bc41b17dd2fc69bcf9c38d0a8d83c0e55120253a4883e7d2f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AAGUIDGroupObject.md okta-bulk/okta-sdk-golang/okta/docs/AAGUIDGroupObject.md 2519 7eb23678f829d777a6769b2ab94e8f87688153d7d8374c1172815a8f05ec9733
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APIServiceIntegrationInstance.md okta-bulk/okta-sdk-golang/okta/docs/APIServiceIntegrationInstance.md 8571 530c6e9f3bfa70a674b206b511748e6abbc523128dbd67e5f0a6a34e72c18e0f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APIServiceIntegrationInstanceSecret.md okta-bulk/okta-sdk-golang/okta/docs/APIServiceIntegrationInstanceSecret.md 6041 502f4a11a66c0abce7fc7d55495b46da03c22f1adc6894acdda1f349f8b1e258
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APIServiceIntegrationLinks.md okta-bulk/okta-sdk-golang/okta/docs/APIServiceIntegrationLinks.md 3262 85becc1500678e45d3998ae9b717f4d71da1899520c89ee0829398cecb52dc36
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APIServiceIntegrationSecretLinks.md okta-bulk/okta-sdk-golang/okta/docs/APIServiceIntegrationSecretLinks.md 3610 62d7620f2ce6e4f57b2e4afc1809ff6b2627eedffa8b9661fe51d7a55ecb9a32
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APNSConfiguration.md okta-bulk/okta-sdk-golang/okta/docs/APNSConfiguration.md 3882 7633252a54edbeeecf2a84c658743b5f49d919ba1c9b39c808b4824f5950c167
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/APNSPushProvider.md okta-bulk/okta-sdk-golang/okta/docs/APNSPushProvider.md 1791 9dace36c18f619b36ffc62c68cf8f5faa004ad43586db05e28a11f909b0f7c0f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicy.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicy.md 2775 f5773ca9a7dbe0e9bdf2d61cffa41a5d14bb1a4eaa6f003544227dcc0acc6ea1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyAllOfEmbedded.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyAllOfEmbedded.md 2119 616f50f1a8dc41f18b5630a36b344901e2f349ff7928053271acce5d76214111
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyConstraint.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyConstraint.md 7141 8e963f24c5bfcec2245fdc2f2ad487b85fec2191c7219f0b5273e882d5cd79a7
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyConstraints.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyConstraints.md 2645 311415bd0f63e1b8493306f699370234052382ad0b675141f5ee373af3a096dc
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyLink.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyLink.md 4181 fe1c923f520f4cd7e710c43d6661104db1fbcaf6a9b9b6e711a5871806d9d7a3
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyRule.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyRule.md 2530 0528c4dfb52d73d28774d9d1a1ec9d92e2e48f8cfe1d4f51c7bab17bdc4234df
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyRuleActions.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyRuleActions.md 1912 1e6861c1ab25672af01c3613b46cdabb59ec0888bdd6249722da65bd37418ba6
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyRuleApplicationSignOn.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyRuleApplicationSignOn.md 3704 e9a9eccaabc865ec4dbd4b9b9fc44a66bab809f018c6d91ec739195b3f665ad5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyRuleConditions.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyRuleConditions.md 7658 ffd657d4b9a76c6c9a074a0e11d8aabf77da820a63125a34c45c59291fb1e5a5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccessPolicyRuleCustomCondition.md okta-bulk/okta-sdk-golang/okta/docs/AccessPolicyRuleCustomCondition.md 2214 ce7053ae6d28bedcc0d006c01d1586062ded352f8e7688c14424e1d81f00f091
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccountLinkedEnrollment.md okta-bulk/okta-sdk-golang/okta/docs/AccountLinkedEnrollment.md 6414 41ba6321f4449bb06434814b747619e0235959309dd126cd767e18c9fa6d1a39
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccountLinkedEnrollmentEmbedded.md okta-bulk/okta-sdk-golang/okta/docs/AccountLinkedEnrollmentEmbedded.md 2103 f3e14910518d7c32e58bcbcf127b13f6ba65660f4b5ee47a012e4688a5e2d4f9
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AccountLinkedEnrollmentProfile.md okta-bulk/okta-sdk-golang/okta/docs/AccountLinkedEnrollmentProfile.md 2090 354544829098d08d878b1669e3c09b31f60b66ef9583bf167ed586557adaed6d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AcsEndpoint.md okta-bulk/okta-sdk-golang/okta/docs/AcsEndpoint.md 1909 6d451ea531b6ab0c0c4892f771b13d9b39e344126548645a9f1ab338a5c3e38a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/Actions.md okta-bulk/okta-sdk-golang/okta/docs/Actions.md 1720 45b586f31d3deacbfcf2a255e6729c3ff2c13857eb4737f306d592bbe55383b9
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ActivateFactor200Response.md okta-bulk/okta-sdk-golang/okta/docs/ActivateFactor200Response.md 8894 7254bdb64130eb35a7990f7093929f371541840dc21884973fdde86c598caa51
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AddGroupRequest.md okta-bulk/okta-sdk-golang/okta/docs/AddGroupRequest.md 1695 2527be18faaf5c7fd7a0f8128622771295a9bc1ba4766b6dbae49c83f64fc33d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AddJwkRequest.md okta-bulk/okta-sdk-golang/okta/docs/AddJwkRequest.md 5501 43d1325fa5b7988fce0dc86e57cac8baf5d2d7f72005add54c56f85595c6fdc1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AdminConsoleSettings.md okta-bulk/okta-sdk-golang/okta/docs/AdminConsoleSettings.md 3138 e1ea6ca01ca9a9c3aa31db87d2c4b1eb5090fec2938afdc2701517d7c73d247c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/Agent.md okta-bulk/okta-sdk-golang/okta/docs/Agent.md 8978 f7c63b0c1031ec1c670907b8523a839da27b6b26195874162ef423eacffdfc85
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentAction.md okta-bulk/okta-sdk-golang/okta/docs/AgentAction.md 1997 7ef79c74007504f3977f784cd75a70f94cfa8d35333cddee70be2c1f47277b44
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentPool.md okta-bulk/okta-sdk-golang/okta/docs/AgentPool.md 6272 c0da4167c9dd9758a853e28fd85855ac3dd0db9478fef9ac873ed668282a2a53
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentPoolUpdate.md okta-bulk/okta-sdk-golang/okta/docs/AgentPoolUpdate.md 9128 a72178c1a12f61abfcd8fb359877c05d65d3cc309b479dbc3d4a8ff5b97fa181
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentPoolUpdateSettingRequest.md okta-bulk/okta-sdk-golang/okta/docs/AgentPoolUpdateSettingRequest.md 6717 e3307be6abcb800c8b84679833128c6299164ecbbd96591bfc1ede69f279c907
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentPoolUpdateSettingResponse.md okta-bulk/okta-sdk-golang/okta/docs/AgentPoolUpdateSettingResponse.md 6897 48e00c92aef166d1766c6fb91c71d485a59c4a0263301ec752ecfb1bf146c810
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AgentPoolsAPI.md okta-bulk/okta-sdk-golang/okta/docs/AgentPoolsAPI.md 31542 f02a196154ddc46add8e3ad82f73435f7ecfc8a1c236df595ddd3beb5574a01c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AndroidDeviceTrust.md okta-bulk/okta-sdk-golang/okta/docs/AndroidDeviceTrust.md 7716 da37a14a379b0c671890b6c50b729446225335f9fe2edec0a6517f9faf3612a6
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApiServiceIntegrationsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApiServiceIntegrationsAPI.md 22782 3e3b0f984762cbb957059e130eaa9702d111d018d684d3972337f787c4e890b5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApiToken.md okta-bulk/okta-sdk-golang/okta/docs/ApiToken.md 7200 07aa79c01678a080e8a4592fee2bc9529bcfac9fe7091d2fa780782c2ecbd5c1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApiTokenAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApiTokenAPI.md 8494 66659169d309253ca9e98f62aea736effac97e8202c1ad59265df0d848799865
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApiTokenNetwork.md okta-bulk/okta-sdk-golang/okta/docs/ApiTokenNetwork.md 3066 ad11339c1b5b04f5d746d5a65747830ca7f06b00376e116aca0584c547be0da4
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApiTokenUpdate.md okta-bulk/okta-sdk-golang/okta/docs/ApiTokenUpdate.md 4382 2fbf1f7e85053e710f2b696bcd704a037c71ea09f5029c3f58f7f48500bb1585
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppAccountContainerDetails.md okta-bulk/okta-sdk-golang/okta/docs/AppAccountContainerDetails.md 8534 8e47f82c6e608d8764e21e06ec54e2d6d224f7929b3e113fc9adb895bdd8c6e7
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppAccountContainerLink.md okta-bulk/okta-sdk-golang/okta/docs/AppAccountContainerLink.md 2465 465c9e8eee28eaf32b3c3679a79e26fddbcc3754eabe083a0b06d7de317b8541
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppAndInstanceConditionEvaluatorAppOrInstance.md okta-bulk/okta-sdk-golang/okta/docs/AppAndInstanceConditionEvaluatorAppOrInstance.md 3450 49f474af04b653f87a8c3583d1ea196b52e68321e23e5040d283ad4302a62f14
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppAndInstancePolicyRuleCondition.md okta-bulk/okta-sdk-golang/okta/docs/AppAndInstancePolicyRuleCondition.md 3115 92ceac063b9e1231e34822efc017426efab1e6955f557160336798c58780be25
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppConfig.md okta-bulk/okta-sdk-golang/okta/docs/AppConfig.md 1397 959adcbc5a7c95facce7e924cb800c9b06a02ec94ef8b7fc45ad155fbadacb97
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppConfigActiveDirectory.md okta-bulk/okta-sdk-golang/okta/docs/AppConfigActiveDirectory.md 3746 4ed0efa60fc3bdc8452bf7b97ddbe07b07a5e7d55b2d01d4b24bc811013d147f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppConfigOffice365.md okta-bulk/okta-sdk-golang/okta/docs/AppConfigOffice365.md 1723 f5195133a68e1098c417f7360987fa0c160d6d69af98e72a2a43347a947f800d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppConnectionUserProvisionJWKList.md okta-bulk/okta-sdk-golang/okta/docs/AppConnectionUserProvisionJWKList.md 1746 aebe7268ed0f1b299660ae50b0bacb47c7479e12f4cac793ffe9a274fe387dcf
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppConnectionUserProvisionJWKResponse.md okta-bulk/okta-sdk-golang/okta/docs/AppConnectionUserProvisionJWKResponse.md 1930 7fee8df5723c2170f1ad2ed3799969e7ffe969b346c18868bfb0158a682377a0
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppCustomHrefObject.md okta-bulk/okta-sdk-golang/okta/docs/AppCustomHrefObject.md 3565 f81061ff0080e230430fe2357003b2a4dd0e870bd10fe3ef274d2fdb59b31d54
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppCustomHrefObjectHints.md okta-bulk/okta-sdk-golang/okta/docs/AppCustomHrefObjectHints.md 1723 25567a8151b634129a223dde077f08a369f640aa48b6438fe04bb244cf6d8676
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppGroup.md okta-bulk/okta-sdk-golang/okta/docs/AppGroup.md 2032 1e3c86652fe3c1a82d2a82ff46e39755c5fabe0cf409c517720c83b5ab08d52b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppInstancePolicyRuleCondition.md okta-bulk/okta-sdk-golang/okta/docs/AppInstancePolicyRuleCondition.md 2553 784ee34b3a239d65f6f019e485806393807d31d454b001597494e9ca62292793
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppInstanceSearchType.md okta-bulk/okta-sdk-golang/okta/docs/AppInstanceSearchType.md 1597 f27b34aeafa1c274b4e23a6e309a0f62eada972493d0ef9995c45cbba88e3947
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppPropertiesValue.md okta-bulk/okta-sdk-golang/okta/docs/AppPropertiesValue.md 2270 21e34106df54edb50646cb940823a62b3770f997fac9cb0c8b108b6f61ac40c9
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppResourceHrefObject.md okta-bulk/okta-sdk-golang/okta/docs/AppResourceHrefObject.md 2303 bc0148c512269a1217d03ab085c135d3bfe42322b00f497cc66250a3ebb35f41
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppServiceAccount.md okta-bulk/okta-sdk-golang/okta/docs/AppServiceAccount.md 11416 efd571e92e708df9286863a1376569449adb86f0ffa609e2fdacc71dfd833b00
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppServiceAccountCredentials.md okta-bulk/okta-sdk-golang/okta/docs/AppServiceAccountCredentials.md 2489 4fb611baa904fdc9a2e67ba9676172fc06ec33c6cd69cd3e3ec8cbe8e173341a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppServiceAccountForUpdate.md okta-bulk/okta-sdk-golang/okta/docs/AppServiceAccountForUpdate.md 4255 18512b93e9f6ba9807dfeba4e93d849a80e80fb3c9cb1585311e8681ef35aff7
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUser.md okta-bulk/okta-sdk-golang/okta/docs/AppUser.md 12451 04d8a1accc380c765704f2f1c381f4001897fc1a6ce25b5d95bf340cfae1032c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserAssignRequest.md okta-bulk/okta-sdk-golang/okta/docs/AppUserAssignRequest.md 13348 e152eb7fbd2bd162aa3b38a3b8e0a9628bece445f7c5f9a6d9d5579c43c87226
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserCredentials.md okta-bulk/okta-sdk-golang/okta/docs/AppUserCredentials.md 3235 395dad452a1fbb1c3947afe08a7807add8bff8fcd1e4a2d7371be6ee42a75057
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserCredentialsRequestPayload.md okta-bulk/okta-sdk-golang/okta/docs/AppUserCredentialsRequestPayload.md 2004 fb9b2473ab99122c12d49d309b964498c4008b4ff36d0afe7e309e3a141bb4d3
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserPasswordCredential.md okta-bulk/okta-sdk-golang/okta/docs/AppUserPasswordCredential.md 1744 2d48ecae3657aef99e22c24ba89b2bb3ed34994b0f86fa34afdca360c70e3a20
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserProfileRequestPayload.md okta-bulk/okta-sdk-golang/okta/docs/AppUserProfileRequestPayload.md 2283 bf9d80366bf2309dbd994a5942a3e1eae43c164e2c0348b3cb11c8c603d583d5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppUserUpdateRequest.md okta-bulk/okta-sdk-golang/okta/docs/AppUserUpdateRequest.md 2960 8969b7bc60fd3f11e3508544fbd63bca0ca7649058707e1faa412c61e6e00ffb
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AppleClientSigning.md okta-bulk/okta-sdk-golang/okta/docs/AppleClientSigning.md 3163 84ccc831f1234f1af6e1d7c2b13bdf4cf29572856a3b79ce8058e3fd977ba194
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/Application.md okta-bulk/okta-sdk-golang/okta/docs/Application.md 13774 44cff976f73fa7cf3ad691a18004676605830c5658bb45cbdf0244f56b68ad68
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationAPI.md 18086 53f7bbc36634110eee771fae8025aea15a164f7a8d78403ad2a6ca58daaf9570
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationAccessibility.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationAccessibility.md 4116 62d180b669691c4aa3aafb692ca854a2e1a468ec4698a1c131954fd8d807a321
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationConnectionsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationConnectionsAPI.md 14361 916e6c5e2df0f9165a70c1d7b7b1f79afcc8a5025611a1314e31626604971e75
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationCredentials.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationCredentials.md 2830 cb2cd0e34fda866e6d150817729e3bfbb6a6d3f210aea33609d06c047ba044b7
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationCredentialsOAuthClient.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationCredentialsOAuthClient.md 6700 020ba72186f793fbf0635ec126ad9d3581d9dd3f21e7326b664efe33806ebf39
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationCredentialsSigning.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationCredentialsSigning.md 5062 f5e3d82789e0dc842be18e865be340ab965f255e8eebd37b39bde1047fe2d388
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationCredentialsUsernameTemplate.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationCredentialsUsernameTemplate.md 5899 5ac056e90ff5eb284f21b578f05da72a4fcf1f618933d49ef846af47ad642f02
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationEmbedded.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationEmbedded.md 1843 944b30ba693db22e5499c7291ef66700b44dedbe9e264bd81e0d6381dd288d40
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationExpressConfiguration.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationExpressConfiguration.md 3063 c27aeed42eae5d8d16cd3cb6cedce2ee511a6214768c37fb3ff38489d88bfe0a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationFeature.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationFeature.md 4156 b22bb5d352e0151ab081b813a914d76919904f9092d9618c05e568b37b3dd764
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationFeaturesAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationFeaturesAPI.md 7353 976415ed53c8c7e8bc8fb2360fc91be967895aa2c637a925daf7f3fd89350225
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationGrantsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationGrantsAPI.md 9019 ce7081b09161c4bd904e26dfc96278cb8c430001fc11ae192a1d6d7aa07dc54b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationGroupAssignment.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationGroupAssignment.md 7170 06dc077d878a5c26ae6acc00ed0629f5e0e78e319ee7ea1a3e142d17de628e7a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationGroupAssignmentLinks.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationGroupAssignmentLinks.md 3335 77c3ea4e2021ef0962053ad79a1287ec0c59e9cbec512dc6248a3c004bf83e86
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationGroupsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationGroupsAPI.md 14470 b15ff0e0920f257aa59f7289ae269d61813c98c8365b27b59801fcd1255fea6e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationInterclientTrustMappingsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationInterclientTrustMappingsAPI.md 10999 1f60b12829555e219f357ff8c66c28cfd89f5a4e798b5736da719d8f46580b63
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLayout.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLayout.md 4962 2648895a1b884da9118d6fdbf4174fe35dc73bf604d6013c78774db024361554
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLayoutRule.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLayoutRule.md 2515 ea0d6ea31500327a9eab43a9d965b2d70fc68127da92125c48350f4143a9a6d5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLayoutRuleCondition.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLayoutRuleCondition.md 2553 7f458fb78713b1d39cd2edac4b6ffc89cf17dfac3cd99d6ed8071f240f576c4d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLayouts.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLayouts.md 1723 cd950c715b238847d10798bc27a1a71da9978af7051e05849644093b7f3c487f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLayoutsLinks.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLayoutsLinks.md 3263 f2c9ba04ca3620121eb579b17f729cca77755b3b1a2d929462d56e4cebf2cf35
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLicensing.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLicensing.md 1755 81158cf89e2b1da4469b27caff5a985654e5e76dea32343e0d7c97935dd9e4d4
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLinks.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLinks.md 8000 9f9f680550384c0a26ea30702ae1078c988dac4e603264e6c49a9c04d12966d2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationLogosAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationLogosAPI.md 3255 3d6f8f7e228088dd4d25cd9b85b71e4dd6dc1207fdbce7f56c3f4833d781936d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationPoliciesAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationPoliciesAPI.md 2093 1d9531362c534ef76e8c90bfd236089d921bb6b8b6a492e08ae64d2c3d2b9222
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSSOAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSSOAPI.md 2304 13eef0d39bac9c1c93104c5714aaaddc3eabf714e0fc9602b704fa4ad1f86768
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSSOCredentialKeyAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSSOCredentialKeyAPI.md 19374 88c862f27126078ab769041a638eef176d887a948f5d371d3af7ac1449dcef37
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSSOFederatedClaimsAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSSOFederatedClaimsAPI.md 11057 b41dcafb4270ddf8ed21b38b152294580b9d262f73a61ac665fad9b0bd927a97
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSSOPublicKeysAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSSOPublicKeysAPI.md 26312 765fba4236a09e0961f738cb40d9ac7ab0fc3661f2cd800ed2a9fd59ec18c812
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSettings.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSettings.md 6411 e1fa64a4a2ee4d57e02108699742ebd4c6c712581d85d142fcf643a12737f1d2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSettingsNotes.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSettingsNotes.md 2488 f06fca3e953d1d6f52269387a072a7351f58154f2545c896df0b6fe9486c4c4b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSettingsNotifications.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSettingsNotifications.md 1961 d703feb1e6bc16c4c5241fca7fcc034a9a982bc9c732ac1322589a0066182766
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSettingsNotificationsVpn.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSettingsNotificationsVpn.md 3582 be8cb516a659de0ab90efebf7ff0ee68c56b071d472ef27d4eb2b91e777b8a50
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationSettingsNotificationsVpnNetwork.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationSettingsNotificationsVpnNetwork.md 3800 4a20cdcdaad103551eacd07bd9e09d291502c961b644c6bbfcf13e258fffa669
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationTokensAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationTokensAPI.md 9459 ff66399cfc71db97cfdb6d63c147672be9c810ed53e29b1b2b913418a0d7fbf1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationUniversalLogout.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationUniversalLogout.md 4268 2b5aebb3d8bd32e3c7aa28ac1123a951208cf2d92f417868c347a69a8d0c6239
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationUsersAPI.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationUsersAPI.md 13991 bd4289614b9b6407fc6021954f0342a4106bfd6937515b4ab3898afa271f23ad
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationVisibility.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationVisibility.md 4206 c5a3247e1aa627fc4acef170a4185707b3186b0543451bc0ef72997ca8ae1462
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/ApplicationVisibilityHide.md okta-bulk/okta-sdk-golang/okta/docs/ApplicationVisibilityHide.md 2424 8103edae79c2451a7a8b7cf38542a918753609535a4c5d0ca26fc3d891460a34
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignGroupOwnerRequestBody.md okta-bulk/okta-sdk-golang/okta/docs/AssignGroupOwnerRequestBody.md 2417 081d0578533f3d33b7b0fa1daeebb0a9f8d2dcbd5e8dbf101010f45de1051e38
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleRequest.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleRequest.md 3018 7582381cdb957a7a0ad9fa96272fc0643b759bfd0dce6d3775d87695c4272307
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleToClientRequest.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleToClientRequest.md 2953 8e9c652b322ba1811db99a7370117a7d793e6bf178c2e14def83d6e132c0706d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleToGroup200Response.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleToGroup200Response.md 10489 d5bc6cc45e3a2986cadeee28d01a280af2f809f6bde43b4850d6abaaf2faf666
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleToGroupRequest.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleToGroupRequest.md 2933 d85afede5c8f6572b10e487e6c099755a1b7b0366ab3af96f078e859e69990db
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleToUser201Response.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleToUser201Response.md 10435 6342d66fcca6b88e7fe0d4502f39d06a7a3023b50f48e67161f2281db5117ef9
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignRoleToUserRequest.md okta-bulk/okta-sdk-golang/okta/docs/AssignRoleToUserRequest.md 2913 53b2af47e481c42b5eb434a34c8363d0584d30c127bdc9af1675684218d98134
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignUserToRealm.md okta-bulk/okta-sdk-golang/okta/docs/AssignUserToRealm.md 1657 9a6d082c5e18767b27a71d10b0274f65ce950ce759434f27fd620122be16dcf2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssignedAppLink.md okta-bulk/okta-sdk-golang/okta/docs/AssignedAppLink.md 7734 99a4618acd6233f2af20882f95fceb735ff698b6a5a81a4187bbbbfa8e09ba0a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssociatedDomainCustomizationsAPI.md okta-bulk/okta-sdk-golang/okta/docs/AssociatedDomainCustomizationsAPI.md 14938 7684436535b51b90c93fcbba7b0c7128ec1e2378892029088ad64f4ad7d4c351
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssociatedServerMediated.md okta-bulk/okta-sdk-golang/okta/docs/AssociatedServerMediated.md 1793 f2375df24aefc07f2bec5d8b31d06e3000d02f1237b8dc9a7ba7a71c4592233c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AssuranceMethod.md okta-bulk/okta-sdk-golang/okta/docs/AssuranceMethod.md 4663 eaa1fb7359d63fbeb393689fc542389930366b93ac3bffb3a5afba6908b8e5fc
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AttackProtectionAPI.md okta-bulk/okta-sdk-golang/okta/docs/AttackProtectionAPI.md 7581 daa06319e12b6976b4ed1a4b0d45a5441a59070032b4f497c7020b073de7556e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AttackProtectionAuthenticatorSettings.md okta-bulk/okta-sdk-golang/okta/docs/AttackProtectionAuthenticatorSettings.md 2566 e85d6a467fb03efb5321ec415785e2dbfb10ccc6644424824d2f8baa55bbb939
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AttestationRootCertificatesRequestInner.md okta-bulk/okta-sdk-golang/okta/docs/AttestationRootCertificatesRequestInner.md 1931 83a37cbc458f0aa7ac395537a18af0be1c293b3f4119975b9d42f7cbeba6214b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AttestationRootCertificatesResponseInner.md okta-bulk/okta-sdk-golang/okta/docs/AttestationRootCertificatesResponseInner.md 4135 e578b339c8edc009141c9e2a5018937aa12c477c223cd2e6b70cdcdae0d8bb7a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthServerLinks.md okta-bulk/okta-sdk-golang/okta/docs/AuthServerLinks.md 6038 7ab70bb87ed130539be89d71fb1b2c622c2df8f62befb38da6cfc8ea1d5868f3
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationMethod.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationMethod.md 7106 15e3c78a34102261c53cf59e3cf246aaa39f7e2aed73616c3a1b21738969bd97
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationMethodChain.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationMethodChain.md 4045 dbe5ba524341a55398bcfc6584465e52b95c0f0a77fa85938196acbdee07499a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationMethodChainMethod.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationMethodChainMethod.md 3183 ec4e9973bbb5946e7b67dda53c5fe42e697a5631424e0193c98267555934af24
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationMethodObject.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationMethodObject.md 3410 a853cf8fcf7fdba982c6aa57ff4a8dfc12fb7cc5e914dd425fa5b42c32e0c169
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationProvider.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationProvider.md 2385 6765890a295172165d998a93db325e6fc05a2d4fe5b90b85c25e366ea07681a2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticationProviderWritable.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticationProviderWritable.md 2515 4665a464afb4b319404c015533ac8d815d2fac13fc6238c901a25c0597f28b02
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorAPI.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorAPI.md 42518 7d824c84b5a53f730493bc1e495bd7510a20ece9474848a327de42e205f71edc
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorBase.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorBase.md 7201 29b21aaa79f38b5439d0cfd29cf589e7d2a6c33dc365781ee774d0bde02ed002
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollment.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollment.md 7418 d1809258947cc7c7be767650a5604473a8a1fa0395efe394cac5c5ef237f236e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentCreateRequest.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentCreateRequest.md 2628 254d5f131a65f20bf5f955883b3c0d2d352b594919fdc2d5a821a2eef7c1a051
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentCreateRequestTac.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentCreateRequestTac.md 2848 022c50c28d9c01d42fde8212df8e9684c401f44bdf36fba03a94d999542e73f4
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentLinks.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentLinks.md 3526 8b7e3a147fe12823292fa2bf7facb05da04e1f8d1fdeea4041b35c60451d1b62
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicy.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicy.md 2928 f43107664703381b41211c7c4f038afc079d065adf37b23030f5de90bcd887aa
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettings.md 5594 f1692fa7ec67d430cb96e08531da31215e9807414df06a0b77ec530657f54e6a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettingsConstraints.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettingsConstraints.md 2978 09e03c3aa68a1aae5d32f5a3aedcf4aac9d764b185bcc7acd4ea4f34acbda773
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettingsEnroll.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyAuthenticatorSettingsEnroll.md 6636 2297547b9334d6b26cd29543c9e2e995b381ae512f5e9e3e82121fac1efbebfe
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyConditions.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyConditions.md 2189 f8f87df456f9025e44ea936fb4b19583782bfff8d899cd1896851a1a69607c6c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyConditionsAllOfPeople.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyConditionsAllOfPeople.md 2384 b1e9b376fd88aecb903448d2e9b4c487a2486a741005b9a928a12a63ab67f057
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyConditionsAllOfPeopleGroups.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyConditionsAllOfPeopleGroups.md 2256 ca07fce7bbf3e35dc6f5d954525a542b0a9b393af4d02d706e58a212b260f832
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRule.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRule.md 3023 6ce0f573ab91332b6543cf3dc4d501d824bcfd8a600036beff40cf06e9a2f97b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRuleActionEnroll.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRuleActionEnroll.md 2014 018e50a52f45ec9aeacd408c594e6cbd096b14d518362f3a754cd69fcb93554e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRuleActions.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRuleActions.md 2179 35aa31003e4f554375bd6c5889b85c67128e4e7040622a9f419a8e096d985639
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRuleConditions.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRuleConditions.md 3925 43e84785e78057bc570369ed94e97f25fa2210d0936d428c62cf2bf50d832f5e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRuleConditionsPeople.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRuleConditionsPeople.md 2343 d512704d759556dea5b795deac57374c5b27598fdf4aa5b6d6edd9fb1a2205e1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicyRuleConditionsPeopleUsers.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicyRuleConditionsPeopleUsers.md 2225 ae69a9811ded73a7f485954438ac5ff1b034f31de3d5e3c71f83b974f87b5f68
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorEnrollmentPolicySettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorEnrollmentPolicySettings.md 3972 420c8c5e1927aa5a94bc6536f2717188dde21020399565fa0094bab28f8d1f7a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorIdentity.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorIdentity.md 1638 38d89f953eba5d757ff26e612c14b8005d29f0050e87b02d57b55793cb287166
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomApp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomApp.md 3873 e3e9bdfb93ce8aacdd5bc77035ea4fb68f7f58dbe25944c633f4557f74f4a76c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomAppAllOfProvider.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomAppAllOfProvider.md 2988 87f4d59be65000a5f3335aaa2ae4819615759878c22049881c8ef523033e2120
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfiguration.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfiguration.md 3335 729607147ae878eba4f81b01beaba1c85e9031d852bae16f61c44771d5e02c3c
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfigurationApns.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfigurationApns.md 4347 aea8f9cd18ca289c06c60765e4b61156c8b5ca799b38a3705b1f36d9a711f567
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfigurationFcm.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomAppAllOfProviderConfigurationFcm.md 2245 c3639b111385517e4e96753462a9b3f469b5f131cf4310de65d217369c279d68
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomAppAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomAppAllOfSettings.md 3304 2b7ae8ddbbd4ab2a3d598858a285b2a7e239d5f12759f1d6c1687d59658d2fca
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyCustomOtp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyCustomOtp.md 1082 a658f60d437c953b0f9fe67511bc3b34acf79e0b8fd4b16adc485c71209f4a92
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyDuo.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyDuo.md 1831 249dbb2fa26905c122bc398beb7e3f63a2b195cbb41e46a68841e2c0f9ac09aa
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyDuoAllOfProvider.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyDuoAllOfProvider.md 2844 07a68d7277c22f2a2c0f7b84bd1c73812ad62803dd861688b5df1ca9b7b7b3d1
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyDuoAllOfProviderConfiguration.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyDuoAllOfProviderConfiguration.md 4958 df4cef950e8e8f583ae28b32af540c7e1ff5f9f90b4e392af3780a09606cad55
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyDuoAllOfProviderConfigurationUserNameTemplate.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyDuoAllOfProviderConfigurationUserNameTemplate.md 2353 997e809f5b25d7bebb7b003457c49c899187a0a6e02d2f56aea764e0240d696f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyEmail.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyEmail.md 1871 6379d6fb6d4ca3210853106345ad195e8c155c7c36567ef307b61019c915b00f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyEmailAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyEmailAllOfSettings.md 3035 ff0459ee97fe2ededbcb5a84390966541d726ebbb81b02c700c3d16ed01a83b5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyExternalIdp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyExternalIdp.md 1104 36725528b5f1d6863f37a1fd9605d00492cfd7f77f12ef9fd37ac4a223dad27e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyGoogleOtp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyGoogleOtp.md 1082 750dfe77b4a96b72f48dfbc3cf7ec3ed5eb979861ad332191c969a1b5ec1b7c2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyOktaVerify.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyOktaVerify.md 1971 79e17fb786b93e5f635aad74814226ab895027060c8eda7d6231e046f5fe356b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyOktaVerifyAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyOktaVerifyAllOfSettings.md 7613 ed82d3f9898ba282d4fb403a1855244163fdcd8eda8fba7ee7e3bd7561a2c70b
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyOnprem.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyOnprem.md 1049 36fbbd72a7b84be8222f7e9e9e8a2d9e26cc69e305506f5840f20e54c3ae586a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyPassword.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyPassword.md 1071 3fb4991095939c2ef31cb3259e4293e8eedda27735bbc42c6588849b79dee9b7
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyPhone.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyPhone.md 1871 c66f059d9c9beaef64a85464a5feb787ee3cc062d06d6c903de496e439a16ba6
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyPhoneAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyPhoneAllOfSettings.md 1992 459affba41d264a855dfb0d88232060f4333411840946325ee63828eef38dbcd
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeySecurityKey.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeySecurityKey.md 1104 ae41c578d0405d279c2e1b7cdb3b2fc1303464eb4e23b6db68eec63ada114467
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeySecurityQuestion.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeySecurityQuestion.md 2036 a2007a9eb9d13e6203b7f21d48ef7018365107159c4ae9cd1325af81657e1ca5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeySmartCard.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeySmartCard.md 1082 18711ccf2d57c41c3bf9df4f842a93515d0a1e2906086342c14bf60e28fdc6cf
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeySymantecVip.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeySymantecVip.md 1104 b383639475b98d0c1f0a84dafdfdb3313c0009493722c5a24679cf0643079ec4
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyTac.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyTac.md 1831 334a6ca10e0a9080dbdd067700d276e962f9bd900628a3705e84c698380b0d6f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyTacAllOfProvider.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyTacAllOfProvider.md 2844 77c2a58097ddb400f0e94d7b43af9ee1f75084966dc1b0fbcdce940f6161e4e0
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyTacAllOfProviderConfiguration.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyTacAllOfProviderConfiguration.md 6503 deb8ec9b896bc1ea6dcff2087fa35a3a18fe530661174bc9dd3f9a8bd1315579
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyTacAllOfProviderConfigurationComplexity.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyTacAllOfProviderConfigurationComplexity.md 4100 44d15005b3b3020efc447be5900781aa2990894653d4dbd3a50f5db79b52f90e
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyWebauthn.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyWebauthn.md 1071 3dcddd1a26d91dd527ecb5a6d6fb0f910cb416b34282aed8e6100a675eee1495
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorKeyYubikey.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorKeyYubikey.md 1060 24aa780103bf61bf112135de6cc0723748bfdbcf30b3024dea08ac21cc983787
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorLinks.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorLinks.md 3960 ee0207a40059c68d581c241f280c249073cbdc789e0ecf1a0ed867c4b4142344
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodBase.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodBase.md 3138 9489a9c2d81371c9aaf7e2d558477bea640935d73e829cd8bc8f330d48a671d9
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodConstraint.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodConstraint.md 2822 71cdda0db725bf8d04254f2d340c3502fa77fad2b3594e94f8402d96a4eec15d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodOtp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodOtp.md 7143 d09bde618651aa29a469c9656e8f5c338a0af1b8d6d20385f9d436bb2cbae8c2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodPush.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodPush.md 1911 5ff27d9aae0cd08c302abe178f2bce70ca26fb6a471787916dcc0d8420aac664
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodPushAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodPushAllOfSettings.md 3730 6d2d4b2bf294a47eab422d6354819e01abf5288848d2b74f495d6324bd75e885
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodSignedNonce.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodSignedNonce.md 2051 2d04461da6fc31c32db236df277369957d46245b6061ff8a3aa649d0868df4c5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodSignedNonceAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodSignedNonceAllOfSettings.md 4225 0a7ba442063ef1b3dace04cde1899aa21c2f52fa14803665fed6da6e886a4d99
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodSimple.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodSimple.md 1082 fb7302a53b421796fb250ef68e8d0bb6d958fa17f7452d446c296d18a35eebe5
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodTac.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodTac.md 3115 12ca657bd676a72abd38234a32df8236254d684886bba95219d9e1dda929df6d
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodTotp.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodTotp.md 1911 50b3e670ef931c3d39188f622229769e0f459b75cd0c3ff9b05b6b49d94d534f
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodTotpAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodTotpAllOfSettings.md 4580 16f327821aa80b459e4868a2e43d4a3463cd439e3d4e57d4ccbac493de86aa08
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodWebAuthn.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodWebAuthn.md 1991 5584f447f3537a03243230de9b12f526ba3862b27cc925900e1f4ec7f0a66ad2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodWebAuthnAllOfSettings.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodWebAuthnAllOfSettings.md 13687 5bc20214ee88f60468185a3e2118b2d9eccea2f5389c6af1c0aa29bb3ad508d2
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorMethodWithVerifiableProperties.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorMethodWithVerifiableProperties.md 2248 008782835b7323c8bd4e76464e36cbdfd4336f44ba58a7730d97bfb3ea0e19ec
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorProfile.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorProfile.md 1699 250f585be56ce57298b926e403ab25a3c92b96641460dc6af05e953307cd9acd
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorProfileTacRequest.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorProfileTacRequest.md 2935 aa621c3bee839b039a4707bae58ef86241d5df172b6ad2857ede02f5a6e44b0a
+https://github.com/okta/okta-sdk-golang.git 9b9ccab705d864095ce554834e2959e93efb82db okta/docs/AuthenticatorProfileTacResponsePost.md okta-bulk/okta-sdk-golang/okta/docs/AuthenticatorProfileTacResponsePost.md 3812 8cf04dd4dca64cbe9981e99ad8213f6117e6e5d45d07237afa58560a9eec0f0c
+https://github.com/okta/okta-sdk-nodejs.git f4794e3e2d61da08c37c8c348bdb2fc698e90e4a oas-spec/management.yaml okta-bulk/okta-sdk-nodejs/oas-spec/management.yaml 3438586 1bba3fa7ca5dd8dcff930e6775aeb033e4d6f0349dda0583ef8eee58d3f8d637
+https://github.com/okta/okta-sdk-python.git 95f2e0fddc23f29a814e85b9e969e85386bf5b03 openapi/api.yaml okta-bulk/okta-sdk-python/openapi/api.yaml 3287809 1b87cadc20cc39756c9ac4a865a690a9b6d57c5c1a34b8b4543a12468610e152
diff --git a/benchmarks/ingest-neural-bulk/corpus/SHA256SUMS b/benchmarks/ingest-neural-bulk/corpus/SHA256SUMS
new file mode 100644
index 000000000..1d6108196
--- /dev/null
+++ b/benchmarks/ingest-neural-bulk/corpus/SHA256SUMS
@@ -0,0 +1 @@
+289afe6af7d2d3a86b98a97895357543d6a45dc360e64accb743bee59a313699 okta-bulk.tar.zst
diff --git a/benchmarks/ingest-neural-bulk/corpus/okta-bulk.tar.zst b/benchmarks/ingest-neural-bulk/corpus/okta-bulk.tar.zst
new file mode 100644
index 000000000..d50162fbe
Binary files /dev/null and b/benchmarks/ingest-neural-bulk/corpus/okta-bulk.tar.zst differ
diff --git a/scripts/bench/neural-bulk-ab.sh b/scripts/bench/neural-bulk-ab.sh
new file mode 100755
index 000000000..577d449ef
--- /dev/null
+++ b/scripts/bench/neural-bulk-ab.sh
@@ -0,0 +1,467 @@
+#!/usr/bin/env bash
+# ─────────────────────────────────────────────────────────────────────────────
+# neural-bulk-ab.sh — A/B harness for neural-embedding `_bulk` ingest.
+#
+# scripts/bench/neural-bulk-ab.sh [--dry-run] CONTROL_SHA CANDIDATE_SHA [N=5]
+#
+# Builds the server at each SHA in its own git worktree, then runs
+# control, candidate, control, candidate, ... (N each, interleaved) against a
+# cold throwaway data dir per run, driving the real client
+# (`xerj autoindex`) at a frozen corpus. Writes, per run,
+# benchmarks/ingest-neural-bulk/results/<date>/run-<arm>-<i>/
+# server.log client.log summary.json
+# and, per invocation, meta.json + report.md in the same <date> directory.
+# Column definitions and the control-vs-control precondition are in
+# benchmarks/ingest-neural-bulk/README.md.
+#
+# Pass the same SHA twice for the control-vs-control precondition run.
+#
+# Environment (all optional):
+# SCRATCH work dir (default: mktemp under /tmp)
+# RESULTS_DATE results subdirectory name (default: today, UTC)
+# CONTROL_BIN / CANDIDATE_BIN prebuilt `xerj` binaries; skips that build
+# HF_HOME model cache (default: $HOME/.cache/huggingface)
+# RUN_TIMEOUT seconds allowed per client run (default 3600)
+# PORT_MIN first port tried (default 9560; 9200 is never used)
+# KEEP=1 keep each run's data dir (repro.sh knob)
+# ─────────────────────────────────────────────────────────────────────────────
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib/repro.sh
+source "$HERE/lib/repro.sh"
+
+DRY=0
+POS=()
+for a in "$@"; do
+ case "$a" in
+ --dry-run) DRY=1 ;;
+ -h | --help) sed -n '2,29p' "${BASH_SOURCE[0]}"; exit 0 ;;
+ *) POS+=("$a") ;;
+ esac
+done
+if [ "${#POS[@]}" -lt 2 ] || [ "${#POS[@]}" -gt 3 ]; then
+ echo "usage: $0 [--dry-run] CONTROL_SHA CANDIDATE_SHA [N=5]" >&2
+ exit 2
+fi
+CONTROL_REF="${POS[0]}"
+CANDIDATE_REF="${POS[1]}"
+N="${POS[2]:-5}"
+case "$N" in '' | *[!0-9]* | 0) echo "N must be a positive integer, got '$N'" >&2; exit 2 ;; esac
+
+BENCH_DIR="$REPO_ROOT/benchmarks/ingest-neural-bulk"
+CORPUS_TAR="$BENCH_DIR/corpus/okta-bulk.tar.zst"
+DATE="${RESULTS_DATE:-$(date -u +%F)}"
+RESULTS="$BENCH_DIR/results/$DATE"
+SCRATCH="${SCRATCH:-$(mktemp -u /tmp/neural-bulk-ab.XXXXXX)}"
+HF_HOME="${HF_HOME:-$HOME/.cache/huggingface}"
+RUN_TIMEOUT="${RUN_TIMEOUT:-3600}"
+PORT_MIN="${PORT_MIN:-9560}"
+CLK_TCK="$(getconf CLK_TCK)"
+
+# run CMD... executes, or prints the command in dry-run mode.
+run() {
+ if [ "$DRY" = 1 ]; then
+ printf '+ '
+ printf '%q ' "$@"
+ printf '\n'
+ else
+ "$@"
+ fi
+}
+say() { printf '%s\n' "$*"; }
+
+CONTROL_SHA="$(git -C "$REPO_ROOT" rev-parse --verify "$CONTROL_REF^{commit}")"
+CANDIDATE_SHA="$(git -C "$REPO_ROOT" rev-parse --verify "$CANDIDATE_REF^{commit}")"
+
+# Isolation: user scope with a CPU quota and memory cap, else taskset.
+ISOLATION=""
+if systemd-run --user --scope -q -p CPUQuota=400% -- true >/dev/null 2>&1; then
+ ISOLATION="systemd-scope"
+elif command -v taskset >/dev/null 2>&1; then
+ ISOLATION="taskset"
+ echo "WARNING: systemd-run --user --scope unavailable; falling back to taskset -c 0-3" \
+ "(no memory cap, no CPU quota)." >&2
+else
+ echo "ERROR: neither systemd-run --user --scope nor taskset is available." >&2
+ exit 1
+fi
+if [ ! -d "$HF_HOME" ]; then
+ echo "WARNING: HF_HOME=$HF_HOME does not exist; run 1 would include the model download." >&2
+fi
+
+say "control $CONTROL_SHA"
+say "candidate $CANDIDATE_SHA"
+say "runs $N per arm, interleaved; isolation=$ISOLATION; results=$RESULTS"
+[ "$DRY" = 1 ] && say "(dry run: commands are printed, nothing is built, started or written)"
+
+# ── cleanup: stop the server, drop arm worktrees ────────────────────────────
+cleanup() {
+ set +e
+ repro_node_stop
+ local w
+ for w in "$SCRATCH"/wt-*; do
+ [ -e "$w/.git" ] && git -C "$REPO_ROOT" worktree remove --force "$w" >/dev/null 2>&1
+ done
+ git -C "$REPO_ROOT" worktree prune
+ return 0
+}
+trap cleanup EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
+
+# ── corpus ──────────────────────────────────────────────────────────────────
+CORPUS_DIR="$SCRATCH/corpus"
+run mkdir -p "$SCRATCH" "$CORPUS_DIR"
+if [ "$DRY" = 1 ]; then
+ say "+ (cd $BENCH_DIR/corpus && sha256sum -c SHA256SUMS)"
+ say "+ zstd -dc $CORPUS_TAR | tar -x -C $CORPUS_DIR"
+ say "+ verify extracted files against corpus/MANIFEST.tsv (sha256 column)"
+else
+ (cd "$BENCH_DIR/corpus" && sha256sum -c --quiet SHA256SUMS)
+ zstd -dc "$CORPUS_TAR" | tar -x -C "$CORPUS_DIR"
+ (cd "$CORPUS_DIR" && tail -n +2 "$BENCH_DIR/corpus/MANIFEST.tsv" \
+ | awk -F'\t' '{print $6 " " $4}' | sha256sum -c --quiet)
+fi
+CORPUS="$CORPUS_DIR/okta-bulk"
+
+# ── build each distinct SHA once ────────────────────────────────────────────
+# build_arm SHA PREBUILT -> prints path of the binary to use
+build_arm() {
+ local sha="$1" prebuilt="$2" short wt out
+ short="${sha:0:9}"
+ out="$SCRATCH/bin/xerj-$short"
+ if [ -n "$prebuilt" ]; then
+ echo "$prebuilt"
+ return
+ fi
+ if [ ! -x "$out" ] || [ "$DRY" = 1 ]; then
+ wt="$SCRATCH/wt-$short"
+ run git -C "$REPO_ROOT" worktree add --detach "$wt" "$sha" >&2
+ run bash -c "cd '$wt/engine' && cargo build --release -j 2 -p xerj-server" >&2
+ run mkdir -p "$SCRATCH/bin" >&2
+ run cp "$wt/engine/target/release/xerj" "$out" >&2
+ fi
+ echo "$out"
+}
+# BIN_<arm> are read indirectly in run_one.
+# shellcheck disable=SC2034
+BIN_control="$(build_arm "$CONTROL_SHA" "${CONTROL_BIN:-}")"
+if [ "$CANDIDATE_SHA" = "$CONTROL_SHA" ] && [ -z "${CANDIDATE_BIN:-}" ]; then
+ # shellcheck disable=SC2034
+ BIN_candidate="$BIN_control"
+else
+ # shellcheck disable=SC2034
+ BIN_candidate="$(build_arm "$CANDIDATE_SHA" "${CANDIDATE_BIN:-}")"
+fi
+
+# ── server launcher (isolation wrapper; XERJ_BIN points at this) ───────────
+LAUNCH="$SCRATCH/launch.sh"
+if [ "$DRY" != 1 ]; then
+ if [ "$ISOLATION" = systemd-scope ]; then
+ cat >"$LAUNCH" <<'EOF'
+#!/usr/bin/env bash
+exec systemd-run --user --scope -q -p CPUQuota=400% -p MemoryMax=16G -- "$REAL_BIN" "$@"
+EOF
+ else
+ cat >"$LAUNCH" <<'EOF'
+#!/usr/bin/env bash
+exec taskset -c 0-3 "$REAL_BIN" "$@"
+EOF
+ fi
+ chmod +x "$LAUNCH"
+fi
+
+# proc_stat PID -> "utime_ticks stime_ticks" (comm may contain spaces)
+proc_cpu_ticks() {
+ local s
+ s="$(cat "/proc/$1/stat")"
+ s="${s##*) }"
+ # shellcheck disable=SC2086
+ set -- $s
+ echo "${12} ${13}"
+}
+
+# resolve_server_pid: the pid whose exe is the server binary (the launcher
+# execs, so this is normally REPRO_PID itself).
+resolve_server_pid() {
+ local real="$1" p c
+ local -a kids
+ p="$REPRO_PID"
+ if [ "$(readlink -f "/proc/$p/exe" 2>/dev/null)" = "$(readlink -f "$real")" ]; then
+ echo "$p"
+ return
+ fi
+ for c in /proc/"$p"/task/*/children; do
+ read -r -a kids <"$c" 2>/dev/null || continue
+ for p in "${kids[@]}"; do
+ if [ "$(readlink -f "/proc/$p/exe" 2>/dev/null)" = "$(readlink -f "$real")" ]; then
+ echo "$p"
+ return
+ fi
+ done
+ done
+ echo "$REPRO_PID"
+}
+
+# ── one run ─────────────────────────────────────────────────────────────────
+run_one() {
+ local arm="$1" i="$2" bin rundir state port_msg
+ local binvar="BIN_$arm"
+ bin="${!binvar}"
+ rundir="$RESULTS/run-$arm-$i"
+ state="$SCRATCH/state-$arm-$i"
+ say ""
+ say "== run $arm #$i (binary $bin) =="
+ if [ "$DRY" = 1 ]; then
+ say "+ mkdir -p $rundir $state"
+ say "+ port=\$(repro_free_port $PORT_MIN) # never 9200"
+ say "+ env REAL_BIN=$bin XERJ_NUM_CPUS=4 XERJ_PROF=1 XERJ_EMBED_MODE=neural HF_HOME=$HF_HOME \\"
+ if [ "$ISOLATION" = systemd-scope ]; then
+ say " systemd-run --user --scope -q -p CPUQuota=400% -p MemoryMax=16G -- $bin --insecure --port <port> --data-dir <fresh mktemp dir> &"
+ else
+ say " taskset -c 0-3 $bin --insecure --port <port> --data-dir <fresh mktemp dir> &"
+ fi
+ say "+ wait for GET / on <port>"
+ say "+ timeout $RUN_TIMEOUT $bin autoindex $CORPUS --url http://127.0.0.1:<port> --prefix bench --bulk-mb 8 --workers 4 --progress json --state-dir $state --approve proceed > $rundir/client.log 2>&1"
+ say "+ read VmHWM from /proc/<pid>/status and utime+stime from /proc/<pid>/stat"
+ say "+ copy server log to $rundir/server.log; stop server; write $rundir/summary.json"
+ return
+ fi
+ if [ -e "$rundir" ]; then
+ echo "ERROR: $rundir already exists; set RESULTS_DATE to a new name." >&2
+ return 1
+ fi
+ mkdir -p "$rundir" "$state"
+
+ repro_free_port "$PORT_MIN"
+ if [ "$REPRO_PORT" = 9200 ]; then echo "refusing port 9200" >&2; return 1; fi
+ XERJ_BIN="$LAUNCH"
+ export XERJ_BIN
+ repro_node_start - "REAL_BIN=$bin" XERJ_NUM_CPUS=4 XERJ_PROF=1 \
+ XERJ_EMBED_MODE=neural "HF_HOME=$HF_HOME"
+ repro_node_wait
+
+ local spid t0 t1 rc=0 hwm cpu_ticks
+ spid="$(resolve_server_pid "$bin")"
+ t0="$(date +%s.%N)"
+ timeout "$RUN_TIMEOUT" "$bin" autoindex "$CORPUS" \
+ --url "http://127.0.0.1:$REPRO_PORT" --prefix bench \
+ --bulk-mb 8 --workers 4 --progress json \
+ --state-dir "$state" --approve proceed >"$rundir/client.log" 2>&1 || rc=$?
+ t1="$(date +%s.%N)"
+
+ hwm="$(awk '/^VmHWM:/ {print $2}' "/proc/$spid/status" 2>/dev/null || true)"
+ cpu_ticks="$(proc_cpu_ticks "$spid" 2>/dev/null || true)"
+ cp "$REPRO_LOG" "$rundir/server.log"
+ repro_node_stop
+ port_msg="$REPRO_PORT"
+ REPRO_PID=""
+
+ python3 -I - "$rundir" "$rc" "$(awk -v a="$t0" -v b="$t1" 'BEGIN{printf "%.3f", b-a}')" \
+ "${hwm:-}" "${cpu_ticks:-}" "$CLK_TCK" "$arm" "$i" "$port_msg" <<'PY'
+import json, math, re, sys
+
+(rundir, rc, wrapper_s, hwm, ticks, clk, arm, idx, port) = sys.argv[1:10]
+
+
+def pct(vals, p):
+ if not vals:
+ return None
+ s = sorted(vals)
+ return s[max(0, math.ceil(p * len(s)) - 1)]
+
+
+def read(path):
+ try:
+ with open(path, errors="replace") as f:
+ return f.read().splitlines()
+ except OSError:
+ return []
+
+
+client = read(rundir + "/client.log")
+server = read(rundir + "/server.log")
+
+done = None
+for line in client:
+ line = line.strip()
+ if line.startswith("{"):
+ try:
+ o = json.loads(line)
+ except ValueError:
+ continue
+ if o.get("event") == "done":
+ done = o
+if done is None:
+ for line in client:
+ if line.startswith("xerj-done "):
+ done = {}
+ for tok in line.split()[1:]:
+ if "=" in tok:
+ k, v = tok.split("=", 1)
+ done[k] = v
+ if "wall" in done:
+ done["wall_s"] = done["wall"].rstrip("s")
+
+
+def num(v):
+ try:
+ return float(v)
+ except (TypeError, ValueError):
+ return None
+
+
+bulk, embed = [], []
+for line in server:
+ if "XERJ_PROF" not in line:
+ continue
+ m = re.search(r"XERJ_PROF bulk .*?\btotal_ms=(\d+)", line)
+ if m:
+ bulk.append(float(m.group(1)))
+ m = re.search(r"\bembed_ms=(\d+)", line)
+ if m:
+ embed.append(float(m.group(1)))
+
+retry_item = sum("semantic embedding window failed; retrying per item" in l for l in server)
+retry_embed = sum("embed retry" in l for l in server)
+r429 = sum(bool(re.search(r"\b429\b", l)) for l in server + client)
+
+wall = num(done.get("wall_s")) if done else None
+records = num(done.get("records")) if done else None
+cpu_s = None
+if ticks.strip():
+ a, b = ticks.split()
+ cpu_s = round((int(a) + int(b)) / int(clk), 2)
+
+summary = {
+ "arm": arm,
+ "run": int(idx),
+ "port": int(port),
+ "ok": bool(done and done.get("ok") in (True, "true") and rc == "0"),
+ "client_exit": int(rc),
+ "wall_s": wall,
+ "wall_wrapper_s": num(wrapper_s),
+ "records": int(records) if records is not None else None,
+ "records_per_s": round(records / wall, 2) if records and wall else None,
+ "bulk_total_ms_n": len(bulk),
+ "bulk_total_ms_p50": pct(bulk, 0.50),
+ "bulk_total_ms_p95": pct(bulk, 0.95),
+ "embed_ms_n": len(embed),
+ "embed_ms_p50": pct(embed, 0.50),
+ "embed_ms_p95": pct(embed, 0.95),
+ "retries_window_failed": retry_item,
+ "retries_embed": retry_embed,
+ "client_bulk_retries": int(num(done.get("bulk_retries")) or 0) if done else None,
+ "http_429": r429,
+ "peak_rss_kb": int(hwm) if hwm.strip() else None,
+ "cpu_s": cpu_s,
+}
+summary["retries"] = retry_item + retry_embed + (summary["client_bulk_retries"] or 0)
+with open(rundir + "/summary.json", "w") as f:
+ json.dump(summary, f, indent=2)
+ f.write("\n")
+print(json.dumps(summary))
+PY
+}
+
+# ── interleave ──────────────────────────────────────────────────────────────
+run mkdir -p "$RESULTS"
+if [ "$DRY" != 1 ]; then
+ python3 -I - "$RESULTS/meta.json" "$CONTROL_SHA" "$CANDIDATE_SHA" "$N" "$ISOLATION" <<'PY'
+import json, platform, os, sys, time
+path, c, k, n, iso = sys.argv[1:6]
+json.dump({
+ "control_sha": c, "candidate_sha": k, "n": int(n), "isolation": iso,
+ "started_utc": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
+ "host": platform.platform(), "nproc": os.cpu_count(),
+}, open(path, "w"), indent=2)
+PY
+fi
+for i in $(seq 1 "$N"); do
+ run_one control "$i"
+ run_one candidate "$i"
+done
+
+# ── report ──────────────────────────────────────────────────────────────────
+if [ "$DRY" = 1 ]; then
+ say ""
+ say "+ write $RESULTS/report.md (median and p95 per arm; control max-min spread; delta flagged only if > spread)"
+ exit 0
+fi
+python3 -I - "$RESULTS" <<'PY'
+import glob, json, math, statistics, sys
+
+res = sys.argv[1]
+meta = json.load(open(res + "/meta.json"))
+runs = {"control": [], "candidate": []}
+for p in sorted(glob.glob(res + "/run-*/summary.json")):
+ s = json.load(open(p))
+ runs[s["arm"]].append(s)
+
+METRICS = [
+ ("wall_s", "wall_s"),
+ ("records_per_s", "records/s"),
+ ("bulk_total_ms_p50", "bulk total_ms p50"),
+ ("bulk_total_ms_p95", "bulk total_ms p95"),
+ ("embed_ms_p50", "embed_ms p50"),
+ ("embed_ms_p95", "embed_ms p95"),
+ ("retries", "retries"),
+ ("http_429", "http_429"),
+ ("peak_rss_kb", "peak_rss_kb"),
+ ("cpu_s", "cpu_s"),
+]
+
+
+def vals(arm, key):
+ return [r[key] for r in runs[arm] if r.get("ok") and r.get(key) is not None]
+
+
+def p95(v):
+ s = sorted(v)
+ return s[max(0, math.ceil(0.95 * len(s)) - 1)]
+
+
+def f(x):
+ return "-" if x is None else ((f"{x:.2f}" if abs(x) < 10 else f"{x:.1f}") if isinstance(x, float) else str(x))
+
+
+out = []
+out.append(f"# neural-bulk A/B report {meta['started_utc']}\n")
+out.append(f"- control `{meta['control_sha']}`")
+out.append(f"- candidate `{meta['candidate_sha']}`")
+out.append(f"- runs per arm requested {meta['n']}; isolation {meta['isolation']}; host {meta['host']}; nproc {meta['nproc']}")
+for arm in runs:
+ good = sum(1 for r in runs[arm] if r.get("ok"))
+ bad = [f"#{r['run']}(exit {r['client_exit']})" for r in runs[arm] if not r.get("ok")]
+ out.append(f"- {arm}: {good} ok runs" + (f", EXCLUDED failed runs {', '.join(bad)}" if bad else ""))
+out.append("")
+out.append("Spread is the control arm's max-min over its ok runs. A delta is flagged only if |delta| exceeds it.")
+out.append("p95 over N runs is the nearest-rank value (the max when N <= 20).\n")
+out.append("| metric | control median | control p95 | control spread | candidate median | candidate p95 | delta (cand - ctrl median) | verdict |")
+out.append("|---|---|---|---|---|---|---|---|")
+for key, label in METRICS:
+ c, k = vals("control", key), vals("candidate", key)
+ if not c or not k:
+ out.append(f"| {label} | {f(statistics.median(c)) if c else '-'} | - | - | {f(statistics.median(k)) if k else '-'} | - | - | no data |")
+ continue
+ cm, km = statistics.median(c), statistics.median(k)
+ spread = max(c) - min(c)
+ d = km - cm
+ verdict = "EXCEEDS SPREAD" if abs(d) > spread else "within noise"
+ pctd = f" ({100 * d / cm:+.1f}%)" if cm else ""
+ out.append(f"| {label} | {f(cm)} | {f(p95(c))} | {f(spread)} | {f(km)} | {f(p95(k))} | {f(d)}{pctd} | {verdict} |")
+
+if meta["control_sha"] == meta["candidate_sha"]:
+ c, k = vals("control", "bulk_total_ms_p95"), vals("candidate", "bulk_total_ms_p95")
+ out.append("")
+ if c and k:
+ cm, km = statistics.median(c), statistics.median(k)
+ diff = abs(km - cm) / cm * 100 if cm else float("inf")
+ out.append(f"Control-vs-control precondition: bulk total_ms p95 medians {f(cm)} vs {f(km)}, "
+ f"difference {diff:.1f}% -> {'PASS (<10%)' if diff < 10 else 'FAIL (>=10%): no candidate number may be published from this host/setup'}")
+ else:
+ out.append("Control-vs-control precondition: no data (runs failed or XERJ_PROF lines missing).")
+open(res + "/report.md", "w").write("\n".join(out) + "\n")
+print("\n".join(out))
+PY