:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
chore(review-hog): remove the push gate shadow rules (#127210)
Julian Bez596b34df51a1
diff --git a/products/review_hog/ARCHITECTURE.md b/products/review_hog/ARCHITECTURE.mdindex 2cdc91c326da..e0b56e486260 100644--- a/products/review_hog/ARCHITECTURE.md+++ b/products/review_hog/ARCHITECTURE.md@@ -988,17 +988,16 @@ An explicit Full request remains pending during an active Flash turn and runs ifAutomatic pushes coalesce to the latest head; a rule change that stops the author's automatic review prevents pending automatic reviews from starting without canceling a running turn.`automatic_reviewed_head_sha` advances only after publication succeeds or finds nothing to publish, so a failed publication can retry and an empty review does not repeat.An automatic follow-up turn (one with an `automatic_reviewed_head_sha`) first runs the push gate (`reviewer/push_gate.py`) on the PR's own commits since that head.-Its rules, in order: `no_new_commits` (the PR gained no commit and its full diff is unchanged; a force-push that drops commits runs the review), `reviewhog_commits_only` (the ReviewHog app authored every new commit, such as resolution fixes; this includes the PostHog app's commits on self-driving PRs, which share the app identity), `merge_only` (the PR's full diff against its base is unchanged, as after a base merge or a rebase), `docs_only` (the new own commits touch only docs, lockfiles, snapshots, images, or generated files), `small_change` (the compare of the last reviewed head and the push head changes 1 to `SMALL_CHANGE_MAX_NET_CODE_LINES` code lines; a per-PR Redis counter allows `SMALL_CHANGE_MAX_DEFERRALS` deferrals in a row, then the next small push runs the review), `system_one_below_threshold` (System One rates the own code patches below `SYSTEM_ONE_SKIP_BELOW`), `draft` (the PR is a draft and no earlier rule skips the push), `late_follow_up` (the push would start review number `LATE_FOLLOW_UP_RUN_INDEX` or a later one), `small_pr` (the PR has fewer than `SMALL_PR_REVIEWABLE_LINES_BELOW` reviewable lines, the size the lens parts use), `quiet_small_push` (the last review kept no finding and the push changes at most `QUIET_PUSH_MAX_OWN_CODE_LINES` own code lines; skipped pushes add up until one crosses the limit), `next_push_likely` (the PR is not a draft, no earlier rule matched, and the chance of another automatic follow-up is at least `NEXT_PUSH_SKIP_AT`), and `throttled` (the PR spent its review budget).-The `draft` rule reads the draft state from the turn's own PR snapshot: every turn fetches the PR before the gate and the snapshot load is latest-wins, so a PR marked ready at an unchanged head reads as ready. A draft skip reads the PR from GitHub again after the gate's other reads: a PR marked ready while the turn runs sends a ready request that the queue drops, because it asks for the head the running turn gates. If the PR is ready by then, the gate uses the decision of the rules before `draft`. A failed read reviews the push. Only a PR marked ready between that read and the end of the turn keeps its changes unreviewed until the next trigger. A draft decision keeps the System One answer and `own_code_lines` of the rules that ran before it.+Its rules, in order: `no_new_commits` (the PR gained no commit and its full diff is unchanged; a force-push that drops commits runs the review), `reviewhog_commits_only` (the ReviewHog app authored every new commit, such as resolution fixes; this includes the PostHog app's commits on self-driving PRs, which share the app identity), `merge_only` (the PR's full diff against its base is unchanged, as after a base merge or a rebase), `draft` (the PR is a draft and no earlier rule skips the push), `late_follow_up` (the push would start review number `LATE_FOLLOW_UP_RUN_INDEX` or a later one), `small_pr` (the PR has fewer than `SMALL_PR_REVIEWABLE_LINES_BELOW` reviewable lines, the size the lens parts use), `quiet_small_push` (the last review kept no finding and the push changes at most `QUIET_PUSH_MAX_OWN_CODE_LINES` own code lines; skipped pushes add up until one crosses the limit), and `throttled` (the PR spent its review budget).+The `draft` rule reads the draft state from the turn's own PR snapshot: every turn fetches the PR before the gate and the snapshot load is latest-wins, so a PR marked ready at an unchanged head reads as ready. A draft skip reads the PR from GitHub again after the gate's other reads: a PR marked ready while the turn runs sends a ready request that the queue drops, because it asks for the head the running turn gates. If the PR is ready by then, the gate uses the decision of the rules before `draft`. A failed read reviews the push. Only a PR marked ready between that read and the end of the turn keeps its changes unreviewed until the next trigger. A draft decision keeps the `own_code_lines` of the rules that ran before it.The `throttled` rule keeps a token bucket per PR in Redis (`PUSH_BUDGET`): it holds two reviews and gets one back every 30 minutes. Only a push that every earlier rule lets through takes a token, so a draft push or a merge from the base costs nothing. A push that finds the bucket empty waits for the next trigger like any other skip. A Redis failure reviews the push. A head pays once: a turn that runs again at the same head, as an activity retry or a retried `review-pr` child, does not take a second token. A retry after a denial asks the bucket again, so it cannot get past the budget. The first review and human triggers never reach the gate, so they neither take nor need a token.Two per-PR splits run on top of these rules. Their flags are keyed like the coordinator flag, and a flag that is off puts every PR in the control arm.`reviewhog-zero-change-skip` adds the `zero_change` rule after `quiet_small_push`: the gate compares the stored single-agent PR snapshots of the last automatically reviewed head and this head (`ChangedSinceReview`), and a PR whose reviewed files did not change skips in the treatment arm. `merge_only` already skips a push whose whole PR diff is unchanged (a base merge, a rebase, an amend), so this rule catches pushes that change only files the review leaves out, such as generated files and snapshots. The snapshots leave lockfiles out too, so a PR with a lockfile at either head never matches, and neither does a PR with a file whose patch GitHub left out (`PRChange.comparable`). The control arm reviews the push and records the match, so it is the audit. A missing snapshot never matches. A skipped push does not move the baseline, so the next comparison covers it.`reviewhog-delta-band` lets a turn that passes every rule review only what changed: from run `DELTA_BAND_FIRST_RUN_INDEX` (3) on, with changed reviewable lines at most `DELTA_BAND_MAX_CHANGE_SHARE` (3%) of the PR's reviewable lines, no dependency file changed, and a comparison the snapshots show in full: no lockfile in the PR, no file without a patch, and no file that left the PR (`plan_delta_review`). The first follow-up always reviews the whole PR. A delta turn passes the baseline head to its sessions (`SandboxStageInput.delta_since_head_sha`), runs one session per lens, and each prompt shows only the changed files with the diff blocks around the changed lines (`delta_files`), plus the earlier findings, the full repository at the head, and the git command for the whole diff since the baseline, which also shows code the PR no longer changes. A gate failure, or a baseline snapshot that is gone when the prompt is built, gives the whole-PR prompt, also in each lens session.The gate event carries `pr_changed_lines`, both arms, `coordinator`, `delta_eligible` and `delta_review`; every completed event carries `zero_change_skip_arm`, `delta_band_arm` and `delta_review`, so each split compares by arm over PR lifecycles. A coordinated PR always records the control arm of the delta split, so a delta comparison leaves out gate events with `coordinator` set.-The `next_push_likely` chance comes from a table fitted offline on closed PRs and refitted by hand. Its key is the follow-up number, capped at 8, and the PR size band (additions plus deletions: up to 200, up to 1000, more). A size cell with fewer than 10 PRs falls back to the chance for the follow-up number alone. The follow-up number is `run_index - 1`, so it also counts comment and UI turns, which the fit left out.-Each rule has a `SKIP_*` switch. `no_new_commits`, `reviewhog_commits_only`, `merge_only`, `draft`, `late_follow_up`, `small_pr`, `quiet_small_push`, and `throttled` skip; the three value rules also take over a shadow match of an earlier rule, but never a gate failure, and they do not run on a retry of a turn that already finalized its head; `docs_only`, `small_change`, System One, and `next_push_likely` (`SKIP_NEXT_PUSH`) only record a shadow decision until production data confirms them.+Each rule has a `SKIP_*` switch, and every rule skips. The three value rules never take over a gate failure, and they do not run on a retry of a turn that already finalized its head. A push that no rule matches reviews with the reason `no_rule_matched`.A skipped turn posts no status comment and leaves `automatic_reviewed_head_sha` in place, so the next push is judged against the last reviewed head.-Any gate failure reviews the push. Each decision emits `reviewhog_push_gate_decided` with `skipped`, `would_skip`, the reason, the System One probability and model, `own_code_lines`, the lines the new own commits change in code files (empty when GitHub could not list every file of a commit), `net_code_lines` and `small_change_deferrals` (empty when the gate did not read them), `pr_draft`, the draft state the gate used (null when the snapshot is unavailable), `reviewable_lines` and `previous_findings`, the inputs of the value rules (null when the gate could not read them), and `next_push_probability` (null on a draft and when the snapshot is unavailable; set on every other decision, also when another rule decided); the event key includes it and the reason, so a draft or throttle skip and a later review at the same head stay two events, so skip rules that weigh push size can be calibrated offline.+Any gate failure reviews the push. Each decision emits `reviewhog_push_gate_decided` with `skipped`, `would_skip`, the reason, `own_code_lines`, the lines the new own commits change in code files (empty when GitHub could not list every file of a commit), `pr_draft`, the draft state the gate used (null when the snapshot is unavailable), and `reviewable_lines` and `previous_findings`, the inputs of the value rules (null when the gate could not read them); the event key includes the draft state and the reason, so a draft or throttle skip and a later review at the same head stay two events, so skip rules that weigh push size can be calibrated offline.The first automatic review and every comment, UI, inbox, Stamphog, and manual trigger never reach the gate.An automatic review of a draft that posts findings ends with a note: it is the only automatic review while the PR is a draft, `@posthog review` (or the Code review page when `github-commands` is off) reviews new changes sooner, and marking the PR ready reviews them otherwise. Comment, UI, and Stamphog reviews leave the note out, because they also run for authors without automatic reviews.Reviews start only from a trigger (a push, a comment, the UI, the inbox, a Stamphog handoff, or a manual run) and run right away.diff --git a/products/review_hog/DECISIONS.md b/products/review_hog/DECISIONS.mdindex 61ef8ee1df44..5018783bbfd1 100644--- a/products/review_hog/DECISIONS.md+++ b/products/review_hog/DECISIONS.md@@ -198,6 +198,22 @@ read `FINAL_REPORT.md` there first (config glossary + coverage matrix + ranking)rate drops materially (toward ≤50%) on frozen-PR evals with the valid-finding set intact (item 5'scoverage matrix as the guard); kill if valid findings drop with the noise.+### ✅ BUILT 2026-11-02 — push gate: the four shadow rules are gone++- **What.** `docs_only`, `small_change`, `system_one_below_threshold`, and `next_push_likely` are removed, with their `SKIP_*` switches, the small-change Redis counter, the next-push table, and the System One call.+ A push that no remaining rule matches reviews with the reason `no_rule_matched`.+ The gate event drops `system_one_probability`, `system_one_model`, `system_one_skip_below`, `net_code_lines`, `small_change_deferrals`, and `next_push_probability`.+- **Why.** Each rule logged a shadow decision while the review still ran, so every match is a measured review (review-eval `experiments/2026-11-02-shadow-rules`, Oct 17 to Nov 2).+ None of them earned a live switch:+ - `next_push_likely` matched the most productive follow-ups: about 1.7 times the must-fix labels per review of other follow-ups. About 30% of its matches were the PR's last automatic review. Live, it also needed a fix first, because a skip does not advance `run_index`, so it would match every later push.+ - `small_change` held must-fix labels at close to the rate of other follow-ups, and almost half of its matches were the PR's last automatic review. The preconditions in the 2026-10-15 entry (a review on Stamphog's request, a "latest push not reviewed" state) were never built. `quiet_small_push` already skips small pushes after a review that kept nothing.+ - `system_one_below_threshold` matched few pushes, and about 15% of System One calls came back unavailable. The earlier replays already found its below-threshold pushes carried must-fix labels at the same rate as the rest.+ - `docs_only` saved the least, and the Markdown it treats as docs includes agent skill files, which this repo treats as instructions.+- **Behavior change.** A System One outage was a gate failure, so it kept those pushes out of rules 5 to 7, `zero_change`, and the delta split. Those pushes now meet every rule like any other push.+ A push GitHub cannot show in full (a left-out patch, or more than `MAX_OWN_PATCH_CHARS` of patch text) still fails open as `interdiff_too_large`.+- **Replay.** The gate activity returns `PushGateDecision`, and Temporal's converter rejects a literal it does not know.+ `RetiredReason` keeps the old reason strings so a `review-pr` history recorded before the removal still decodes. Remove it once no such history is open.+### ✅ BUILT 2026-10-29 — no label starts a review; Stamphog hands off through the finished-review hook (#124327)- **What.** The `reviewhog` label no longer starts a review, and a `labeled` delivery starts nothing.@@ -236,7 +252,7 @@ read `FINAL_REPORT.md` there first (config glossary + coverage matrix + ranking)- System One below 0.7: it saves less per must-fix label held back than the push-size rule.- A cap from review 5: it saves about half as much and holds back almost as many must-fix labels.-### ✅ BUILT 2026-10-15 — push gate: small-change deferral (shadow)+### ✅ BUILT 2026-10-15 — push gate: small-change deferral (shadow; removed 2026-11-02)- **What.** A new push gate rule, `small_change`, sits after `docs_only` and before `system_one_below_threshold`. Itmatches a push that changes 1 to `SMALL_CHANGE_MAX_NET_CODE_LINES` (10) code lines net: the additions plus deletionsdiff --git a/products/review_hog/backend/reviewer/push_gate.py b/products/review_hog/backend/reviewer/push_gate.pyindex 4a527ffcb770..d6e5127771d6 100644--- a/products/review_hog/backend/reviewer/push_gate.py+++ b/products/review_hog/backend/reviewer/push_gate.py@@ -11,47 +11,30 @@ The rules, in order. The first rule that matches gives the reason:resolution stage pushed. Reviewing them again would review ReviewHog's own answer to its review.3. `merge_only`: the PR's full diff against its base is the same at both heads (a base merge or arebase). A merge that changes the PR's own lines runs the review.-4. `docs_only`: the new own commits touch only docs, lockfiles, snapshots, images, and generated files.-5. `small_change`: the push changes at most `SMALL_CHANGE_MAX_NET_CODE_LINES` code lines net, read from- the compare of the last reviewed head and the push head. The rule matches only when the push head- descends from the last reviewed head: after a rebase or a force-push the compare does not show the- push's own change. After `SMALL_CHANGE_MAX_DEFERRALS` deferrals in a row the next small push runs the- review, so small pushes cannot pile up without a review. A push that every rule lets through to a review- sets the count back to 0. A push that a rule skips keeps the count, because nothing was reviewed. The- count is stored only after the last rule has decided, so a draft or throttle skip also keeps it. When- the count cannot be stored, because another turn of the PR stored one after this turn read it or Redis- refuses the write, a small push runs the review.-6. `system_one_below_threshold`: System One rates the new own commits below a threshold.-7. `draft`: the PR is a draft and no rule above skips the push. A draft cannot merge, and marking it- ready starts a turn that reviews everything since the last reviewed head. The rules above still run,- so a draft decision keeps the System One answer and the push size for calibration.-8. `late_follow_up`: the push would start review number `LATE_FOLLOW_UP_RUN_INDEX` of the PR or a later one.-9. `small_pr`: the PR has fewer than `SMALL_PR_REVIEWABLE_LINES_BELOW` reviewable lines, the changed lines+4. `draft`: the PR is a draft and no rule above skips the push. A draft cannot merge, and marking it+ ready starts a turn that reviews everything since the last reviewed head. A draft decision keeps the+ push size the rules above read.+5. `late_follow_up`: the push would start review number `LATE_FOLLOW_UP_RUN_INDEX` of the PR or a later one.+6. `small_pr`: the PR has fewer than `SMALL_PR_REVIEWABLE_LINES_BELOW` reviewable lines, the changed lineswithout generated and vendored files that the lens parts are sized by.-10. `quiet_small_push`: the last review kept no finding and the push changes at most+7. `quiet_small_push`: the last review kept no finding and the push changes at most`QUIET_PUSH_MAX_OWN_CODE_LINES` own code lines. Skipped pushes add up, so a later push that brings theown code lines over the limit runs the review.- Rules 8 to 10 match the follow-ups that surfaced the least per review in production. A skipped push- does not move the baseline, so the next review covers its changes. These rules take over a shadow- match of the rules above, so a push that only a shadow rule matched can still skip. They never take- over a gate failure, and they do not run on a retry of a turn that already finalized its head.-11. `next_push_likely`: on a PR that is not a draft, when no rule above matched, another automatic- follow-up is likely to come. Skipped changes add up and the next review covers them. The chance- comes from a table fitted offline on the follow-up number and the PR size. Every decision on a- PR that is not a draft records the chance, also when another rule decided.-12. `throttled`: the PR spent its review budget. Each PR has a token bucket (Turner 1986) that holds- `PUSH_BUDGET.burst` reviews and gets one back every `3600 / PUSH_BUDGET.per_hour` seconds. A push- that every rule above lets through takes a token, and a push that finds the bucket empty waits:- the next turn that runs reviews everything since the last reviewed head. This caps bursts of- pushes, and it never schedules a review on its own.--13. `zero_change`: the reviewed files of the PR are the same as at the last automatically reviewed head. It compares+ Rules 5 to 7 match the follow-ups that surfaced the least per review in production. A skipped push+ does not move the baseline, so the next review covers its changes. They never take over a gate+ failure, and they do not run on a retry of a turn that already finalized its head.+8. `zero_change`: the reviewed files of the PR are the same as at the last automatically reviewed head. It comparesthe stored PR snapshots of both heads. Rule 3 already skips a push whose whole PR diff is unchanged, so this rulecatches pushes that change only files the review leaves out, such as generated files and snapshots. The snapshotsleave lockfiles out too, so a PR with a lockfile at either head never matches, and neither does a PR with a filewhose patch GitHub left out (`PRChange.comparable`). A push after skipped pushes compares against the same- baseline. It runs after rule 10 and before rule 11, and only a PR in the treatment arm of- `reviewhog-zero-change-skip` skips. An unknown comparison (no snapshot at either head) never matches.+ baseline. Only a PR in the treatment arm of `reviewhog-zero-change-skip` skips. An unknown comparison (no snapshot+ at either head) never matches.+9. `throttled`: the PR spent its review budget. Each PR has a token bucket (Turner 1986) that holds+ `PUSH_BUDGET.burst` reviews and gets one back every `3600 / PUSH_BUDGET.per_hour` seconds. A push+ that every rule above lets through takes a token, and a push that finds the bucket empty waits:+ the next turn that runs reviews everything since the last reviewed head. This caps bursts of+ pushes, and it never schedules a review on its own.A turn that passes every rule can review only what changed (`plan_delta_review`): a follow-up from run`DELTA_BAND_FIRST_RUN_INDEX` on, whose changed reviewable lines are at most `DELTA_BAND_MAX_CHANGE_SHARE` of the PR's@@ -63,8 +46,7 @@ Each rule has its own switch. A rule that is switched off still reports the turn(`would_skip`), so production data can calibrate it before it skips anything. Every failure failsopen: the turn runs.-The rules, the non-code file list, the System One question, and the state follow the offline study-in `review-eval/production/push-gate`. The threshold is only valid for the state built the same way.+Rules 1 to 3 and the non-code file list follow the offline study in `review-eval/production/push-gate`."""import re@@ -77,8 +59,6 @@ from redis.exceptions import RedisErrorfrom posthog.dataclasses import frozenfrom posthog.egress.github.transport import GitHubRateLimitError-from posthog.llm.system_one import NoulAnswer, NoulQuestion, SystemOneNotConfigured, SystemOneRequestFailed-from posthog.llm.system_one_client import build_system_one_clientfrom posthog.redis import get_clientfrom posthog.token_bucket import BucketUnavailable, Budget, consume@@ -94,32 +74,12 @@ logger = logging.getLogger(__name__)SKIP_NO_NEW_COMMITS = TrueSKIP_REVIEWHOG_COMMITS_ONLY = TrueSKIP_MERGE_ONLY = True-SKIP_DOCS_ONLY = False-SKIP_SMALL_CHANGE = False-SKIP_SYSTEM_ONE = FalseSKIP_DRAFT = TrueSKIP_LATE_FOLLOW_UP = TrueSKIP_SMALL_PR = TrueSKIP_QUIET_SMALL_PUSH = True-# Before this goes on, `next_push_likely` must defer only once per reviewed head. A skip does not-# advance run_index, so the same chance would match every later push and no review would run.-SKIP_NEXT_PUSH = FalseSKIP_THROTTLED = True-NEXT_PUSH_SKIP_AT = 0.85-# Picked by the replay in `review-eval/experiments/2026-10-15-gate-surfacing`.-SMALL_CHANGE_MAX_NET_CODE_LINES = 10-SMALL_CHANGE_MAX_DEFERRALS = 2-# Long enough to span a PR's quiet days, short enough that an abandoned PR's counter goes away.-_SMALL_CHANGE_DEFERRALS_TTL_SECONDS = 3 * 24 * 3600-# Compare-and-set: two turns of one PR can overlap, and each counts from the value it read.-_STORE_STREAK_IF_UNCHANGED = """-if (redis.call('GET', KEYS[1]) or '') ~= ARGV[1] then- return 0-end-redis.call('SET', KEYS[1], ARGV[2], 'EX', ARGV[3])-return 1-"""# Picked by a replay of production follow-ups in `review-eval/experiments/2026-10-16-follow-up-value`.LATE_FOLLOW_UP_RUN_INDEX = 9SMALL_PR_REVIEWABLE_LINES_BELOW = 150@@ -138,79 +98,13 @@ _DEPENDENCY_FILE = re.compile(# Longer than a turn and its retry take, so a retried turn finds that its head paid already._PAID_HEAD_TTL_SECONDS = 6 * 3600-# The JevK5 build PostHog hosts on the ai-gateway. The study calibrated the threshold on TypeSafe's-# `jev-latest`, so the shadow decisions must confirm it holds for this model before SKIP_SYSTEM_ONE goes on.-PUSH_GATE_MODEL = "posthog/hogference/jevk5-fp8-0.2"-SYSTEM_ONE_SKIP_BELOW = 0.30-# The study sent states up to 64k tokens, at about 3.5 characters per token of code.-MAX_STATE_CHARS = 224_000-SYSTEM_ONE_TIMEOUT_SECONDS = 15.0+# GitHub leaves out the patch of a very large file. A push with more patch text than this runs the review too.+MAX_OWN_PATCH_CHARS = 224_000# Each new own commit costs one GitHub call to read its patch.MAX_OWN_COMMITS = 20# GitHub's compare endpoint pages its commit list at this size._COMPARE_COMMITS_PER_PAGE = 100-SizeBand = Literal["<=200", "201-1000", ">1000"]---@frozen-class _NextPushCell:- probability: float- prs: int---# The chance that another automatic follow-up comes after follow-up number k, from review-eval-# experiments/2026-10-14-draft-deferral (NextPushLookup), fitted on closed PostHog/posthog PRs from-# Oct 8-14 2026. A refit updates these tables by hand. Key 8 holds every follow-up from the eighth on.-_NEXT_PUSH_LAST_FOLLOW_UP = 8-_NEXT_PUSH_MIN_CELL_PRS = 10-_NEXT_PUSH_BY_FOLLOW_UP: dict[int, float] = {- 1: 0.629,- 2: 0.625,- 3: 0.740,- 4: 0.812,- 5: 0.710,- 6: 0.731,- 7: 0.842,- 8: 0.783,-}-_NEXT_PUSH_BY_FOLLOW_UP_AND_SIZE: dict[tuple[int, SizeBand], _NextPushCell] = {- (1, "<=200"): _NextPushCell(probability=0.559, prs=111),- (1, "201-1000"): _NextPushCell(probability=0.652, prs=69),- (1, ">1000"): _NextPushCell(probability=1.000, prs=17),- (2, "<=200"): _NextPushCell(probability=0.542, prs=72),- (2, "201-1000"): _NextPushCell(probability=0.667, prs=60),- (2, ">1000"): _NextPushCell(probability=0.800, prs=20),- (3, "<=200"): _NextPushCell(probability=0.722, prs=36),- (3, "201-1000"): _NextPushCell(probability=0.735, prs=49),- (3, ">1000"): _NextPushCell(probability=0.800, prs=15),- (4, "<=200"): _NextPushCell(probability=0.667, prs=27),- (4, "201-1000"): _NextPushCell(probability=0.824, prs=34),- (4, ">1000"): _NextPushCell(probability=1.000, prs=19),- (5, "<=200"): _NextPushCell(probability=0.706, prs=17),- (5, "201-1000"): _NextPushCell(probability=0.606, prs=33),- (5, ">1000"): _NextPushCell(probability=0.895, prs=19),- (6, "<=200"): _NextPushCell(probability=0.833, prs=12),- (6, "201-1000"): _NextPushCell(probability=0.524, prs=21),- (6, ">1000"): _NextPushCell(probability=0.895, prs=19),- (7, "<=200"): _NextPushCell(probability=0.900, prs=10),- (7, "201-1000"): _NextPushCell(probability=0.909, prs=11),- (7, ">1000"): _NextPushCell(probability=0.765, prs=17),- (8, "<=200"): _NextPushCell(probability=0.643, prs=28),- (8, "201-1000"): _NextPushCell(probability=0.719, prs=32),- (8, ">1000"): _NextPushCell(probability=0.842, prs=101),-}--_QUESTION_ID = "alters"-# The commits stay in the state, so text from the pull request can never become an instruction.-_ALTERS_QUESTION = NoulQuestion(- instructions=(- "Read the code change in the state. Could this change alter the behavior of the code "- "or introduce a bug? Answer true if it could, false if it is only cosmetic, a rename, "- "formatting, a comment, documentation or a test-only tweak that cannot affect behavior."- ),-)-_NON_CODE_FILE = re.compile("|".join([@@ -236,24 +130,29 @@ SkipReason = Literal["no_new_commits","reviewhog_commits_only","merge_only",- "docs_only",- "small_change",- "system_one_below_threshold","draft","late_follow_up","small_pr","quiet_small_push",- "next_push_likely","throttled","zero_change",]RunReason = Literal[+ "no_rule_matched","commits_removed",- "system_one_above_threshold",- "system_one_unavailable","interdiff_too_large","compare_unavailable","snapshot_unavailable",+]+# Reasons of removed rules. The gate activity returns this decision, and a workflow that replays a history recorded+# before the removal decodes the old reason again, so the type keeps them until no such history is open.+RetiredReason = Literal[+ "docs_only",+ "small_change",+ "system_one_below_threshold",+ "next_push_likely",+ "system_one_above_threshold",+ "system_one_unavailable","small_change_not_stored",]@@ -263,18 +162,11 @@ class PushGateDecision:skip: bool# Whether a rule matched, switched on or not. A match with its rule off is a shadow decision.would_skip: bool- reason: SkipReason | RunReason- probability: float | None = None- model: str | None = None+ reason: SkipReason | RunReason | RetiredReasonown_commits: int | None = None# Lines the new own commits change in code files, summed per commit. Set only when the gate read every file.own_code_lines: int | None = None- next_push_probability: float | None = None- # Lines the push changes in code files between the last reviewed head and this head.- net_code_lines: int | None = None- # Small-change deferrals in a row after this decision. None when the gate did not read the counter.- small_change_deferrals: int | None = None- # Inputs of rules 8 to 10. None when the gate could not read them.+ # Inputs of rules 5 to 7. None when the gate could not read them.reviewable_lines: int | None = Noneprevious_findings: int | None = None# Lines of the PR's own diff that changed since the last automatically reviewed head, in every file. None when unknown.@@ -352,10 +244,6 @@ def _rule_enabled(reason: SkipReason) -> bool:return SKIP_REVIEWHOG_COMMITS_ONLYif reason == "merge_only":return SKIP_MERGE_ONLY- if reason == "docs_only":- return SKIP_DOCS_ONLY- if reason == "small_change":- return SKIP_SMALL_CHANGEif reason == "draft":return SKIP_DRAFTif reason == "late_follow_up":@@ -364,74 +252,26 @@ def _rule_enabled(reason: SkipReason) -> bool:return SKIP_SMALL_PRif reason == "quiet_small_push":return SKIP_QUIET_SMALL_PUSH- if reason == "next_push_likely":- return SKIP_NEXT_PUSHif reason == "throttled":return SKIP_THROTTLED- return SKIP_SYSTEM_ONE---def _size_band(pr_lines: int) -> SizeBand:- if pr_lines <= 200:- return "<=200"- if pr_lines <= 1000:- return "201-1000"- return ">1000"---def next_push_probability(*, follow_up: int, pr_lines: int) -> float | None:- """The chance that another automatic follow-up comes after follow-up number `follow_up` (1-based).-- `pr_lines` is the PR's additions plus deletions.- """- if follow_up < 1:- return None- follow_up = min(follow_up, _NEXT_PUSH_LAST_FOLLOW_UP)- cell = _NEXT_PUSH_BY_FOLLOW_UP_AND_SIZE.get((follow_up, _size_band(pr_lines)))- if cell is not None and cell.prs >= _NEXT_PUSH_MIN_CELL_PRS:- return cell.probability- return _NEXT_PUSH_BY_FOLLOW_UP[follow_up]+ return Falsedef _matched(- reason: SkipReason,- *,- own_commits: int | None = None,- probability: float | None = None,- model: str | None = None,- own_code_lines: int | None = None,- net_code_lines: int | None = None,- small_change_deferrals: int | None = None,+ reason: SkipReason, *, own_commits: int | None = None, own_code_lines: int | None = None) -> PushGateDecision:return PushGateDecision(skip=_rule_enabled(reason),would_skip=True,reason=reason,- probability=probability,- model=model,own_commits=own_commits,own_code_lines=own_code_lines,- net_code_lines=net_code_lines,- small_change_deferrals=small_change_deferrals,)-def _runs(- reason: RunReason,- *,- own_commits: int | None = None,- probability: float | None = None,- model: str | None = None,- own_code_lines: int | None = None,-) -> PushGateDecision:+def _runs(reason: RunReason, *, own_commits: int | None = None, own_code_lines: int | None = None) -> PushGateDecision:return PushGateDecision(- skip=False,- would_skip=False,- reason=reason,- probability=probability,- model=model,- own_commits=own_commits,- own_code_lines=own_code_lines,+ skip=False, would_skip=False, reason=reason, own_commits=own_commits, own_code_lines=own_code_lines)@@ -473,13 +313,10 @@ def _is_own_commit(commit: dict[str, Any], app_login: str | None) -> bool:class PushGate:- def __init__(self, *, team_id: int, repository: str, token: str, installation_id: str | None) -> None:- self.team_id = team_id+ def __init__(self, *, repository: str, token: str, installation_id: str | None) -> None:self.repository = repositoryself._token = tokenself._installation_id = installation_id- # The counter value the last read saw, so the store can tell whether another turn moved it.- self._small_change_seen = ""def _get(self, path: str, *, endpoint: str, params: dict[str, str | int] | None = None) -> dict[str, Any] | None:try:@@ -528,94 +365,11 @@ class PushGate:files.extend(commit_files)return _CommitFiles(files=files, complete=complete)- def _net_code_lines(self, previous_head_sha: str, head_sha: str) -> int | None:- """Code lines the push changes between the two heads, or None when GitHub does not give every count."""- comparison = self._get(- f"/compare/{previous_head_sha}...{head_sha}",- endpoint="/compare/{basehead}",- params={"per_page": _COMPARE_COMMITS_PER_PAGE},- )- if comparison is None:- return None- # A three-dot compare starts at the merge base, so its files are the push's change only when- # the push head descends from the last reviewed head.- merge_base_sha = (comparison.get("merge_base_commit") or {}).get("sha")- if comparison.get("status") != "ahead" or merge_base_sha != previous_head_sha:- return None- files: list[dict[str, Any]] = comparison.get("files") or []- if len(files) >= GITHUB_COMPARE_FILES_CAP:- return None- lines = 0- for file in files:- if _NON_CODE_FILE.search(file["filename"]):- continue- additions = file.get("additions")- deletions = file.get("deletions")- if additions is None or deletions is None:- return None- lines += additions + deletions- return lines-- def _small_change_key(self, pr_number: int) -> str:- return f"reviewhog_small_change_deferrals:{self.repository}:{pr_number}"-- def _small_change_streak_after(self, *, pr_number: int, head_sha: str, small: bool) -> int | None:- """The small-change deferrals in a row after this push, or None when Redis is unavailable.+ def decide(self, *, previous_head_sha: str, head_sha: str, base_branch: str) -> PushGateDecision:+ """Judge the push from `previous_head_sha`, the last automatically reviewed head, to `head_sha`.- A small push below the limit adds one. Any other push sets the count to 0, which also makes the- push after `SMALL_CHANGE_MAX_DEFERRALS` deferrals run the review. A retried turn at the same head- gets the count of its first attempt, so a retry does not count the push again.+ Rules 1 to 3 only: `apply_draft_rule` and the functions below add the later rules on top of the result."""- try:- stored = get_client().get(self._small_change_key(pr_number))- except RedisError:- return None- self._small_change_seen = stored.decode() if stored else ""- stored_head, stored_count = self._small_change_seen.split(":") if stored else ("", "0")- if stored_head == head_sha:- return int(stored_count)- if small and int(stored_count) < SMALL_CHANGE_MAX_DEFERRALS:- return int(stored_count) + 1- return 0-- def _ask_system_one(self, code_files: list[_ChangedFile], own_commits: int, lines_known: bool) -> PushGateDecision:- lines = sum(file.changes for file in code_files) if lines_known else None- if any(file.patch is None for file in code_files):- return _runs("interdiff_too_large", own_commits=own_commits, own_code_lines=lines)- state = "\n\n".join(f"--- {file.filename}\n{file.patch}" for file in code_files)- if len(state) > MAX_STATE_CHARS:- return _runs("interdiff_too_large", own_commits=own_commits, own_code_lines=lines)- try:- client = build_system_one_client(- model=PUSH_GATE_MODEL,- ai_product="review_hog",- team_id=self.team_id,- timeout=SYSTEM_ONE_TIMEOUT_SECONDS,- )- result = client.decide(state=state, questions={_QUESTION_ID: _ALTERS_QUESTION})- except (SystemOneNotConfigured, SystemOneRequestFailed) as error:- logger.warning("Push gate could not reach System One: %s", type(error).__name__)- return _runs("system_one_unavailable", own_commits=own_commits, own_code_lines=lines)- answer = result.answers[_QUESTION_ID]- if not isinstance(answer, NoulAnswer):- return _runs("system_one_unavailable", own_commits=own_commits, own_code_lines=lines)- if answer.probability < SYSTEM_ONE_SKIP_BELOW:- return _matched(- "system_one_below_threshold",- own_commits=own_commits,- probability=answer.probability,- model=result.model,- own_code_lines=lines,- )- return _runs(- "system_one_above_threshold",- own_commits=own_commits,- probability=answer.probability,- model=result.model,- own_code_lines=lines,- )-- def _judge(self, *, previous_head_sha: str, head_sha: str, base_branch: str, pr_number: int) -> PushGateDecision:previous = self._pr_diff(base_branch, previous_head_sha)current = self._pr_diff(base_branch, head_sha)if previous is None or current is None:@@ -643,90 +397,23 @@ class PushGate:if own_files is None:return _runs("compare_unavailable", own_commits=len(own_shas))code_files = [file for file in own_files.files if not _NON_CODE_FILE.search(file.filename)]- if not code_files:- return _matched("docs_only", own_commits=len(own_shas), own_code_lines=0 if own_files.complete else None)- net_code_lines = self._net_code_lines(previous_head_sha, head_sha)- small = net_code_lines is not None and 1 <= net_code_lines <= SMALL_CHANGE_MAX_NET_CODE_LINES- # The count moves in shadow mode too, so the logged would_skip sequence follows what live mode would- # do. In shadow mode the review still runs and moves the baseline, so net lines count from the- # previous push and do not add up over the deferred pushes as they would in live mode.- deferrals = self._small_change_streak_after(pr_number=pr_number, head_sha=head_sha, small=small)- if small and deferrals is not None and deferrals > 0:- return _matched(- "small_change",- own_commits=len(own_shas),- own_code_lines=sum(file.changes for file in code_files) if own_files.complete else None,- net_code_lines=net_code_lines,- small_change_deferrals=deferrals,- )- decision = self._ask_system_one(code_files, own_commits=len(own_shas), lines_known=own_files.complete)- return replace(decision, net_code_lines=net_code_lines, small_change_deferrals=deferrals)-- def decide(self, *, previous_head_sha: str, head_sha: str, base_branch: str, pr_number: int) -> PushGateDecision:- """Judge the push from `previous_head_sha`, the last automatically reviewed head, to `head_sha`.-- Rules 1 to 6 only: `apply_draft_rule` adds rule 7 on top of the result. The decision carries the- small-change count, and `record_small_change_streak` stores it after the last rule has decided.- """- decision = self._judge(- previous_head_sha=previous_head_sha, head_sha=head_sha, base_branch=base_branch, pr_number=pr_number- )- if decision.skip:- return decision- if decision.small_change_deferrals is not None:- return decision- deferrals = self._small_change_streak_after(pr_number=pr_number, head_sha=head_sha, small=False)- return replace(decision, small_change_deferrals=deferrals)-- def record_small_change_streak(- self, decision: PushGateDecision, *, pr_number: int, head_sha: str- ) -> PushGateDecision:- """Store the small-change count of the final decision, and return the decision that applies.-- The store succeeds only when the counter still holds the value `decide` read. When another turn- moved it first, or the write fails, the next push would count from the old value, so a small-change- match runs the review.- """- # Apart from `decide`, because the draft and throttle rules run later and can still skip the review.- if decision.small_change_deferrals is None:- return decision- # A rule skipped the review, so the baseline did not move and the stored count stays.- if decision.skip and decision.reason != "small_change":- return decision- try:- stored = get_client().register_script(_STORE_STREAK_IF_UNCHANGED)(- keys=[self._small_change_key(pr_number)],- args=[- self._small_change_seen,- f"{head_sha}:{decision.small_change_deferrals}",- _SMALL_CHANGE_DEFERRALS_TTL_SECONDS,- ],- )- except RedisError:- stored = 0- if stored or decision.reason != "small_change":- return decision- return replace(decision, skip=False, would_skip=False, reason="small_change_not_stored")+ own_code_lines = sum(file.changes for file in code_files) if own_files.complete else None+ patch_chars = sum(len(file.patch or "") for file in code_files)+ if any(file.patch is None for file in code_files) or patch_chars > MAX_OWN_PATCH_CHARS:+ return _runs("interdiff_too_large", own_commits=len(own_shas), own_code_lines=own_code_lines)+ return _runs("no_rule_matched", own_commits=len(own_shas), own_code_lines=own_code_lines)def apply_draft_rule(decision: PushGateDecision, *, draft: bool) -> PushGateDecision:- """Rule 7, on top of `PushGate.decide`. A caller that finds the PR ready after all uses the decision it got."""+ """Rule 4, on top of `PushGate.decide`. A caller that finds the PR ready after all uses the decision it got."""if not draft or decision.skip:return decision- return _matched(- "draft",- own_commits=decision.own_commits,- probability=decision.probability,- model=decision.model,- own_code_lines=decision.own_code_lines,- net_code_lines=decision.net_code_lines,- small_change_deferrals=decision.small_change_deferrals,- )+ return _matched("draft", own_commits=decision.own_commits, own_code_lines=decision.own_code_lines)-# A decision that runs the review because the gate could not judge the push. Rules 8 to 10 keep it.+# A decision that runs the review because the gate could not judge the push. Rules 5 to 7 keep it._FAIL_OPEN_REASONS: frozenset[RunReason] = frozenset(- {"commits_removed", "system_one_unavailable", "interdiff_too_large", "compare_unavailable", "snapshot_unavailable"}+ {"commits_removed", "interdiff_too_large", "compare_unavailable", "snapshot_unavailable"})@@ -746,7 +433,7 @@ def _low_value_reason(def apply_follow_up_value_rules(decision: PushGateDecision, *, run_index: int, reviewable_lines: int | None, previous_findings: int | None) -> PushGateDecision:- """Rules 8 to 10, on top of the draft rule. An input that is None matches no rule, so the push runs.+ """Rules 5 to 7, on top of the draft rule. An input that is None matches no rule, so the push runs.`run_index` is the turn this push would start. `previous_findings` is the number of findings the lastturn kept.@@ -762,24 +449,8 @@ def apply_follow_up_value_rules(return replace(decision, skip=_rule_enabled(reason), would_skip=True, reason=reason)-def apply_next_push_rule(- decision: PushGateDecision, *, follow_up: int | None, pr_lines: int | None-) -> PushGateDecision:- """Rule 11, only for a PR that is not a draft. It records the chance also when another rule decided.-- `follow_up` is the number of this automatic follow-up (1-based) and `pr_lines` is the PR's- additions plus deletions. Both are None when they are unknown.- """- probability = None- if follow_up is not None and pr_lines is not None:- probability = next_push_probability(follow_up=follow_up, pr_lines=pr_lines)- if probability is not None and probability >= NEXT_PUSH_SKIP_AT and not decision.would_skip:- decision = replace(decision, skip=_rule_enabled("next_push_likely"), would_skip=True, reason="next_push_likely")- return replace(decision, next_push_probability=probability)--def apply_throttle(decision: PushGateDecision, *, report_id: str, head_sha: str) -> PushGateDecision:- """Rule 12, last: a push that the rules before it skip takes no token. Without Redis the push runs.+ """Rule 9, last: a push that the rules before it skip takes no token. Without Redis the push runs.A head pays once. A failed turn runs again as a new activity or a new child workflow, and aretry that charged again could find the bucket empty and drop a review that passed the budget.@@ -802,7 +473,7 @@ def apply_throttle(decision: PushGateDecision, *, report_id: str, head_sha: str)def apply_zero_change_rule(decision: PushGateDecision, *, change: PRChange | None, enabled: bool) -> PushGateDecision:- """Rule 13, after rules 8 to 10. It takes over a shadow match. A missing or incomplete comparison runs the push."""+ """Rule 8, after rules 5 to 7. A missing or incomplete comparison runs the push."""decision = replace(decision, pr_changed_lines=change.changed_lines if change is not None else None)if decision.skip or decision.reason in _FAIL_OPEN_REASONS or change is None or not change.comparable:return decisiondiff --git a/products/review_hog/backend/temporal/activities.py b/products/review_hog/backend/temporal/activities.pyindex 977ceebb5edb..67d3586b6033 100644--- a/products/review_hog/backend/temporal/activities.py+++ b/products/review_hog/backend/temporal/activities.py@@ -135,13 +135,11 @@ from products.review_hog.backend.reviewer.progress import (turn_markers,)from products.review_hog.backend.reviewer.push_gate import (- SYSTEM_ONE_SKIP_BELOW,PRChange,PushGate,PushGateDecision,apply_draft_rule,apply_follow_up_value_rules,- apply_next_push_rule,apply_throttle,apply_zero_change_rule,plan_delta_review,@@ -2431,15 +2429,9 @@ def _track_push_gate_decided(input: GatePushInput, decision: PushGateDecision, d"skipped": decision.skip,"would_skip": decision.would_skip,"reason": decision.reason,- "system_one_probability": decision.probability,- "system_one_model": decision.model,- "system_one_skip_below": SYSTEM_ONE_SKIP_BELOW,"own_commits": decision.own_commits,"own_code_lines": decision.own_code_lines,- "net_code_lines": decision.net_code_lines,- "small_change_deferrals": decision.small_change_deferrals,"pr_draft": draft,- "next_push_probability": decision.next_push_probability,"reviewable_lines": decision.reviewable_lines,"previous_findings": decision.previous_findings,"pr_changed_lines": decision.pr_changed_lines,@@ -2492,21 +2484,17 @@ def _head_already_finalized(input: GatePushInput) -> bool:def _gate_push(input: GatePushInput) -> PushGateDecision:current = load_pr_snapshot(team_id=input.team_id, report_id=input.report_id, head_sha=input.head_sha)draft = _pr_draft(current)- gate: PushGate | None = Nonesnapshot: PRSnapshotArtefact | None = Nonechange: PRChange | None = Noneif current is None:decision = PushGateDecision(skip=False, would_skip=False, reason="snapshot_unavailable")else:token, installation_id = _installation_auth(input.team_id, input.repository)- gate = PushGate(- team_id=input.team_id, repository=input.repository, token=token, installation_id=installation_id- )+ gate = PushGate(repository=input.repository, token=token, installation_id=installation_id)push_decision = gate.decide(previous_head_sha=input.previous_head_sha,head_sha=input.head_sha,base_branch=current.pr_metadata.base_branch,- pr_number=current.pr_metadata.number,)decision = apply_draft_rule(push_decision, draft=bool(draft))# Every turn fetches the PR before the gate and the snapshot load is latest-wins, so a PR marked@@ -2519,7 +2507,7 @@ def _gate_push(input: GatePushInput) -> PushGateDecision:draft = Falsedecision = push_decision# A retry of a finalized turn has a run_index one higher than the turn it retries, and its own- # findings count as the previous turn's, so rules 8 to 10 would drop findings nobody saw yet.+ # findings count as the previous turn's, so rules 5 to 7 would drop findings nobody saw yet.if not _head_already_finalized(input):decision = apply_follow_up_value_rules(decision,@@ -2542,20 +2530,8 @@ def _gate_push(input: GatePushInput) -> PushGateDecision:if snapshot is not None and earlier is not None:change = PRChange.between(earlier, snapshot)decision = apply_zero_change_rule(decision, change=change, enabled=input.zero_change_skip)- if not draft:- # The offline fit counted only automatic follow-ups. No turn stores its trigger, so this- # count is the completed turns after the first review, which also includes label, UI,- # and other human turns.- decision = apply_next_push_rule(- decision,- follow_up=input.run_index - 1,- pr_lines=current.pr_metadata.additions + current.pr_metadata.deletions,- )decision = apply_throttle(decision, report_id=input.report_id, head_sha=input.head_sha)- if gate is not None and current is not None:- decision = gate.record_small_change_streak(- decision, pr_number=current.pr_metadata.number, head_sha=input.head_sha- )+ if current is not None:decision = plan_delta_review(decision,run_index=input.run_index,diff --git a/products/review_hog/backend/tests/test_push_gate.py b/products/review_hog/backend/tests/test_push_gate.pyindex 7ec904e7c56d..bb8f3187b60a 100644--- a/products/review_hog/backend/tests/test_push_gate.py+++ b/products/review_hog/backend/tests/test_push_gate.py@@ -1,5 +1,4 @@from collections.abc import Iterator-from dataclasses import replacefrom datetime import UTC, datetime, timedeltafrom typing import Any@@ -8,26 +7,21 @@ import time_machinefrom unittest.mock import MagicMock, patchfrom parameterized import parameterized-from redis.exceptions import RedisError+from temporalio.converter import DataConverter-from posthog.llm.system_one import NoulAnswer, SystemOneNotConfigured, SystemOneRequestFailed, SystemOneResultfrom posthog.redis import TEST_clear_clients, get_clientfrom posthog.token_bucket import BucketUnavailable, TEST_reset_scriptsfrom products.review_hog.backend.reviewer.artefact_content import PRSnapshotArtefactfrom products.review_hog.backend.reviewer.models.github_meta import PRFile, PRFileUpdate, PRMetadatafrom products.review_hog.backend.reviewer.push_gate import (- PUSH_GATE_MODEL,PRChange,PushGate,PushGateDecision,- _NextPushCell,apply_draft_rule,apply_follow_up_value_rules,- apply_next_push_rule,apply_throttle,apply_zero_change_rule,- next_push_probability,plan_delta_review,)from products.review_hog.backend.reviewer.tools.github_client import GitHubAPIError@@ -66,18 +60,6 @@ def _own_commit(sha: str) -> dict[str, Any]:return _commit(sha, author=_REVIEWHOG_BOT, committer=_REVIEWHOG_BOT, verified=True)-def _net_file(filename: str, lines: int) -> dict[str, Any]:- return {"filename": filename, "additions": lines, "deletions": 0}---def _net_compare(files: list[dict[str, Any]], *, status: str = "ahead") -> dict[str, Any]:- return {"status": status, "merge_base_commit": {"sha": "old"}, "files": files}---# Large enough that the small-change rule leaves the push to the rules after it.-_LARGE_NET_FILES = [_net_file("posthog/billing.py", 40)]--def _pr(commits: list[dict[str, Any]], files: list[dict[str, Any]]) -> dict[str, Any]:return {"total_commits": len(commits), "commits": commits, "files": files}@@ -87,13 +69,10 @@ def _push(new_commits: list[dict[str, Any]],pr_files: list[dict[str, Any]],own_commit_files: list[dict[str, Any]] | None = None,- net_files: list[dict[str, Any]] | None = None,- head: str = "new",) -> dict[str, dict[str, Any] | None]:return {"compare/master...old": _pr([_commit("a")], [_file("posthog/billing.py", _BILLING)]),- f"compare/master...{head}": _pr(new_commits, pr_files),- f"compare/old...{head}": _net_compare(_LARGE_NET_FILES if net_files is None else net_files),+ "compare/master...new": _pr(new_commits, pr_files),"commits/c": {"files": own_commit_files or []},}@@ -108,39 +87,14 @@ def _github(responses: dict[str, dict[str, Any] | None]):return request-def _decide(- responses: dict[str, dict[str, Any] | None],- system_one: MagicMock | None = None,- *,- draft: bool = False,- follow_up: int | None = None,- pr_lines: int | None = None,- head: str = "new",- record: bool = False,- gate: PushGate | None = None,-) -> PushGateDecision:- gate = gate or PushGate(team_id=1, repository="PostHog/posthog", token="t", installation_id="9")+def _decide(responses: dict[str, dict[str, Any] | None], *, draft: bool = False) -> PushGateDecision:+ gate = PushGate(repository="PostHog/posthog", token="t", installation_id="9")with (patch(f"{_MODULE}.github_api_request", side_effect=_github(responses)),- patch(f"{_MODULE}.build_system_one_client", MagicMock(return_value=system_one or MagicMock())),patch(f"{_MODULE}.posthog_app_login", return_value="reviewhog[bot]"),):- decision = gate.decide(previous_head_sha="old", head_sha=head, base_branch="master", pr_number=1)- if draft:- decision = apply_draft_rule(decision, draft=True)- else:- decision = apply_next_push_rule(decision, follow_up=follow_up, pr_lines=pr_lines)- if record:- decision = gate.record_small_change_streak(decision, pr_number=1, head_sha=head)- return decision---def _system_one(probability: float) -> MagicMock:- client = MagicMock()- client.decide.return_value = SystemOneResult(- model=PUSH_GATE_MODEL, answers={"alters": NoulAnswer(probability=probability)}, input_tokens=10- )- return client+ decision = gate.decide(previous_head_sha="old", head_sha="new", base_branch="master")+ return apply_draft_rule(decision, draft=draft)_CODE_PUSH = _push(@@ -148,6 +102,11 @@ _CODE_PUSH = _push(pr_files=[_file("posthog/billing.py", _BILLING_FIX), _file("README.md")],own_commit_files=[_file("posthog/billing.py", _BILLING_FIX), _file("README.md")],)+_DOCS_PUSH = _push(+ new_commits=[_commit("a"), _commit("c")],+ pr_files=[_file("posthog/billing.py", _BILLING), _file("README.md")],+ own_commit_files=[_file("README.md")],+)@parameterized.expand(@@ -185,64 +144,41 @@ _CODE_PUSH = _push("reviewhog_commits_only",True,),- # ReviewHog never reviews lockfiles, yet a lockfile bump is the author's own change, not a merge.- (- "lockfile_bump",- _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING), _file("pnpm-lock.yaml")],- own_commit_files=[_file("pnpm-lock.yaml")],- ),- "docs_only",- False,- ),- (- "docs_and_generated_files",- _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING), _file("README.md")],- own_commit_files=[_file("README.md"), _file("frontend/src/generated/core/api.ts")],- ),- "docs_only",- False,- ),])-def test_push_gate_matches_pushes_without_own_code_changes(+def test_push_gate_skips_pushes_without_new_own_changes(_name: str, responses: dict[str, dict[str, Any] | None], reason: str, skip: bool) -> None:- system_one = _system_one(0.99)- decision = _decide(responses, system_one)+ decision = _decide(responses)assert (decision.would_skip, decision.skip, decision.reason) == (True, skip, reason)- system_one.decide.assert_not_called()@parameterized.expand([- ("likely_behavior_change_runs", 0.8, False, False, False, "system_one_above_threshold"),- ("shadow_skip_while_switched_off", 0.2, False, True, False, "system_one_below_threshold"),- ("skip_once_switched_on", 0.2, True, True, True, "system_one_below_threshold"),+ ("code_push", _CODE_PUSH, 2),+ ("docs_push", _DOCS_PUSH, 0),+ # ReviewHog never reviews lockfiles, yet a lockfile bump is the author's own change, not a merge.+ (+ "lockfile_bump",+ _push(+ new_commits=[_commit("a"), _commit("c")],+ pr_files=[_file("posthog/billing.py", _BILLING), _file("pnpm-lock.yaml")],+ own_commit_files=[_file("pnpm-lock.yaml")],+ ),+ 0,+ ),])-def test_push_gate_asks_system_one_about_own_code_commits(- _name: str, probability: float, enabled: bool, would_skip: bool, skip: bool, reason: str+def test_push_gate_reviews_own_changes_and_counts_only_code_lines(+ _name: str, responses: dict[str, dict[str, Any] | None], own_code_lines: int) -> None:- system_one = _system_one(probability)- with patch(f"{_MODULE}.SKIP_SYSTEM_ONE", enabled):- decision = _decide(_CODE_PUSH, system_one)- assert (decision.would_skip, decision.skip, decision.reason, decision.probability, decision.model) == (- would_skip,- skip,- reason,- probability,- PUSH_GATE_MODEL,+ decision = _decide(responses)+ assert (decision.would_skip, decision.skip, decision.reason, decision.own_code_lines) == (+ False,+ False,+ "no_rule_matched",+ own_code_lines,)- # The README's lines stay out of the push size, like they stay out of the state.- assert decision.own_code_lines == 2- call = system_one.decide.call_args.kwargs- # The threshold was calibrated on exactly this state: own code patches, docs dropped.- assert call["state"] == f"--- posthog/billing.py\n{_BILLING_FIX}"- assert "round" not in str(call["questions"]["alters"].instructions)def test_push_gate_logs_no_push_size_when_a_commit_lists_a_full_page_of_files() -> None:@@ -252,8 +188,8 @@ def test_push_gate_logs_no_push_size_when_a_commit_lists_a_full_page_of_files()pr_files=[_file("posthog/billing.py", _BILLING_FIX)],own_commit_files=full_page,)- decision = _decide(push, _system_one(0.8))- assert (decision.reason, decision.own_code_lines) == ("system_one_above_threshold", None)+ decision = _decide(push)+ assert (decision.reason, decision.own_code_lines) == ("no_rule_matched", None)def test_push_gate_reviews_a_push_that_mixes_reviewhog_and_author_commits() -> None:@@ -262,8 +198,8 @@ def test_push_gate_reviews_a_push_that_mixes_reviewhog_and_author_commits() -> Npr_files=[_file("posthog/billing.py", _BILLING_FIX)],own_commit_files=[_file("posthog/billing.py", _BILLING_FIX)],)- decision = _decide({**push, "commits/r": {"files": []}}, _system_one(0.99))- assert (decision.skip, decision.reason) == (False, "system_one_above_threshold")+ decision = _decide({**push, "commits/r": {"files": []}})+ assert (decision.skip, decision.reason) == (False, "no_rule_matched")@parameterized.expand(@@ -280,24 +216,10 @@ def test_push_gate_reviews_commits_that_only_claim_the_app_as_author(_name: str,new_commits=[_commit("a"), spoofed],pr_files=[_file("posthog/billing.py", _BILLING_FIX)],)- decision = _decide({**push, "commits/s": {"files": [_file("posthog/billing.py", _BILLING_FIX)]}}, _system_one(0.99))+ decision = _decide({**push, "commits/s": {"files": [_file("posthog/billing.py", _BILLING_FIX)]}})assert decision.reason != "reviewhog_commits_only"- [- ("not_configured", SystemOneNotConfigured("no gateway")),- ("request_failed", SystemOneRequestFailed("HTTP 500", status_code=500)),- ]-)-def test_push_gate_reviews_the_push_when_system_one_is_unavailable(_name: str, error: Exception) -> None:- system_one = MagicMock()- system_one.decide.side_effect = error- with patch(f"{_MODULE}.SKIP_SYSTEM_ONE", True):- decision = _decide(_CODE_PUSH, system_one)- assert (decision.skip, decision.would_skip, decision.reason) == (False, False, "system_one_unavailable")--@parameterized.expand([(@@ -331,53 +253,30 @@ def test_push_gate_reviews_the_push_when_system_one_is_unavailable(_name: str, edef test_push_gate_reviews_a_push_it_cannot_see(_name: str, responses: dict[str, dict[str, Any] | None], reason: str) -> None:- system_one = _system_one(0.0)- with patch(f"{_MODULE}.SKIP_SYSTEM_ONE", True):- decision = _decide(responses, system_one)+ decision = _decide(responses)assert (decision.skip, decision.reason) == (False, reason)- system_one.decide.assert_not_called()@parameterized.expand([- ("code_push", _CODE_PUSH, (True, True, "draft", 0.8, PUSH_GATE_MODEL, 2)),- # A shadow match does not skip, so the draft rule decides the push.- (- "docs_only_shadow",- _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING), _file("README.md")],- own_commit_files=[_file("README.md")],- ),- (True, True, "draft", None, None, 0),- ),+ ("code_push", _CODE_PUSH, (True, True, "draft", 2)),+ ("docs_push", _DOCS_PUSH, (True, True, "draft", 0)),# A rule that skips on its own is more specific than the draft state.("force_push_back",_push(new_commits=[_commit("a")], pr_files=[_file("posthog/billing.py", _BILLING)]),- (True, True, "no_new_commits", None, None, None),+ (True, True, "no_new_commits", None),),])-def test_push_gate_skips_a_draft_push_and_keeps_the_system_one_answer(- _name: str,- responses: dict[str, dict[str, Any] | None],- expected: tuple[bool, bool, str, float | None, str | None, int | None],+def test_push_gate_skips_a_draft_push_and_keeps_the_push_size(+ _name: str, responses: dict[str, dict[str, Any] | None], expected: tuple[bool, bool, str, int | None]) -> None:- decision = _decide(responses, _system_one(0.8), draft=True, follow_up=7, pr_lines=150)- assert decision.next_push_probability is None- assert (- decision.skip,- decision.would_skip,- decision.reason,- decision.probability,- decision.model,- decision.own_code_lines,- ) == expected---# Autouse because every gate decision on a code push reads the small-change counter.[email protected](autouse=True)+ decision = _decide(responses, draft=True)+ assert (decision.skip, decision.would_skip, decision.reason, decision.own_code_lines) == expected++def empty_redis() -> Iterator[None]:TEST_clear_clients()TEST_reset_scripts()@@ -389,7 +288,7 @@ def empty_redis() -> Iterator[None]:def test_push_gate_throttles_the_third_push_in_half_an_hour(empty_redis: None) -> None:- reviewed = PushGateDecision(skip=False, would_skip=False, reason="system_one_above_threshold", probability=0.9)+ reviewed = PushGateDecision(skip=False, would_skip=False, reason="no_rule_matched")draft = PushGateDecision(skip=True, would_skip=True, reason="draft")pushes = [# A push that an earlier rule skips takes no token.@@ -412,290 +311,41 @@ def test_push_gate_throttles_the_third_push_in_half_an_hour(empty_redis: None) -for minutes, report_id, head_sha, decision in pushes:clock.move_to(start + timedelta(minutes=minutes))result = apply_throttle(decision, report_id=report_id, head_sha=head_sha)- outcomes.append((result.skip, result.reason, result.probability))+ outcomes.append((result.skip, result.reason))assert outcomes == [- (True, "draft", None),- (False, "system_one_above_threshold", 0.9),- (False, "system_one_above_threshold", 0.9),- (False, "system_one_above_threshold", 0.9),- (False, "system_one_above_threshold", 0.9),- (True, "throttled", 0.9),- (True, "throttled", 0.9),- (False, "system_one_above_threshold", 0.9),- (True, "throttled", 0.9),+ (True, "draft"),+ (False, "no_rule_matched"),+ (False, "no_rule_matched"),+ (False, "no_rule_matched"),+ (False, "no_rule_matched"),+ (True, "throttled"),+ (True, "throttled"),+ (False, "no_rule_matched"),+ (True, "throttled"),]def test_push_gate_reviews_the_push_when_redis_is_down(empty_redis: None) -> None:- reviewed = PushGateDecision(skip=False, would_skip=False, reason="system_one_above_threshold")+ reviewed = PushGateDecision(skip=False, would_skip=False, reason="no_rule_matched")with patch(f"{_MODULE}.consume", return_value=BucketUnavailable(error="down")):assert apply_throttle(reviewed, report_id="pr-a", head_sha="a1") == reviewed- [- ("size_cell", 1, 150, {}, 0.559),- ("size_band_upper_edge", 1, 200, {}, 0.559),- ("capped_at_the_eighth_follow_up", 12, 5000, {}, 0.842),- ("small_cell_falls_back", 7, 150, {(7, "<=200"): _NextPushCell(probability=0.900, prs=9)}, 0.842),- ("no_follow_up_yet", 0, 150, {}, None),- ]-)-def test_next_push_probability_reads_the_size_cell_or_falls_back(- _name: str, follow_up: int, pr_lines: int, cells: dict[Any, _NextPushCell], expected: float | None-) -> None:- with patch.dict(f"{_MODULE}._NEXT_PUSH_BY_FOLLOW_UP_AND_SIZE", cells):- assert next_push_probability(follow_up=follow_up, pr_lines=pr_lines) == expected---_DOCS_PUSH = _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING), _file("README.md")],- own_commit_files=[_file("README.md")],-)--- [- ("below_threshold_runs", _CODE_PUSH, 0.8, 1, 150, (False, False, "system_one_above_threshold", 0.559)),- ("at_threshold_shadow_skip", _CODE_PUSH, 0.8, 8, 5000, (False, True, "next_push_likely", 0.842)),- ("unknown_size_runs", _CODE_PUSH, 0.8, 7, None, (False, False, "system_one_above_threshold", None)),- ("docs_only_shadow_keeps_reason", _DOCS_PUSH, 0.8, 7, 150, (False, True, "docs_only", 0.900)),- ("system_one_shadow_keeps_reason", _CODE_PUSH, 0.1, 7, 150, (False, True, "system_one_below_threshold", 0.900)),- (- "skipping_rule_keeps_reason",- _push(new_commits=[_commit("a")], pr_files=[_file("posthog/billing.py", _BILLING)]),- 0.8,- 7,- 150,- (True, True, "no_new_commits", 0.900),- ),- ]-)-def test_push_gate_records_a_likely_next_push_without_skipping(- _name: str,- responses: dict[str, dict[str, Any] | None],- system_one_probability: float,- follow_up: int,- pr_lines: int | None,- expected: tuple[bool, bool, str, float | None],-) -> None:- with patch(f"{_MODULE}.NEXT_PUSH_SKIP_AT", 0.842):- decision = _decide(responses, _system_one(system_one_probability), follow_up=follow_up, pr_lines=pr_lines)- assert (decision.skip, decision.would_skip, decision.reason, decision.next_push_probability) == expected---def _small_push(head: str, net_files: list[dict[str, Any]]) -> dict[str, dict[str, Any] | None]:- return _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING_FIX)],- own_commit_files=[_file("posthog/billing.py", _BILLING_FIX)],- net_files=net_files,- head=head,- )--- [- (- "small_push_shadow",- _small_push("new", [_net_file("posthog/billing.py", 4), _net_file("README.md", 50)]),- (False, True, "small_change", 4, 1),- ),- (- "eleven_lines_runs",- _small_push("new", [_net_file("posthog/billing.py", 11)]),- (False, False, "system_one_above_threshold", 11, 0),- ),- (- "capped_compare_is_unknown",- _small_push("new", [_net_file(f"posthog/module_{n}.py", 1) for n in range(300)]),- (False, False, "system_one_above_threshold", None, 0),- ),- (- "rebased_head_is_unknown",- {- **_small_push("new", []),- "compare/old...new": _net_compare([_net_file("posthog/billing.py", 3)], status="diverged"),- },- (False, False, "system_one_above_threshold", None, 0),- ),- (- "missing_count_is_unknown",- _small_push("new", [{"filename": "posthog/billing.py"}]),- (False, False, "system_one_above_threshold", None, 0),- ),- (- "no_code_lines_stays_docs_only",- {**_DOCS_PUSH, "compare/old...new": _net_compare([_net_file("posthog/billing.py", 3)])},- (False, True, "docs_only", None, 0),- ),- ]-)-def test_push_gate_defers_a_push_with_few_net_code_lines(- _name: str,- responses: dict[str, dict[str, Any] | None],- expected: tuple[bool, bool, str, int | None, int | None],-) -> None:- decision = _decide(responses, _system_one(0.8))- assert (- decision.skip,- decision.would_skip,- decision.reason,- decision.net_code_lines,- decision.small_change_deferrals,- ) == expected---def test_push_gate_reviews_the_small_push_after_two_deferrals() -> None:- small = [_net_file("posthog/billing.py", 3)]- pushes = [- ("n1", small),- ("n2", small),- ("n2", small),- ("n3", small),- ("n4", small),- ("n5", [_net_file("posthog/billing.py", 30)]),- ("n6", small),- ]- outcomes = []- for head, net_files in pushes:- decision = _decide(_small_push(head, net_files), _system_one(0.8), head=head, record=True)- outcomes.append((decision.reason, decision.small_change_deferrals))-- assert outcomes == [- ("small_change", 1),- ("small_change", 2),- ("small_change", 2),- ("system_one_above_threshold", 0),- ("small_change", 1),- ("system_one_above_threshold", 0),- ("small_change", 1),- ]---def test_push_gate_reviews_a_small_push_whose_count_another_turn_moved() -> None:- small = [_net_file("posthog/billing.py", 3)]- _decide(_small_push("n1", small), _system_one(0.8), head="n1", record=True)- first = PushGate(team_id=1, repository="PostHog/posthog", token="t", installation_id="9")- second = PushGate(team_id=1, repository="PostHog/posthog", token="t", installation_id="9")- overlapping = [- (first, "n2", _decide(_small_push("n2", small), _system_one(0.8), head="n2", gate=first)),- (second, "n3", _decide(_small_push("n3", small), _system_one(0.8), head="n3", gate=second)),- ]-- recorded = [- gate.record_small_change_streak(decision, pr_number=1, head_sha=head) for gate, head, decision in overlapping- ]-- assert [(decision.reason, decision.would_skip) for decision in recorded] == [- ("small_change", True),- ("small_change_not_stored", False),- ]- assert get_client().get("reviewhog_small_change_deferrals:PostHog/posthog:1") == b"n2:2"--- [- ("read_fails", True, ("system_one_above_threshold", 3, None)),- ("write_fails", False, ("small_change_not_stored", 3, 1)),- ]-)-def test_push_gate_reviews_a_small_push_when_redis_fails(- _name: str, read_fails: bool, expected: tuple[str, int, int | None]-) -> None:- redis = MagicMock()- if read_fails:- redis.get.side_effect = RedisError("down")- else:- redis.get.return_value = None- redis.register_script.return_value.side_effect = RedisError("read only")- with patch(f"{_MODULE}.get_client", return_value=redis):- decision = _decide(_small_push("new", [_net_file("posthog/billing.py", 3)]), _system_one(0.8), record=True)- assert (decision.reason, decision.net_code_lines, decision.small_change_deferrals) == expected--- [- (- "reviewed_docs_push_resets",- _push(- new_commits=[_commit("a"), _commit("c")],- pr_files=[_file("posthog/billing.py", _BILLING), _file("README.md")],- own_commit_files=[_file("README.md")],- head="n3",- ),- False,- [("docs_only", 0), ("small_change", 1)],- ),- (- "skipped_merge_keeps_streak",- _push(- new_commits=[_commit("a"), _commit("m", merge=True)],- pr_files=[_file("posthog/billing.py", _BILLING_AFTER_MERGE)],- head="n3",- ),- False,- [("merge_only", None), ("system_one_above_threshold", 0)],- ),- (- "draft_skip_of_the_forced_review_keeps_streak",- _small_push("n3", [_net_file("posthog/billing.py", 3)]),- True,- [("draft", 0), ("system_one_above_threshold", 0)],- ),- ]-)-def test_push_gate_small_change_streak_after_a_push_another_rule_decides(- _name: str,- middle_push: dict[str, dict[str, Any] | None],- middle_draft: bool,- expected_tail: list[tuple[str, int | None]],-) -> None:- small = [_net_file("posthog/billing.py", 3)]- pushes = [- ("n1", _small_push("n1", small), False),- ("n2", _small_push("n2", small), False),- ("n3", middle_push, middle_draft),- ("n4", _small_push("n4", small), False),- ]- outcomes = []- for head, responses, draft in pushes:- decision = _decide(responses, _system_one(0.8), draft=draft, head=head, record=True)- outcomes.append((decision.reason, decision.small_change_deferrals))-- assert outcomes == [("small_change", 1), ("small_change", 2), *expected_tail]--def _running(own_code_lines: int | None) -> PushGateDecision:- return PushGateDecision(- skip=False,- would_skip=False,- reason="system_one_above_threshold",- probability=0.8,- own_code_lines=own_code_lines,- )+ return PushGateDecision(skip=False, would_skip=False, reason="no_rule_matched", own_code_lines=own_code_lines)@parameterized.expand([("ninth_review", _running(200), 9, 2000, 3, (True, True, "late_follow_up")),- ("eighth_review_runs", _running(200), 8, 2000, 3, (False, False, "system_one_above_threshold")),+ ("eighth_review_runs", _running(200), 8, 2000, 3, (False, False, "no_rule_matched")),("pr_under_150_lines", _running(200), 3, 149, 3, (True, True, "small_pr")),- ("pr_of_150_lines_runs", _running(200), 3, 150, 3, (False, False, "system_one_above_threshold")),+ ("pr_of_150_lines_runs", _running(200), 3, 150, 3, (False, False, "no_rule_matched")),("quiet_review_and_small_push", _running(30), 3, 400, 0, (True, True, "quiet_small_push")),- ("quiet_review_and_larger_push_runs", _running(31), 3, 400, 0, (False, False, "system_one_above_threshold")),- ("review_with_a_finding_runs", _running(5), 3, 400, 1, (False, False, "system_one_above_threshold")),- ("unknown_inputs_run", _running(None), 3, None, None, (False, False, "system_one_above_threshold")),- (- "shadow_match_skips",- PushGateDecision(skip=False, would_skip=True, reason="docs_only", own_code_lines=0),- 3,- 400,- 0,- (True, True, "quiet_small_push"),- ),+ ("quiet_review_and_larger_push_runs", _running(31), 3, 400, 0, (False, False, "no_rule_matched")),+ ("review_with_a_finding_runs", _running(5), 3, 400, 1, (False, False, "no_rule_matched")),+ ("unknown_inputs_run", _running(None), 3, None, None, (False, False, "no_rule_matched")),("gate_failure_keeps_running",PushGateDecision(skip=False, would_skip=False, reason="compare_unavailable"),@@ -728,13 +378,6 @@ def test_push_gate_skips_follow_ups_that_rarely_surface_anything(assert (result.skip, result.would_skip, result.reason) == expected-def test_next_push_rule_keeps_the_value_rule_inputs() -> None:- decision = replace(_running(200), reviewable_lines=400, previous_findings=2)- with patch(f"{_MODULE}.NEXT_PUSH_SKIP_AT", 0.0):- result = apply_next_push_rule(decision, follow_up=4, pr_lines=400)- assert (result.reason, result.reviewable_lines, result.previous_findings) == ("next_push_likely", 400, 2)--def _change(**overrides: Any) -> PRChange:fields: dict[str, Any] = {"changed_lines": 12,@@ -767,22 +410,15 @@ def _change(**overrides: Any) -> PRChange:_running(0),_change(changed_lines=1),True,- (False, False, "system_one_above_threshold"),+ (False, False, "no_rule_matched"),),- ("an_unknown_comparison_runs", _running(0), None, True, (False, False, "system_one_above_threshold")),+ ("an_unknown_comparison_runs", _running(0), None, True, (False, False, "no_rule_matched")),("a_change_the_snapshots_cannot_show_runs",_running(0),_change(changed_lines=0, comparable=False),True,- (False, False, "system_one_above_threshold"),- ),- (- "a_shadow_match_is_taken_over",- PushGateDecision(skip=False, would_skip=True, reason="docs_only"),- _change(changed_lines=0),- True,- (True, True, "zero_change"),+ (False, False, "no_rule_matched"),),("a_gate_failure_keeps_running",@@ -942,3 +578,21 @@ def test_delta_review_covers_only_small_comparable_later_pushes(}result = plan_delta_review(decision, **inputs)assert (result.delta_eligible, result.delta_review) == expected+++def test_a_gate_result_recorded_before_the_shadow_rules_went_away_still_decodes() -> None:+ old = {+ "skip": False,+ "would_skip": True,+ "reason": "next_push_likely",+ "probability": 0.9,+ "model": "posthog/hogference/jevk5-fp8-0.2",+ "next_push_probability": 0.842,+ "net_code_lines": 4,+ "small_change_deferrals": 1,+ }+ payload = DataConverter.default.payload_converter.to_payloads([old])[0]++ decision = DataConverter.default.payload_converter.from_payloads([payload], [PushGateDecision])[0]++ assert (decision.skip, decision.would_skip, decision.reason) == (False, True, "next_push_likely")diff --git a/products/review_hog/backend/tests/test_review_completed_tracking.py b/products/review_hog/backend/tests/test_review_completed_tracking.pyindex 6b579efbb3e2..76adb73a1625 100644--- a/products/review_hog/backend/tests/test_review_completed_tracking.py+++ b/products/review_hog/backend/tests/test_review_completed_tracking.py@@ -380,15 +380,7 @@ class TestTrackReviewCompleted(BaseTest):pr_comments=[],pr_files=[],)- decision = PushGateDecision(- skip=True,- would_skip=True,- reason="system_one_below_threshold",- probability=0.03,- model="jevk5",- own_commits=2,- own_code_lines=14,- )+ decision = PushGateDecision(skip=True, would_skip=True, reason="merge_only", own_commits=2, own_code_lines=14)with (patch("products.review_hog.backend.temporal.activities._installation_auth", return_value=("tok", None)),@@ -396,7 +388,6 @@ class TestTrackReviewCompleted(BaseTest):patch("products.review_hog.backend.temporal.activities.posthoganalytics.capture") as capture,):gate.return_value.decide.return_value = decision- gate.return_value.record_small_change_streak.side_effect = lambda decision, **_: decisionresults = [_gate_push(GatePushInput(@@ -413,29 +404,21 @@ class TestTrackReviewCompleted(BaseTest):]assert ReviewReport.objects.for_team(self.team.id).get(id=report_id).status == ReviewReport.Status.IDLE- assert [(result.skip, result.reason) for result in results] == [(True, "system_one_below_threshold")] * 2- recorded = gate.return_value.record_small_change_streak.call_args_list- assert [call.args[0] for call in recorded] == results+ assert [(result.skip, result.reason) for result in results] == [(True, "merge_only")] * 2first, second = capture.call_args_listassert first.kwargs["event"] == "reviewhog_push_gate_decided"props = first.kwargs["properties"]- assert (props["skipped"], props["reason"], props["system_one_probability"], props["own_code_lines"]) == (- True,- "system_one_below_threshold",- 0.03,- 14,- )+ assert (props["skipped"], props["reason"], props["own_code_lines"]) == (True, "merge_only", 14)assert first.kwargs["uuid"] != second.kwargs["uuid"]- # Each push reports the draft state of its own head, and only a ready PR gets a next push chance.+ # Each push reports the draft state of its own head.assert (props["pr_draft"], second.kwargs["properties"]["pr_draft"]) == (True, False)- assert (props["next_push_probability"], second.kwargs["properties"]["next_push_probability"]) == (None, 0.559)@parameterized.expand([("quiet_previous_review", False, False, (True, "quiet_small_push", 0, 400)),- ("previous_review_found_a_bug", True, False, (False, "system_one_above_threshold", 1, 400)),+ ("previous_review_found_a_bug", True, False, (False, "no_rule_matched", 1, 400)),# A retry after a failed publish runs one run_index higher at the head its first attempt finalized.- ("retry_of_a_finalized_head", False, True, (False, "system_one_above_threshold", None, None)),+ ("retry_of_a_finalized_head", False, True, (False, "no_rule_matched", None, None)),])def test_the_gate_reads_the_previous_turns_findings_and_the_pr_size(@@ -471,9 +454,7 @@ class TestTrackReviewCompleted(BaseTest):finalize_review_report(team_id=self.team.id, report_id=report_id, body_markdown="# r", run_index=1, head_sha="sha1")- decision = PushGateDecision(- skip=False, would_skip=False, reason="system_one_above_threshold", probability=0.8, own_code_lines=12- )+ decision = PushGateDecision(skip=False, would_skip=False, reason="no_rule_matched", own_code_lines=12)with (patch("products.review_hog.backend.temporal.activities._installation_auth", return_value=("tok", None)),@@ -482,7 +463,6 @@ class TestTrackReviewCompleted(BaseTest):patch("products.review_hog.backend.temporal.activities.posthoganalytics.capture") as capture,):gate.return_value.decide.return_value = decision- gate.return_value.record_small_change_streak.side_effect = lambda decision, **_: decisionresult = _gate_push(GatePushInput(team_id=self.team.id,@@ -508,15 +488,15 @@ class TestTrackReviewCompleted(BaseTest):"limit = 10",["pnpm-lock.yaml"],True,- (False, "system_one_above_threshold", 0, False),+ (False, "no_rule_matched", 0, False),),- ("small_code_change", "limit = 20", [], True, (False, "system_one_above_threshold", 2, True)),+ ("small_code_change", "limit = 20", [], True, (False, "no_rule_matched", 2, True)),("small_code_change_with_a_lockfile_in_the_pr","limit = 20",["pnpm-lock.yaml"],True,- (False, "system_one_above_threshold", 2, False),+ (False, "no_rule_matched", 2, False),),])@@ -558,9 +538,7 @@ class TestTrackReviewCompleted(BaseTest):review_design=REVIEW_DESIGN_SINGLE_AGENT,skipped_paths=skipped_paths,)- decision = PushGateDecision(- skip=False, would_skip=False, reason="system_one_above_threshold", own_code_lines=100- )+ decision = PushGateDecision(skip=False, would_skip=False, reason="no_rule_matched", own_code_lines=100)with (patch("products.review_hog.backend.temporal.activities._installation_auth", return_value=("tok", None)),@@ -569,7 +547,6 @@ class TestTrackReviewCompleted(BaseTest):patch("products.review_hog.backend.temporal.activities.posthoganalytics.capture") as capture,):gate.return_value.decide.return_value = decision- gate.return_value.record_small_change_streak.side_effect = lambda decision, **_: decisionresult = _gate_push(GatePushInput(team_id=self.team.id,diff --git a/products/review_hog/backend/tests/test_temporal_workflow.py b/products/review_hog/backend/tests/test_temporal_workflow.pyindex c089fcdd382a..bff3424811e6 100644--- a/products/review_hog/backend/tests/test_temporal_workflow.py+++ b/products/review_hog/backend/tests/test_temporal_workflow.py@@ -266,7 +266,7 @@ async def _run_full_review_pr_workflow(return PushGateDecision(skip=gate_skips,would_skip=gate_skips,- reason="merge_only" if gate_skips else "system_one_above_threshold",+ reason="merge_only" if gate_skips else "no_rule_matched",delta_eligible=gate_delta_review,delta_review=gate_delta_review,)