Repository navigation
fix: delta re-reviews run on drafts that already have a review - #107
Conversation
Draft status skipped every automatic trigger, so a draft carrying a themis review never re-reviewed the commits answering its findings: authors pushed fix after fix and the open threads were never re-checked. Reviewing before marking ready is the normal way to work, which left delta unreachable for whole teams. A draft that carries a review was already asked about, so gate the draft skip on first reviews only; a push to a never-reviewed draft still gets nothing, since the delta base resolves to None.
| full review. | ||
| - Delta re-reviews are automatic triggers, but neither `auto_review` nor draft | ||
| status gates them: they answer a review that already exists, so | ||
| `delta_review` alone enables them. Drafts included — a draft carrying a |
There was a problem hiding this comment.
🟠 Major · ⚡ Quick win
Draft delta re-reviews never enter the pipeline.
Observed: _parse_pull_request in src/themis/events.py returns None for every payload whose PR is a draft, before it creates the ReviewJob that carries delta=True; the existing parser regression test asserts this exact outcome for draft synchronize events. Predicted: a push to a draft with a valid prior Themis checkpoint would still be ignored, so this documented behavior would not occur.
Allow draft synchronize events through the parser while continuing to skip draft opened and ready_for_review events, and add the parser-level regression test.
⚖️ Themis review: 🟠 Fix before mergeThis change updates the service-level draft guard, but draft push events are still discarded at the webhook parsing boundary, so the promised delta re-reviews cannot run. CI is still running.
🟠 Majors
📝 Walkthrough
🧪 How to verify
Product take: This is a solid workflow improvement for teams that review drafts, but it currently has no effect for webhook-driven deployments because the event never enters the review pipeline. 🧭 Assumptions & unverified claimsNo unverified assumptions or claims. The delta engine has the green light, but the webhook bouncer is still checking drafts off the list. · reviewed at 7e96dc1 |
The worker-side change alone was inert: _parse_pull_request dropped every draft payload before building the ReviewJob that carries delta=True, so a push to a reviewed draft never became a job at all. Allow the delta action through while first-review actions (opened, ready_for_review) keep skipping drafts.
What
Delta re-reviews now run on draft PRs that already carry a themis review. First reviews still skip drafts, unchanged.
Why
Draft status gated every automatic trigger, so a draft with an existing review never re-reviewed the commits that answered its findings. Observed on
Flagsmith/flagsmith-cli#43: a maintainer iterating on 12 open bot threads pushed fix after fix, and nothing re-checked them — the threads stayed open and delta never fired once.Reviewing a draft and iterating before marking it ready is the normal way to work; on some teams it is the only way PRs get reviewed. For them
delta_review: truewas unreachable — they had opted into a feature that could never trigger.The consent argument is #103's, one step further.
auto_reviewasks "review PRs nobody asked about?"; draft asks "is this ready to be looked at?". A draft carrying a themis review has already been looked at, deliberately (on a draft, a first review can only come from an explicit mention). The pushes that follow are answers to those findings.How
One condition: the draft skip applies to first reviews only.
Nothing else needed to stay safe — a push to a never-reviewed draft still gets nothing, because
_resolve_delta_basereturnsNonewhen no checkpoint exists and the run exits before any engine starts. So this cannot turn drafts into first-review noise, anddelta_reviewremains the switch.Tests
delta_review→ deltas (was: skipped entirely)delta_review: false→ nothingDocs updated; the delta section previously stated the old rule.