Reading your first review
Your first review is also a calibration exercise. Here is what appears, and what to look at before you decide whether the panel suits your codebase.
What appears on the pull request
Two status checks and, at most, a handful of comments. Each comment carries its vote as three discs — filled for a juror that affirmed the finding, hollow for one that refuted it.
●●● Blocking all three jurors affirmed
●●○ Warning two affirmed, one refuted
○○○ Struck never posted; kept on the record
If it seems too quiet
Fewer comments than your previous tool is the expected outcome, not a fault. Confirm it is working by opening the struck record: if the scan produced twelve candidates and the panel posted three, the filter is doing its job. If the scan produced three, the panel is not your problem — check scan.paths in your configuration.
If a finding is wrong
Reply @dicast why to see each juror’s reasoning, including the dissent. If it is genuinely wrong, @dicast dismiss records that and lowers the prior for that pattern in this repository from then on.
A reasonable first two weeks
- Keep your current reviewer running in parallel. Compare comments posted against comments acted on — that ratio is the only benchmark that reflects your code.
- Leave the merge gate advisory. Do not make it required until you have seen a week of verdicts.
- Read the struck record twice, not the comments. What the panel threw away tells you more about calibration than what it kept.
- Add house rules only after you have seen a finding you wish had been weighted differently.