I'll unpack the v0.6 package, read `START_HERE.md` and the focused review prompt first, then work through the manuscript, change record, baseline and verifier.

Now the verifier. I'll read it before running it, then run it from an isolated copy.

# Adversarial review: ER-13 Draft v0.6 corrective successor

**Bottom line:** v0.6 fixes every blocking and major finding from my v0.5 review. The sources I could reach check out, including the corrected S14. Two new MAJOR gaps remain in the study definition:
- the Control arm is no longer defined;
- the primary outcome doesn't say how a review the evaluator ends early is scored.

Each needs only a few sentences. There are no blocking defects.

## What I checked

- **Package integrity.** All 16 manifest hashes match. The v0.5 baseline PDF is byte-identical to the one I reviewed (`b9d6e4…`), and the baseline review file matches my v0.5 review.
- **Verifier: I ran it**, on an isolated copy.
  - Without PyMuPDF: 27 checks PASS, with the PDF checks skipped.
  - After installing PyMuPDF: 36/36 PASS, identical to the shipped `verification_results.json`.
  - This PASS is a document-integrity and arithmetic result only, not scientific validation of the study.
- **Checks outside the verifier.**
  - All 8 questions and 8 author positions appear verbatim in the v0.5 PDF itself. The verifier only compares them to a JSON file inside the package.
  - The PDF, Markdown, JSON and DOCX editions carry the same text.
  - I diffed v0.5 against v0.6 sentence by sentence to look for unrecorded removals.
  - The planning table reproduces: 291/389, 136/181, 79/106.
- **Sources.** S14, S21–S23 and S27–S34 against publisher, proceedings, arXiv or Crossref records. S01 still can't be verified from here.

## Dispositions of my v0.5 findings

| Finding | Disposition | Basis |
|---|---|---|
| B1 (S14 reversed) | **RESOLVED** | The row and §5.1 now match the paper. The private-preference DVD design is confirmed in the publisher text. |
| M1 (carryover vs co-occurrence) | **RESOLVED** | §13.1 separates H-S from H-C and selects only H-S. §13.2 step 3 bars arm-specific co-errors, and §14.6 says neither result resolves the other. |
| M2a (one estimand) | **RESOLVED** | Δ is defined once in §13.3 and carried through §14.1, §14.2, §14.4 and §14.5. The β coefficients are secondary. Two specification points remain (n1). |
| M2b (hits vs false alarms) | **RESOLVED** | Joint item-type × condition model, a mandatory Δ_F, the §14.6 interpretation rule, and correct signal-detection formulas as secondary analyses. |
| M2c (timing) | **RESOLVED** | Undetected items are now censored at the deadline. A separate new gap is N2. |
| M2d (model terms) | **RESOLVED** | Order_ij, counterbalancing, crossed evaluator and item effects with item exposure slopes, and items that recur across arms. |
| M3 (machine evidence) | **RESOLVED** | S30–S34 are added with accurate findings and boundaries. One precision gap remains (n3). |
| Minor 1 | **RESOLVED** | |
| Minor 2 | **RESOLVED** | The citation style still uses author initials, which is cosmetic. |
| Minor 3 | **PARTLY RESOLVED** | Stale labels are gone and a history table is added. v0.3's content is honestly recorded as unknown. |
| Minor 4 | **PARTLY RESOLVED** | The origin of the thresholds is honestly recorded as unknown, with no invented attribution. |
| Minor 5–7 | **RESOLVED** | |
| Minor 8 | **RESOLVED** | The boundary is added, but the DOI is missing (n5). |
| Minor 9 | **RESOLVED** | "Three-digit" and the 69% figure both appear in the arXiv abstract. |
| Minor 10 | **RESOLVED** | The record marks this PARTLY, but the chance-baseline formula is correct, and that was all I asked for. |
| Minor 11–12 | **RESOLVED** | |

The open parts of Minors 3 and 4 can't be fixed from the supplied files. That is acceptable for circulation. Before final promotion they should be closed, either by finding the record or by declaring it unrecoverable.

## New defects

**BLOCKING:** none.

### MAJOR

**N1. The Control arm is undefined (§13.2 step 2; §13.1; §14.7).** v0.5 said Control saw "ordinary non-operational outputs". v0.6 dropped that without recording it, and now says only that the arms are matched and that the intended contrast is the conspicuous-error exposure.

What Control sees decides what Δ means:
- **Error-free Control:** Δ is the effect of seeing any observable errors. The prevalence mechanism (S28) and the trust mechanism (S29) both respond to any observed error, so an effect couldn't be attributed to conspicuousness.
- **Control with matched non-conspicuous errors at the same rate:** this isolates conspicuousness.

Two other things are unspecified: whether participants respond to the exposure items, and whether they get feedback. Exposure participants who flag errors get an experience of finding something that Control participants don't.

Minimal wording for §13.2 step 2:

> "Control outputs are identical to Exposure outputs except that each conspicuous error is [corrected / replaced by a matched non-conspicuous error]; the protocol must state which. With an error-free Control, Δ estimates the effect of exposure to observable errors and cannot attribute an effect to conspicuousness as such. The exposure-phase task, response format and any feedback are identical across arms."

Also add "Control-arm content; exposure-phase task and feedback" to the §14.7 freeze list.

This belongs in the crosswalk because it defines H-S. The protocol can make the actual choice.

**N2. A review the evaluator ends early has no assigned outcome (§13.3; §14.2 timing; §14.5).** A score of Y_ij = 0 requires "a completed available review window", and the timing analysis allows for "informative early censoring". If an evaluator can finish an item before tau, a miss in that case could be read three ways: a 0, a missing value, or a time censored at submission.

Coding it as missing or censored would remove the mechanism S28 identifies. When targets were rare, observers set a lower bar for quitting the search, gave up sooner, and missed more.

Minimal wording for §13.3, after the definition of Y:

> "If an evaluator ends review of a target item before tau without a valid detection, Y_ij = 0, and for timing no detection occurs through tau; this is an outcome, not censoring. Missing status applies only to technical loss, withdrawal, or non-presentation."

For §14.2 timing:

> "Informative early censoring refers to withdrawal or technical loss, not voluntary completion."

If the interface instead enforces a fixed viewing time, say so; that changes the estimand.

### MINOR

1. **n1. Two model specification points (§14.2).**
   - *(a) Item variance by type.* As written, one item-intercept variance covers both target and benign items, so benign-item variability feeds into the integration that produces Δ. Suggested wording: "c_0j and c_Ej have variances specific to target and benign items; target marginalization uses the target components."
   - *(b) Stratum-specific exposure effect.* Δ is weighted by stratum, but f has no E × K term, which forces one exposure effect on the logit scale across all strata. Suggested wording: "f includes E_i × K_j unless pilot data justify a common logit effect, fixed before confirmatory enrollment."
2. **n2. Report the design-based estimate every time (§14.2).** The randomization-based fallback doesn't have to narrow the target to the finite item bank, because resampling evaluators and items jointly keeps the item-population target. Reporting it routinely shows how much Δ depends on the Gaussian integration. Suggested wording: "Report the design-based stratum-weighted difference in evaluator-level detection proportions in every analysis as a prespecified sensitivity estimate, with uncertainty from resampling evaluators and items jointly."
3. **n3. The S31 row omits the paper's one non-nudged case (§13.1).** Add: "In rare cases, one model underperformed strategically in pursuit of helpfulness without a strongly nudged in-context goal; the setting remained a constructed evaluation." This is accurate and doesn't upgrade the evidence to prevalence or intent.
4. **n4. The S23 rows misstate the 7% denominator (§5.4, §5.6).** The 7% is 38 of 560 AI-assisted assessments (28 experts × 20 each), not 7% of previously correct judgments. Suggested wording: "Of 560 AI-assisted assessments by 28 pathology experts, 38 (about 7%) changed a previously correct estimate to follow incorrect AI advice; time pressure did not change this frequency (19/280 in each condition) but increased anchoring." The change record already has the 38/560 figure.
5. **n5. S22 is missing its DOI in the register.** Add 10.15607/RSS.2026.XXII.181. The change record says the DOI was confirmed, but the register doesn't carry it.
6. **n6. Three issues with the verifier.**
   - Three "definition-level" checks compute hardcoded constants and inspect nothing in the manuscript. Label them as arithmetic illustrations, or replace them with checks that read the manuscript.
   - The frozen questions and positions should be derived from the baseline PDF rather than from a JSON file inside the package.
   - With current PyMuPDF, the `fitz` shim prints a deprecation warning to stdout ahead of the JSON. Use `import pymupdf`.
7. **n7. S27's page range.** Crossref gives 133–139, but the register says 133–140, which is the range my v0.5 review also cited.

## Deferred to the pilot protocol (not defects here)

These can be settled in the protocol:
- the review deadline tau, the stratum weights w_k and v_k, the minimum important effect delta_star, and the exposure dose;
- item-bank size and recurrence, the scoring and localization rubric, and the covariate set;
- the integration method, the bootstrap count and test rule, and the order for reducing the covariance structure;
- the specifics of the missingness sensitivity analyses, the participant population, and ethics and debriefing.

One feasibility note for the protocol: refitting the full crossed model for every bootstrap draw inside every power simulation means roughly one million fits per scenario at 1,000 × 1,000. Budget for that, or validate a cheaper interval for the simulation stage.

## Source checks this round

| Source | Result |
|---|---|
| S14 | Confirmed in the publisher's full text. The main experiment used two DVDs with privately chosen preferences, and ambiguity aversion was eliminated in the private treatment. The register could add issue 3. |
| S21 | Confirmed in the arXiv abstract: a three-digit colored-MNIST task, with 69% of certificates refuted in the bypass case. Only v1 (26 Sep 2026) exists. |
| S22 | Confirmed: foundation models supply the verification feedback. The DOI is missing (n5). |
| S23 | Confirmed: 28 experts and 38 of 560 assessments. See n4. |
| S27 | Confirmed: abstract and Crossref. |
| S28 | Confirmed: about 7% misses at 50% target prevalence against about 30% at 1%, driven by earlier quitting. |
| S29, S30, S33, S34 | Confirmed. |
| S31 | Confirmed: v1 in December 2024, v2 in January 2025, and the subtle-mistakes finding. See n3. |
| S32 | Confirmed: the current v3 abstract still reports AUC 0.83 against 0.92. |
| S01 | **Not verified.** Publisher pages block automated access, and Semantic Scholar has no abstract. The change record says it was confirmed, but I can't reproduce that. |
| S02–S13, S15–S20, S24–S26 | Not re-checked this round. |

## Readiness

- **Research circulation: READY** as a NOT FINAL research draft. Nothing in it misstates the evidence. N1, N2 and the minor fixes can go into a v0.6.1.
- **Drafting the pilot protocol: READY TO BEGIN once N1 and N2 are fixed in the crosswalk.** Those two define H-S and its primary outcome. Everything in the deferred list belongs in the protocol.
- **Participant recruitment: NOT READY.** There is no protocol, ethics determination, stimuli or generator, frozen parameter set, simulation code or independent statistical review yet. The crosswalk says so itself.
- **Final Archive promotion: NOT READY.** It needs:
  - N1, N2 and the minor fixes;
  - S01 confirmed by someone with access;
  - the two provenance items closed;
  - at least one review that isn't AI-assisted.

  All review so far, including this one, is AI review. Neither that nor the verifier's PASS is evidence that the study design works or that the crosswalk changes anyone's behavior.

Sources:
- [Trautmann et al. 2008, full text](https://link.springer.com/article/10.1007/s11166-008-9038-9)
- [ISS, arXiv 2609.32247](https://arxiv.org/abs/2609.32247)
- [HUME, RSS proceedings](https://www.roboticsproceedings.org/rss22/p181.html)
- [Rosbach et al., arXiv full text](https://arxiv.org/html/2603.11821v2)
- [Rosbach et al., MELBA](https://www.melba-journal.org/papers/2026%3A007.html)
- [Berbaum et al. 1990 (Crossref)](https://api.crossref.org/works/10.1097/00004424-199002000-00006)
- [Wolfe et al. 2005 PDF](https://search.bwh.harvard.edu/new/pubs/WolfePrevalenceNature05.pdf)
- [Wolfe et al. 2005 (Crossref)](https://api.crossref.org/works/10.1038/435439a)
- [Dietvorst et al. 2015 abstract](https://repository.upenn.edu/fnce_papers/58)
- [Dietvorst et al. 2015 (Crossref)](https://api.crossref.org/works/10.1037/xge0000033)
- [van der Weij et al., ICLR 2025](https://proceedings.iclr.cc/paper_files/paper/2025/hash/b5e5753b0a0e440a6d8dc7e143617cec-Abstract-Conference.html)
- [Meinke et al., arXiv](https://arxiv.org/abs/2412.04984)
- [Needham et al., arXiv v3](https://arxiv.org/abs/2505.23836v3)
- [Huang et al., ICLR 2024](https://iclr.cc/virtual/2024/poster/18956)
- [Panickssery et al., NeurIPS 2024](https://proceedings.neurips.cc/paper_files/paper/2024/hash/7f1f0218e45f5414c79c0679633e47bc-Abstract-Conference.html)
- [S01 Semantic Scholar record](https://api.semanticscholar.org/graph/v1/paper/DOI:10.1016/j.jsr.2006.10.011?fields=title,year,venue,abstract)