R3: three-tier evidence report

Written 2026-09-20 (chunk 2). Deliverable, not a hypothesis; the criteria are consistency checks.

Design

report.py produces, per suspect model, a JSON and a Markdown report with three tiers, each of which can stop the chain:

• Signal gate. Reference-normalised mean alignment of the suspect with each reasoning-teacher family; the best must exceed an absolute gate (provisionally +0.05, from the H5 open-world numbers; to be set from the R2 negatives). Reports the matched control's value alongside. A suspect below the gate gets "no reasoning-teacher signal" and nothing further, which is what the ranking tests could never say.
• Family attribution. The family winning most probes; an anytime-valid e-process on family wins vs chance (valid at any stopping time, no Bonferroni), with the probe at which evidence crossed 1/alpha; the style classifier's probability mass on that family from the suspect's own generations.
• Member / pipeline attribution. Within the family: exact-lineage likelihood ranking and margin test where the probe set has several members; style probabilities over members and sub-sources (e.g. the two R1 pipelines). Explicitly labelled as the weakest tier.

Every report discloses the candidate pool and the classifier pool, and carries the sentence that a strong preference for a wrong near-relative is what a missing true teacher looks like.

Consistency criteria

• E1: every Phase 1 positive (S_r1, S_qwen3, S_gemini, S_mix*, S_r1_e9, S_r1_8k, S_r1_4k) passes tiers 1 and 2 with the correct family.
• E2: every negative (S_human, S_gemini-as-negative-for-R1, C, the R2 negatives) stops at tier 1, or, if it passes tier 1, fails tier 2.
• E3: the real distill passes tiers 1 and 2 with family r1 under its base reference.

Smoke test (2026-09-20, before R2 numbers)

Reports under reports/<suspect>.{json,md}. Batch generation: make_reports.sh.