R2: false-positive sweep

Written before the run (2026-09-20). Chunk 2 of the roadmap.

Question

How often does each signal accuse an innocent model, and what absolute thresholds keep that rate near zero? Phase 1 had three negatives (the control instruct model, a Gemini-trained student, a human-solution student). This sweep adds negatives of the kinds the plan lists as hard, and two positives with a different base, to separate "no R1 signal" from "R1 signal under a mismatched reference".

Suspects

Negatives (no exposure to R1 or Qwen3 outputs):

• S_merge_gem_hum: weight-average of S_gemini and S_human (a merge of clean fine-tunes)
• S_human, S_gemini, C (Qwen2.5-Math-1.5B-Instruct): from Phase 1, re-scored in the same pool
• X_smol_it: HuggingFaceTB/SmolLM2-1.7B-Instruct (other family, instruct)
• X_phi4mini: microsoft/Phi-4-mini-instruct (other family, 3.8B, reasoning-tuned by Microsoft, not on R1 to our knowledge; a deliberately hard negative)
• X_qwen25_it: Qwen/Qwen2.5-1.5B-Instruct (same family, instruct; already scored as R2)

Positives with a different base (R1 traces, s1K, 800 x 3 epochs, same recipe):

• S_r1_smol: base HuggingFaceTB/SmolLM2-1.7B (other family, other tokenizer); exact-lineage reference = SmolLM2-1.7B base (R4 scores exist for the teachers)
• S_r1_mathbase: base Qwen/Qwen2.5-Math-1.5B (the real distill's own base); reference = R (scored in Phase 0)

References: each suspect is scored against its own base where available, and against Qwen2.5-1.5B (R3) as the mismatched reference.

Pre-registered criteria

• F1 (Tier 1 gate): with reference = own base, every negative has max mean alignment over reasoning teachers below +0.05, and every positive has r1_orig mean alignment above +0.05. (Threshold from the H5 open-world numbers.)
• F2 (family attribution, style): every negative's generations have combined {r1_orig, r1_openr1, qwen3_32b} mean probability below 0.25 on reasoning-only text; every positive above 0.60.
• F3 (mismatched reference): under R3 as reference, count how many negatives cross the +0.05 gate; report as the FPR cost of a wrong reference. No threshold, this is the measurement.
• F4 (hard negative): Phi-4-mini is the case most likely to trip the family gate (Microsoft's own reasoning tuning). Record whichever way it goes; a trip here would mean the family tier detects "reasoning-trace style" rather than lineage.

Numbers

(to fill)