H2: mixture estimation instead of single-teacher attribution

Written before the run (2026-09-19).

Question

When two candidate teachers tie under the likelihood statistic, "which one" is the wrong question. Can we estimate proportions of teacher styles in a student's generations, with a confidence interval, so that "about half R1, half Qwen3-like" is itself reportable evidence?

Design (first cut)

Reuse the H3 style classifier (trained on the five teacher sets with a uniform prior). For a set of student samples, run EM prior adjustment (Saerens et al. 2002) on the classifier posteriors: the re-estimated class prior is the mixture estimate. Bootstrap over samples for a 95% interval. Apply to S (the R1 distill) and C (control instruct). Script: mixture_em.py.

This is a style mixture, not a training-data mixture; the two coincide only to the extent that style is inherited. The Phase 1 testbed (students distilled from known 70/30 and 50/50 teacher mixes) is the real test of calibration.

Pre-registered criteria

• M1: for S, the r1_orig weight is the largest and its 95% CI excludes 0.20 (i.e. clearly a major component).
• M2: for C, the r1_orig weight's 95% CI includes 0 or is below 0.10, and no reasoning teacher dominates.
• M3 (sanity): applying the estimator to each teacher's own held-out texts recovers that teacher with weight above 0.8.

Calibration on known-mixture students (pre-registered 2026-09-19 before the run)

Phase 1 showed a 50/50 R1+Qwen3 student reads as 0.24 r1_orig by style (reasoning-only). Train two more students from the same base and recipe with 70/30 and 30/70 R1/Qwen3 data, estimate their style mixtures, and fit a monotone map from style weight to data weight through the five points (data r1 share 1.0, 0.7, 0.5, 0.3, 0.0 from S_r1, S_mix70, S_mix, S_mix30, S_qwen3).

• M4: the style r1_orig weights are monotone in the data share (0.0 < w(0.3) < w(0.5) < w(0.7) < w(1.0)).
• M5: a two-parameter map (e.g. logistic in the data share) fitted on all five points has leave-one-out error at most 0.10 in data share.
• Then apply the map to the real distill's style weight (1.00 r1_orig) and report the calibrated data share with its interval.

Calibration numbers (2026-09-19; S_mix70 and S_mix30 trained and evaluated in one 50-min burst, ~$3)

• M4 met: both the style weight and the likelihood win share are strictly monotone in the data share: a clean dose-response across five students.
• M5 not met: a logistic map on the style weight has leave-one-out error max 0.21 / mean 0.12 in data share (the curve is convex: weight rises 0.37 to 0.70 between 70% and 100% data). A linear map on the likelihood win share does slightly better (max 0.19 / mean 0.10) and is the more natural estimator, but neither meets the 0.10 bar with five points.
• Scale dependence, the important finding: the real distill's style weight is 0.998, above every calibration point, so the map extrapolates to a nonsensical data share of 2.6. The real distill was trained on ~800k R1 examples; our students on 800. Distillation strength, not just data proportion, sets the style weight. A mixture estimate is therefore only interpretable against calibration students of comparable strength, or with strength modelled as a second axis (e.g. total reasoning-teacher weight or likelihood gain over the reference). Files: results_calibration_head800.json, results_calibration_full.json.

Strength axis (pre-registered 2026-09-19 before the run)

The calibration map failed to transfer because the real distill is a far stronger distillation than 800-example students. Add strength as a second axis with two more R1-only students from the same base and recipe: S_r1_8k (8,000 original-R1 traces from OpenR1-Math-220k, 3 epochs) and S_r1_e9 (the same 800 s1K traces, 9 epochs). With S_r1 (800 x 3) that gives three strength levels for a pure teacher.

• M6 (monotone in strength): the style r1_orig weight (reasoning-only) and the likelihood r1_orig win share both increase from S_r1 to S_r1_e9 and to S_r1_8k.
• M7 (approaches the real distill): S_r1_8k's style weight is at least 0.85, i.e. within reach of the real distill's 0.998, so a strength-conditioned map becomes fittable.
• Prediction to record: if data volume matters more than epochs, S_r1_8k > S_r1_e9 on both signals.

Interim (S_r1_e9 done; trained on an L40S in 37 min after the H100 pool ran dry, loss 0.996): likelihood r1_orig 128/71 with mean alignment +0.190 (vs +0.159 at 3 epochs); style reasoning-only 139/60, r1_orig weight 0.78 [0.70, 0.84] (vs 0.70). Monotone so far, but three times the epochs bought only +0.08 in style weight; still far from the real distill's 0.998. S_r1_8k pending (first attempt OOM'd on the L40S at batch 4; relaunched on an H100 at batch 2).

Strength numbers (2026-09-19; S_r1_8k trained on an H100 in 50 min, loss 0.66; burst 82 min, ~$4.75)

• M6 not met, M7 not met, and the reason is a confound, not strength: the 8k set comes from a different source. Diagnostic (../h3_idiolect/results/source_*.json): adding an r1_openr1 class (200 OpenR1 traces) to the style classifier shows the two R1 sources are separable from each other at 0.91 CV on full text (0.76 on reasoning-only 800 words): the same checkpoint, sampled on different prompts through a different pipeline, has a measurably different fingerprint, though far closer to each other than to Qwen3 (0.95). With both classes present, S_r1_8k's generations go to r1_openr1 138 vs 16 (its actual source); S_r1 and S_r1_e9 also lean r1_openr1 (124 vs 29, 127 vs 38), and the real distill splits 100 vs 89. The 8k student's absolute likelihood alignment did rise with strength (+0.159 -> +0.190 -> +0.205), but Qwen3's rose more because the OpenR1 register is closer to Qwen3's than the s1K register is.
• Family level is robust regardless of source: R1-trained students land in {r1_orig, r1_openr1} 77% (S_r1_8k), 77% (S_r1), 83% (S_r1_e9), 95% (real distill); the Qwen3 student stays qwen3_32b 83% with both R1 sources in the pool.
• Conclusions: (1) a teacher fingerprint is checkpoint x prompt distribution x sampling pipeline; a lab must build its reference from its own serving pipeline's outputs, and family-level attribution is the reliable tier. (2) The strength axis must be scaled within one source; epochs gave +0.08 style weight for 3x, so the gap to the real distill (0.998) is mostly data volume and diversity, which needs a same-source 8k set (e.g. sampling the checkpoint ourselves) rather than a different public dump. (3) The real distill's 0.998 under the two-class R1 pool becomes a 100/89 split, so its "R1-ness" is family-level too.

Numbers

M3, held-out recovery (train on half of each teacher's texts, estimate the other half): every teacher recovered at 0.99 to 1.00 weight, on both full text and reasoning-only 800-word text. Files results/h3/m3_full.json, m3_head800.json on the i9.

M3b, synthetic blends of held-out halves (reasoning-only 800 words, 100 texts per blend):

Within 4 points on every blend, including the hard pair. M3 met.

M1 / M2, student and control (EM prior adjustment on the H3 classifier posteriors, 200 generations each, 500-sample bootstrap):

M1 met (r1_orig largest, CI excludes 0.20 by a wide margin). M2 met (control's r1_orig weight CI within [0, 0.02]). Together with M3/M3b, the style-mixture estimator is calibrated on known blends and gives an honest, interval-bounded proportion on real students. Files: ../h3_idiolect/results/h2_mixture_full.json, h2_mixture_head800.json.