Chapter 2 of 6

Training: from a teacher’s answers to a small student

A large teacher model wrote the answers. Each one was cut to its shortest complete form, and the small student learned to predict them word by word.

Fig. 1Trimmer and tuning board

Click or press Space to advance

English · 43% of characters kept · 22,067 answers rewritten

lit: the blade while trimming, then the teacher’s word at 1.0
pin height: student probability (illustrative)

Fig. 1 — Trimmer and tuning board. The teacher’s answers were cut to their shortest complete form, then the 0.6B student was trained to predict each next word of them; pin heights are illustrative, and the counts and losses are measured.

In numbers

35B → 0.6B

Teacher and student

Qwen3.5-35B-A3B wrote the answers; Qwen3-0.6B learned from them.

43%

Of each English answer kept

22,067 answers were rewritten. Turkish kept 33% of 30,036.

148

Training steps for English

Two passes over 15,834 conversations; validation loss 2.165 → 1.311. Turkish: 300 steps.

≈ 2 h

On one GPU

Rewriting 57 min, training 7 min for English and 15 min for Turkish, on one RTX PRO 6000.

Blazing Kiwi learned from 39,627 conversations the same way, and also matched the word probabilities of a larger assistant model, SmolLM2-1.7B, with 4-bit rounding simulated throughout.