Flyingkiwi · Blog
How a 0.6B language model is trained, shrunk and checked to run in a browser tab, how to read what any AI tells you, and what open data and AGI have to do with it.
How a small model answers, and how to read what it says.
What the data, the tests and the big definitions decide.
AGI
Four ways to define artificial general intelligence, what each would count as evidence, and where a 0.6B model honestly sits.
Open data
Reproducibility, contamination checks, licences and languages: what open data makes possible, using Flyingkiwi’s own training sources as the example.
Evaluation
Freeze the cases, compare against the unchanged model, grade blind, know your noise and publish the losers.
From the author · Ebook
Why public, free and open datasets matter
Artificial general intelligence will be shaped by the data that feeds it, not only by algorithms and compute. A short book for researchers, engineers, policymakers and citizens: the major open datasets, the legal and economic forces behind data policy, and what to do about them.
How the models are trained, shrunk, checked and run in a browser tab.
Architecture
Every limit came from arithmetic: why the vocabulary was cut instead of layers, how the model makes its own search key, and what the measurements say.
Quantisation
Plain 4-bit rounding cost 0.17 nats per token, 8 bits cost 0.0003. The measurements, the GPTQ and mixed-precision results, and a trap worth knowing.
Engine
Integer kernels, relaxed SIMD, six shared-memory threads and an 8-bit KV cache: what it takes to generate tokens on the CPU inside a tab.
Training
A 35B teacher, written rules, honesty probes and calibration from the student’s own knowledge: sequence-level distillation as Flyingkiwi used it.
Türkçe
Suffix chains, more tokens per sentence and less data. The Turkish model went from 12 to 37 out of 100, and why that is still not enough.
Browser AI · Release
One download, a checked 8-bit bundle trained for it, and a small C engine that answers on your device.
Asking clearly, reading answers critically and knowing what a small model is for.
Hallucination
Longer answers gave a 0.6B model more chances to be wrong. Retraining on the shortest complete answers brought invented details back to the base model’s level.
Small models
Good at rewording, drafting and simple explanations; poor at arithmetic, niche facts and anything recent. The measured strengths and failures.
Reading AI answers
Separate a useful explanation from a claim that needs evidence. A five-step check, with an everyday example.
English and Türkçe
Give a small model a task, the information it needs, and a concrete definition of a helpful answer.
Browser AI
Model downloads, device support, storage, and the difference between a local reply and a private website.
