On-device AI, in plain words.

A program that predicts the next piece of text from the text so far. A chat assistant is a language model trained to continue a conversation in the assistant’s turn. It produces likely text, not checked facts.

A language model small enough to run on ordinary phones and laptops, from a few hundred million to a few billion parameters. Flyingkiwi’s models have about 0.6 billion. Small models are private and cheap to run, and know much less than large ones.

One learned number inside a model. A 0.6B model has about 600 million of them; Flyingkiwi English has 528 million after its vocabulary was trimmed. Every parameter is read once for every token the model writes.

The unit a model reads and writes: a word, part of a word, a punctuation mark or a single byte. “evlerinizden” (“from your houses”) is four tokens for Flyingkiwi’s Turkish model: ev, ler, iniz, den.

The component that turns text into tokens and back. Flyingkiwi uses byte-level BPE (byte-pair encoding), which builds tokens by merging frequent pairs of smaller pieces and keeps all 256 bytes, so any text can be encoded.

The set of tokens a model knows. Qwen3-0.6B has 151,669; Flyingkiwi keeps the 85,121 (English) or 78,255 (Turkish) that its language actually uses, which nearly halves the embedding and output layers.

How many tokens the model can see at once: the conversation so far plus the reply it is writing. Flyingkiwi uses 4,096 tokens, or 2,048 on devices that report 4 GB of memory or less.

The mechanism that lets each token draw on earlier tokens in the context. Flyingkiwi uses full attention in all 28 layers; a run that limited most layers to the last 1,024 tokens did worse on long conversations.

Rotary position embedding: attention learns how far apart two tokens are because their vectors are rotated by angles that depend on position. Qwen3 and Flyingkiwi use a base of θ = 1,000,000.

Instructions placed before the conversation that set the assistant’s behaviour. Flyingkiwi’s is short, and its cache ships precomputed with the model, so the first reply does not wait for it.

A plain-text chat format that marks the start and end of each turn with special tokens and a role: system, user or assistant. Flyingkiwi uses ChatML without the “thinking” blocks some Qwen3 models write.

Further training of an existing model on new examples to change what it does. Flyingkiwi fine-tuned every parameter of Qwen3-0.6B for two epochs; the English model took 148 training steps.

Training a small “student” model to imitate a larger “teacher”. Flyingkiwi’s students learned from answers written by Qwen3.5-35B-A3B under written rules about accuracy, uncertainty and false premises.

How well a model’s confidence matches how often it is right. A calibrated assistant answers what it knows and says “I’m not sure” about what it does not. Flyingkiwi built calibration examples from the TriviaQA questions its own base model could and could not answer.

A fluent statement the model has no support for, such as an invented date, source or rule. Flyingkiwi counts replies with invented details in its blind tests: 6 of 100 in English for v3.1, the same as the unchanged base model.

A question built on something untrue, such as asking why an event happened when it did not. A good assistant corrects the premise instead of answering as if it were true. English v3.1 did this in 5 of 10 test cases, the base model in 7.

Storing a model’s numbers with fewer bits. Flyingkiwi stores each weight as an 8-bit integer with one 16-bit scale per 32 values: half the size of 16-bit weights for 0.0003 nats per token of measured loss in English. Also spelled quantization.

A quantisation method (Frantar et al., 2023) that rounds a layer’s weights column by column and moves each rounding error onto the columns not yet rounded, using statistics of the layer’s inputs. On Flyingkiwi it halved the cost of 4-bit rounding, which was still not enough to ship.

Training while simulating low-precision rounding, so the model learns to tolerate it. A QAT model can make plain rounding look free on exactly that grid, which once made 4-bit weights look harmless in a Flyingkiwi measurement.

The stored keys and values of earlier tokens, so attention does not recompute them for every new token. Flyingkiwi keeps them as 8-bit numbers with one scale per 32 values; the first four positions stay in full precision.

Running a trained model to produce output. In Flyingkiwi, inference happens inside your browser tab, and your messages are not sent to a server.

Always choosing the single most likely next token. Output becomes repeatable, which is why Flyingkiwi’s evaluations use it; the chat itself samples with a little randomness.

A setting that sharpens or flattens the choice between likely next tokens. Lower values make replies more predictable, higher values more varied and more error-prone. The Flyingkiwi chat uses 0.5.

Nucleus sampling: the model chooses only among the most likely tokens whose probabilities add up to p, cutting off the unlikely tail. The Flyingkiwi chat uses 0.9, together with top-k 40 and min-p 0.05.

A setting that makes recently used tokens less likely, to stop a small model from looping. The Flyingkiwi chat applies 1.1 over the last 256 tokens, and its tests use 1.1 with greedy decoding.

AI that runs on your own phone or computer instead of a remote server. Your input stays on the device; the trade-offs are a smaller model and a one-time download, about 570 MB per language for Flyingkiwi.

A compact binary format that browsers run at near-native speed in a sandbox. Flyingkiwi’s engine is about a thousand lines of C compiled straight to WebAssembly with clang.

Single instruction, multiple data: one processor instruction that works on several numbers at once. WebAssembly SIMD handles 128 bits per instruction, for example sixteen 8-bit values.

A WebAssembly extension with faster instructions whose exact results may differ slightly between processors. Where the browser supports it, Flyingkiwi uses its 8-bit dot product: 16 multiply-adds per instruction with 7-bit activations.

A background thread for a web page. Flyingkiwi runs its engine in a module worker so the page stays responsive, and can share memory across up to six worker threads.

A browser state turned on by the COOP and COEP response headers. It allows shared memory between threads, which multithreaded WebAssembly needs, and requires every resource on the page to be same-origin or to opt in. Flyingkiwi serves everything itself.

A model’s average surprise, in nats per token, on text it was not trained on; lower is better. Flyingkiwi measures it on the assistant’s answers to compare weight formats: 1.327 for both the float and the 8-bit English model.

A measure of how different two probability distributions are; 0 means identical. Flyingkiwi uses it to check that the browser file predicts the same next tokens as its checkpoint: a mean of 0.0114 for English, 0.0513 for Turkish.

Test cases fixed before training and never used for it, so a score cannot improve by tuning on the test. Flyingkiwi has 100 per language, written by the project and kept off every upload.

Grading in which the grader does not know which system wrote which answer. Flyingkiwi mixes its answers at random with the unchanged base model’s before grading.

A benchmark of 817 questions written so that common misconceptions sound like the right answer (Lin, Hilton and Evans, 2021). In its multiple-choice MC1 form, Flyingkiwi English v3.1 scores 25.1% and Qwen3-0.6B 25.8%.

Data anyone can get, under a licence that allows reuse, with documentation of what it contains and who made it. Flyingkiwi’s fine-tuning prompts come from public datasets at pinned revisions.

A model whose trained parameters are published for anyone to download and run under a licence. Open weights do not make the training data open: Flyingkiwi’s weights are Apache-2.0, but its base model’s pretraining data is not public.

A permissive licence that allows use, modification and redistribution, including commercially, as long as the licence and notices are kept. Flyingkiwi’s models and most of its data sources use it.

Artificial general intelligence: AI with broad ability across many tasks at or beyond human level. There is no agreed definition; the main ones measure economic usefulness, levels of performance and generality, or how efficiently a system learns new skills.