Why Turkish is hard for small language models

Turkish is hard for a small model for reasons that go beyond how much text exists. Its grammar packs meaning into long chains of suffixes, and a tokenizer can split Turkish words into many pieces. The training data raises its own questions. Flyingkiwi’s Turkish model is a 0.6B-class network built with these problems in mind. Its results show what helped and where it still fails.

Words built from suffixes

Turkish is agglutinative: a word is built by adding suffixes, one after another, to a root, and each suffix adds one grammatical meaning. Take evlerinizden, which means “from your houses”. The root is ev, house. The suffix -ler makes it plural, -iniz makes it “your” in the plural or polite form, and -den means “from”. A phrase that takes three English words is one Turkish word.

A longer example is gidemeyeceklermiş. Its root is git, go, which becomes gid- before a vowel. The -eme- part is the usual negative ability marker, so it means cannot. A linking consonant, -y-, follows, then -ecek marks the future, -ler means they, and -miş marks something reported or only apparently true. The whole word means something like “apparently they will not be able to go”.

The tokenizer does not follow these boundaries. It split evlerinizden into ev, ler, iniz and den, which matches the grammar. It split gidemeyeceklermiş into gid, em, eye, cek, ler and miş, which cuts across it. The model works with these pieces, so a single grammatical unit can be spread over several of them.

Vowel harmony keeps suffixes in step

Turkish vowels fall into two groups. Front vowels are e, i, ö and ü. Back vowels are a, ı, o and u. Most suffixes take a form that matches the vowels of the root, a pattern called vowel harmony. The plural of ev is evler, with front vowels, while the plural of kapı (door) is kapılar, with back vowels. The “from” form is evden for ev and kapıdan for kapı.

The rule is regular, but a model must learn every combination. The plural and the “from” form of just two nouns already give four different surface forms. Multiply that by every noun, verb and tense, and the learning task is large for a model of about half a billion parameters.

More tokens for the same meaning

A model reads text as tokens, which are pieces of words, and it writes one token at a time. Measure the same meaning in both languages and the Turkish version is longer. The sentence “Yarın sabah saat dokuzda toplantıya katılamayacağımı öğretmenime bildirmem gerekiyor.” has nine words and takes 27 tokens with the Turkish tokenizer. Its English translation, “I need to tell my teacher that I will not be able to attend the meeting at nine tomorrow morning.”, has twenty words and takes 21 tokens with the English tokenizer. With the English model’s trimmed vocabulary, the Turkish sentence takes 37.

That works out to about 1.3 times as many tokens for the same meaning, roughly three tokens per Turkish word against about one per English word. This is one example, not a measure of all Turkish text. The consequences are practical: a reply needs more steps to generate, the context window fills faster, and the same content takes longer to appear on screen.

Where the training data came from

The Turkish model combined several sources. Its training set has 20,778 conversations, with 388 kept for validation. The human-written prompts came from OASST2, Aya and translated OASST2 prompts, 7,761 in all. Another 7,455 requests were written by the teacher model, and 433 were GSM8K word problems.

The rest of the mix targets behaviour the base model lacks. It includes 1,941 honesty probes, which teach the model to say when it does not know, and 460 questions about invented entities, which test whether it will make things up. There are also 474 memory checks and 1,567 chained conversations, which train it to keep track of earlier turns.

What the Turkish model gained

The base model mostly fails in Turkish. On the 100-case frozen set, Qwen3-0.6B answered 12 cases correctly, and the Flyingkiwi Turkish model answered 37. The table shows where the gains came from, with the number of cases in each category.

The largest change is in honesty about the unknown. On the 20 questions the model cannot answer, the base model said so once, and Flyingkiwi 14 times. The gains are real, but 37 out of 100 is low in absolute terms, and the grading was done by an AI system rather than a native speaker.

Where it still fails

The Turkish model still gets arithmetic and facts wrong often. Its replies with invented facts or claims numbered 17 on the frozen set, against 19 for the base model, so the improvement there is small. Its answers are also longer than the base model’s, averaging 92 tokens against 74, even after the rewrite.

Engine fidelity is lower than in English too. Fidelity here means how closely the browser engine’s output probabilities follow the model’s checkpoint, measured as mean KL divergence at the first reply token across 30 development prompts, where lower is closer. The Turkish figure is 0.0513, against 0.0114 for English. It is a weaker match, and it belongs next to any Turkish score.

A vocabulary built for Turkish

The Turkish model keeps 78,255 tokens from the base model’s 151,669. Rare entries were removed and the byte-level BPE merges were rebuilt, but text made only of kept tokens is still tokenised exactly as before. The 256 byte symbols remain, so nothing becomes unencodable. On held-out Turkish text, the trimmed vocabulary needs 0.21% extra tokens. The Turkish download is also slightly smaller: 567 MB against 571 MB for English.

Practical tips for Turkish users

Short, concrete questions work best with this model. Ask for one thing at a time, and check every number before you use it. Give the names, numbers and dates you already know, so the model does not have to guess them. The guide to checking AI answers shows how to verify a claim. If you need code, or a technical term is clearer in English, switch languages. Both models are on the site, and the clear prompts article has examples in both languages.

For the wider picture, see what a 0.6B language model can and cannot do.