Why AI chatbots make things up, and what shorter answers fixed

Ask a small chatbot about a book you have never heard of, and it may still give you an author, a publication year and a plot summary. The answer reads smoothly, so it sounds as if the model knows. Often it does not. This problem is usually called hallucination, and it matters for every chatbot, including a small one that runs on your own device.

What “making things up” means

In this sense, a hallucination is a fluent statement that the model has no good support for. The model has no intention to deceive. It produces a sentence that fits the pattern of an answer, and nothing in that process checks the sentence against a source.

A plain mistake and an invented detail are different problems, although they often appear together. An invented detail is a claim with no support behind it, and it can sound as confident as a claim that happens to be true. That is why the useful question is not only “is this right?” but “what would show that it is right?”

Here is an illustration, not a quotation from any model. You ask about a local festival in a town you name. The reply gives a date, a start time and the name of an organiser. Each detail looks plausible, and any of them could be invented. Only a check against the town’s own calendar would show which, if any, are real.

Why it happens

A language model predicts which token is likely to come next, based on the text it learned from. It does not look up a fact, check it and then write it down. Most of the time the likely continuation and the true one are the same, which is why the output is useful. When they differ, the model can still produce a smooth continuation.

Small models add two pressures. A 0.6B-class model, with 528 million parameters in English after vocabulary trimming, stores much less than a large assistant, so it has fewer facts to draw on and more gaps to fill. Training can add a third. If a model learns from answers that people judged helpful, it may learn that an answer is always expected. This is a common explanation. Flyingkiwi’s results fit it, but they do not prove it.

Every extra sentence is another chance to be wrong. A short answer with one checkable claim gives you one thing to verify. A long answer may contain many claims, and you are likely to check only the first few.

What Flyingkiwi measured in v3

Flyingkiwi’s v3 release was fine-tuned from Qwen3-0.6B on answers written by a larger teacher model. The English release blended that fine-tune 50/50 with the base model, and the Turkish release used the fine-tune alone. Students learn the style of their teacher as well as its facts, and v3 learned to write long. On the frozen test sets, which are 100 project-written cases per language that were never used for training, v3 answers averaged 83 tokens in English against 37 for the unchanged base model. In Turkish the gap was wider: 256 tokens against 74.

The blind grading also counted replies that contained invented facts or claims. In English, v3 had 11 such replies out of 100, against 6 for the base model in the same grading run; in Turkish, 28 against 26. On TruthfulQA, from Lin, Hilton and Evans, a multiple-choice test built from questions on which some people would answer falsely because of a common misconception, v3 scored 21.3% against the base model’s 25.8%. That gap is larger than the test’s sampling error of about 1.5 points.

A brevity instruction did not fix it

The first idea was the obvious one: ask for short answers in the system prompt. It did not work, and the answers stayed long. One plausible reading is that the length had been learned from the training answers themselves, so a line of instruction could not undo it. That is an interpretation rather than a proven cause, but it explains why the fix had to change the training data.

Rewriting the training answers

For v3.1, the teacher saw each whole conversation and rewrote every assistant message into the shortest complete answer. The rules were strict. Code blocks, requested formats, honest statements of uncertainty and anything a later turn refers to had to survive. A rewrite that dropped a code block, added a disclaimer, grew longer or switched language was rejected, and the original answer was kept.

Whole conversations were used because a later turn can depend on an earlier answer. A follow-up question may say “the second option” or repeat a name from before. Shortening an earlier answer had to leave those references intact, or the later turn would no longer make sense.

The rewrite covered 22,067 English answers, keeping 43% of their characters, and 30,036 Turkish answers, keeping 33%. The reasoning was simple. Every extra sentence is another chance for a 0.6B-class model to state something false, so the training answers should contain only sentences that do work.

What the results show

The table compares v3 and v3.1 with the unchanged base model, Qwen3-0.6B, on the same frozen sets. The base-model column comes from the v3.1 grading run; in v3’s own run the base model had 6 English and 26 Turkish invented-detail replies.

On English, v3.1 matches the base model on invented details, and it gets the same 63 of 100 cases overall. The rewrite removed a difference on this test, not every mistake. In Turkish, v3.1 has fewer invented-detail replies than the base model, 17 against 19, and its answers are still longer than the base model’s. Its score of 37 of 100, against 12 for the base model, is a large gain, and it is still not a reliable assistant.

Read these numbers with their limits. The grading was done by AI graders against reference answers, with spot checks, and there was no independent human or native-speaker panel. Each case moves a category score by 3–10 points. The flags count replies, not individual false claims, so a reply with several false claims counts once. English v3.1 also corrects false premises less often than the base model, 5 times in 10 against 7 in 10. Shorter answers did not improve every behaviour. The technical report gives the full method.

What you can do with any chatbot

The same caution applies to any chatbot, large or small. Ask for short answers, which leaves fewer claims to check. Ask the model to separate what it knows from what it is guessing, and treat a confident reply with no such separation as a draft. Then check any name, number or date you would act on against a source you can open. Our guide to checking an AI answer without starting over walks through that step, and clear prompts in English and Turkish shows how to ask for a short, specific reply in either language.