A chatbot like ChatGPT or a local Qwen model generates text one small piece at a time. For each next piece it computes a probability over its whole vocabulary: "given everything so far, what likely comes next?" Pick, append, repeat. That’s the whole trick.
What this explains
- Why answers sound human: training text WAS human writing.
- Why answers can be wrong: high probability ≠ true. A confident wrong answer is called a hallucination.
- Why phrasing changes answers: you change the pattern it is continuing.
- Why it can’t "look things up" unless it’s given tools or sources.
Hallucinations are a side effect of predicting likely text rather than rare bugs. Verification is a core skill in this module for that reason.