How LLMs Actually Write: Token-by-Token Probability Sampling

Large language models generate text by repeatedly predicting the next token based on prior context, then sampling from a probability distribution rather than always picking the top word. This sampling explains why the same prompt can yield different outputs. The process is a loop: tokenize input, compute probabilities, sample, append, and repeat.