1. Predicting the next token
A generative language model estimates which token is likely to come next given a preceding sequence. Tokens may be words, parts of words, punctuation or other learned text fragments. Output is generated one token at a time, each new prediction conditioned on the current context.
Although the objective sounds simple, training on diverse examples teaches a network intricate statistical patterns of language, code, structure and relationships.
2. Turning text into vectors
Text is tokenized into IDs and then represented by learned numerical embeddings. Position information helps distinguish order, so the model can respond differently to “the dog chased the cat” and “the cat chased the dog”.
3. The transformer and attention
Most modern LLMs rely on transformers. Attention mechanisms help a token representation draw information from relevant earlier tokens. Multiple heads can capture different relationships, while feed-forward layers refine the information at each layer.
A causal text generator masks future tokens during prediction. Its internal representation of context is repeatedly transformed as the layers process it.
4. How training works
Pretraining
The network processes extensive training examples and updates its parameters to reduce prediction error. Those updates use gradient-based optimisation, repeated on a vast scale.
Instruction tuning and preferences
Additional training can improve how the model responds to questions, follows formatting instructions and handles harmful requests. These processes shape behaviour but do not guarantee truthfulness.
5. What happens during inference
When you submit a prompt, an inference engine tokenizes it, runs the network and selects the next token according to decoding settings. Temperature and other sampling controls influence output diversity. A KV cache helps avoid recomputing previous attention states, but memory requirements grow with context and concurrent sequences.
6. Quantisation and running models locally
Lower-precision model weights, such as eight-bit or four-bit formats, can reduce memory needs. Real usage also includes scales, buffers, cache and potentially higher-precision layers. Hardware and inference-engine compatibility matter.
For practical experimentation, a framework such as vLLM serves models through an API. See our Ubuntu vLLM guide for a starting workflow.
7. Important limitations
- A fluent answer is not proof that a claim is correct.
- Models can invent convincing-sounding references, commands and explanations.
- Outputs depend on training, model architecture, prompt and decoding settings.
- Private information shared with a hosted model should be treated as information disclosed to an external service.
Useful mental model: an LLM is a powerful learned prediction system, not a database that retrieves guaranteed facts. Verify important claims using authoritative sources.