Skip to content

Foundations

Technical foundations of large language models — what they are, how they work, and why they behave the way they do.

  • llm-architecture — Transformers, self-attention, tokenisation, statelessness, and model scale
  • context-windows — Context window properties, lost-in-the-middle effect, context rot, attention sinks
  • training-and-alignment — Pre-training, RLHF, Constitutional AI, DPO, fine-tuning, training data biases
  • inference-and-generation — Token generation, sampling, temperature, reasoning models, structured output, optimisation