beginner · ai
How LLMs work: from tokens to transformers
What a large language model actually is, built up from tokens, embeddings and attention to training, fine-tuning and why it sometimes makes things up.
·
By Amit Chakraborty — Kolkata, India · Remote worldwide
Who this is for
career_switcher
What you will be able to do
Explain, and sketch in code, how an LLM turns a prompt into text — and reason about cost, context and failure modes from that model.
Syllabus — 10 chapters
- 1. What a language model predicts — Frame an LLM as next-token prediction and see where that framing breaks.
- 2. Tokens and tokenizers — See how text becomes tokens, and why that sets cost and context limits.
- 3. Embeddings — Represent meaning as vectors and measure similarity.
- 4. Attention — Follow how each token decides which others matter.
- 5. The transformer block — Assemble attention, MLPs and residuals into the architecture.
- 6. Pre-training — Understand the data, the loss and the compute behind a base model.
- 7. Fine-tuning and RLHF — Turn a base model into an assistant, and know what that changes.
- 8. Sampling: temperature, top-p and friends — Control output randomness deliberately.
- 9. Hallucination and grounding — Know why models invent facts, and how retrieval and tools reduce it.
- 10. Running a model locally — Run a small open model on a laptop and in the browser.
0 of 10 published. A new chapter every Thursday night. Free to read.
Learn it free, elsewhere too
The documentation, videos, books and practice sites worth your time on this subject. All free. None of it is mine.