What is an LLM (Large Language Model)? A Complete Beginner-to-Builder Guide

A large language model sounds like it must contain something close to understanding. It can explain code, write an essay, or debug an error. But underneath all of that sits one surprisingly simple task: predict the next token. Everything else is that task, repeated, at enormous scale, inside a particular kind of neural network.

This guide takes you from that one idea all the way down to how the machine actually works, with small Python programs you can run yourself, and ends with what to learn to build a career in this field. It is long on purpose. By the end you should have a mental model that holds together.

1. The one idea: next-token prediction

A language model reads the tokens so far and outputs a probability for every possible next token. It does not choose one correct word; it produces a ranked distribution, for example "destination 71%, router 9%, switch 4%, ...". One token is then picked, appended to the text, and the whole thing runs again for the next token.

You can see this core idea with a tiny model that just counts which word tends to follow which:

import re
from collections import defaultdict, Counter
 
corpus = """
a packet travels from the source to the destination.
a packet can be dropped when the network is busy.
the model predicts the next token from the tokens before it.
the model learns patterns from a large amount of text.
a firewall can drop a packet that looks unsafe.
""".lower()
 
words = re.findall(r"[a-z]+|\.", corpus)
model = defaultdict(Counter)
for a, b in zip(words, words[1:]):
    model[a][b] += 1
 
def predict_next(word):
    counts = model[word]
    total = sum(counts.values())
    return [(w, round(c/total, 2)) for w, c in counts.most_common()]
 
print("what follows 'the':", predict_next("the"))

Real output:

what follows 'the': [('model', 0.29), ('source', 0.14), ('destination', 0.14), ('network', 0.14), ('next', 0.14), ('tokens', 0.14)]

The model learned, from the text alone, that "the model" is the most likely pair. That is a language model in miniature. Everything that follows is about making this idea work at scale.

2. Why simple counting is not enough

The counting model has a fatal weakness: it only remembers exact pairs it has seen. Ask it about a three-word combination it never saw, and it has nothing. Language has more possible sentences than atoms in the universe, so no table of counts can ever cover them. This is called the sparsity problem.

The fix is to stop memorising exact sequences and instead learn patterns that generalise. That is what a neural network does: it represents each token as a vector of numbers (an embedding) and learns operations on those vectors, so that even an unseen sentence lands near similar ones it did learn from.

3. The transformer: the engine of a modern LLM

Almost every modern LLM is a transformer. You do not need to build one to work with LLMs, but understanding its core will make everything else make sense.

A transformer processes all tokens together and repeats the same two steps in each of its many layers:

1. Attention — every token looks at every other token and decides which ones matter.
2. A feed-forward network — each token's vector is then transformed on its own.

Stack dozens of these layers, and the model builds up a rich, context-aware representation of the text, which it finally uses to predict the next token.

Attention, shown with real numbers

Attention is the heart of it, so let us actually compute it. Each token gets three vectors derived from its embedding: a query (what am I looking for), a key (what do I offer), and a value (what I will pass on). A token compares its query against every token's key to decide how much to attend to each, then blends their values.

import numpy as np
np.random.seed(0)
 
tokens = ["the", "packet", "was", "dropped"]
d = 6
E  = np.random.randn(len(tokens), d)      # token embeddings
Wq, Wk, Wv = (np.random.randn(d, d) for _ in range(3))
 
Q, K, V = E @ Wq, E @ Wk, E @ Wv
scores = Q @ K.T / np.sqrt(d)             # similarity of every query to every key
 
def softmax(x):
    e = np.exp(x - x.max(axis=1, keepdims=True))
    return e / e.sum(axis=1, keepdims=True)
 
attn = softmax(scores)                    # each row sums to 1

The attn matrix is the interesting part. Each row is one token deciding how much to look at every token. Real output:

                  the   packet      was  dropped
       the      0.63     0.13     0.06     0.18
    packet      0.06     0.19     0.44     0.31
       was      0.84     0.09     0.02     0.04
   dropped      0.00     0.88     0.00     0.12

Read the last row: the token "dropped" puts 0.88 of its attention on "packet". Even with random weights, you can see the shape of the mechanism, each token pulling meaning from the tokens it relates to. In a trained model, those weights are learned so this attention lands on genuinely relevant words, which is how the model tracks that it was the packet that was dropped.

The blended result keeps the same shape as the input, but every token vector is now context-aware:

Shape of token embeddings in : (4, 6)
Shape of token embeddings out: (4, 6)   (same shape, now context-aware)

Positional information

Attention by itself has no sense of order, so "dog bites man" and "man bites dog" would look the same. Transformers add positional encodings to the embeddings so the model knows each token's place in the sequence. Order is information, and this is how it gets in.

4. What makes it large

Our toy pieces are tiny. A real LLM scales three things massively:

• Parameters. The numbers in all those weight matrices. Modern models have billions to hundreds of billions. These are what training adjusts.
• Layers and width. Dozens of transformer layers, each wide enough to hold rich representations.
• Training data. Trillions of tokens of text, far more than any human reads in a lifetime.

There is a well-studied relationship called scaling laws: as you increase parameters, data, and compute together in the right balance, the model's prediction error falls in a predictable way. Scaling is a large part of why recent models feel so capable, though it is not the whole story.

5. How an LLM is trained

Training has two broad stages.

Pretraining

The model is shown enormous amounts of text and asked, again and again, to predict the next token. When it is wrong, an optimisation method called gradient descent nudges its parameters to make the right token slightly more likely next time. The quantity being minimised is the cross-entropy loss (how surprised the model was by the true next token). Repeat this trillions of times and the model is forced to internalise grammar, facts, styles, and even reasoning patterns, because all of those help it predict better.

The result of pretraining is a base model. It only continues text. Ask it a question and it might continue with another question, because that is a plausible continuation.

Post-training (alignment)

To turn a base model into a helpful assistant, two more steps are common:

1. Supervised fine-tuning (SFT). Train it on many examples of instructions paired with good responses, so "continue the text" becomes "respond helpfully to the instruction".
2. Reinforcement learning from human feedback (RLHF). Humans rank competing answers, a reward model learns those preferences, and the LLM is tuned to produce answers people prefer. This shapes tone, helpfulness, and safety.

The engine never changes through all of this. It is still predicting the next token; post-training just changes which continuations it prefers.

6. From model to the thing you chat with

When you send a message, a hidden system prompt is placed above it to set the assistant's rules and role, then the model runs inference: the predict-and-append loop, until the answer is complete. The settings you may have heard of, like temperature, control how the next token is chosen at each step. Each of these has its own post later in this series.

7. Where GPT and Claude fit

GPT (OpenAI) and Claude (Anthropic) are families of large language models. They differ in training data, size, tuning, and guardrails, but share the same foundation: a transformer trained to predict the next token, then aligned with SFT and RLHF. "An LLM" is the general category; these are the best-known members.

8. What an LLM is not

• Not a database. It predicts answers, it does not look them up, which is why it can be fluent and wrong at once (see hallucination, later in the series).
• Not aware. There is no intent or understanding behind the words, only learned statistical structure.
• Not unlimited in memory. It only works with what fits in its context window right now, plus what it absorbed in training. For fresh or private facts, you add retrieval (RAG).

9. How to build a career from here

You do not need to train a model to have a career with them. A sensible path:

1. Use them well. Call an LLM through an API. Understand tokens, context windows, and cost. Learn prompt design.
2. Build real systems. Learn retrieval (RAG), vector search, chunking, and how to evaluate whether your system is actually good.
3. Go deeper if you want. Study the transformer, attention, and training properly, with the linear algebra and probability behind them. Then fine-tuning, quantisation, and serving.
4. Pick a domain. Pair LLM skills with a field you know. For security work, that means retrieval over your own notes, log analysis, and understanding prompt injection and other model-specific risks.

Most well-paid work today is in the first two steps: building reliable products on top of models. The deep internals matter, but they are not the entry gate.

10. Summary

• An LLM predicts the next token from the text before it, one token at a time.
• Counting cannot generalise, so LLMs use neural networks over token embeddings.
• The transformer's attention lets every token look at every other token; we computed a real attention matrix above.
• "Large" means many parameters, many layers, and trillions of training tokens, with scaling laws relating them.
• Training is pretraining (next-token prediction) plus post-training (SFT and RLHF) to make a helpful assistant.
• GPT and Claude are aligned transformer LLMs.
• You can build a strong career using and building on LLMs well before training one yourself.

Next, see exactly what happens the moment you send text to a model, in What is inference? (next in this series).