LLM foundations
Tokens, embeddings and context windows: what a language model actually receives, and the limits that follow from it.
Part of Artificial intelligence. 4 articles.
What are embeddings? Meaning as numbers, explained
Embeddings turn text into lists of numbers so that similar meanings land close together. Learn how they work, how similarity is measured, and where they fail.
Beginner3 minWhat is a context window in an LLM?
The context window is the maximum number of tokens a language model can consider at once. Learn what fills it, what happens when it overflows, and how to budget it.
Beginner3 minWhat is a token in an LLM?
A token is the unit of text a language model reads and writes. Learn how tokenizers split text, why token counts differ by language, and why it matters for cost.
Beginner4 minWhy long context windows still miss things
A million-token context window does not mean a model uses every token well. Learn about the lost-in-the-middle effect, attention cost, and how to test it yourself.
Intermediate3 min