Large Language Model (LLM)

What is the difference between a model and the product using it?

The difference is that a model generates the text and a product decides what the model is given. It matters commercially, because the product is where your content enters the picture. The same model answers differently depending on whether the product searched the web first.

Why does an LLM go out of date?

An LLM goes out of date because training ends on a date and the world does not. Anything after that date is missing unless the product fetches it at the time of the question. That is the whole reason retrieval exists, and retrieval only reaches pages the search bots are allowed to fetch.

What is a context window?

A context window is the amount of text a model can hold in mind at once, counted in tokens rather than words. Anthropic documents a one million token window on its current Claude models, which it puts at roughly 555,000 words. Anything past the window has to be summarised or dropped.

What do people get wrong about LLMs?

• Treating a model name as a product name. Blocking one product's crawler does nothing about another product on the same model.
• Assuming a confident answer is a retrieved one. An answer with no link is usually memory, not research. That is where an AI hallucination lives.
• Citing NIST for a definition. NIST has no glossary entry for large language model. The wording above comes from a 2024 NIST publication quoting an executive order that has since been revoked.