Large Language Models (LLMs) Glossary: Key Terms & Concepts

Context management in AI code generators refers to the system's ability to understand, retain, and apply relevant information throughout the code generation process. This feature enables t...read more

The vendor supplies the software to run on infrastructure the customer controls, whether physical servers, a private data centre, or the customer's own cloud tenancy, instead of using the ...read more

Storing the processed form of a prompt's repeated content, such as a long system instruction or a reference document, so that it does not have to be recomputed on every request to a langua...read more

Code generation produces source code from a higher-level input such as a natural language description, a specification, or an existing schema. Output can range from a single function or co...read more

An embedding model converts text, images, or other content into a list of numbers — a vector — positioned so that items with similar meaning sit close together. Those vectors are what make...read more

A language model is given descriptions of the functions or tools available to it and, when a request needs one, returns a structured call naming the function and its arguments instead of p...read more

Running a model over a large collection of inputs as a scheduled or queued job instead of one request at a time. Since results are not needed instantly, the workload can be packed for thro...read more

An interface that returns a model's output incrementally as it is produced, rather than waiting for the full response and sending it in one block. Tokens arrive over a persistent connectio...read more

The vendor prints a rate per million tokens on its own site, separately for the tokens a request reads and the tokens the model writes, so the cost of a workload can be calculated before a...read more

The model parameters are published for download, so the same family can be run on hardware you control instead of only through the vendor API. This is what gives a buyer somewhere to go i...read more

The context window is how much text, code or other input the model can hold in one request, counted in tokens, together with everything it has already written in that same exchange. It se...read more

Fine-tuning continues training a published model on your own labelled examples, so it adopts a format, a house style or a narrow task without that instruction being repeated in every promp...read more

The model accepts more than text in a single request: images, audio, video frames or documents alongside the prompt. Pricing is usually separate for each kind of input, and audio in parti...read more

The model can be asked to work through a problem step by step before answering, spending extra generated tokens on the working rather than the reply. Accuracy on hard problems improves, a...read more