Risks & limitations
Citation
Definition
A citation links a model claim to the source it came from. Real citations come from grounding in retrieved documents; models asked to cite from memory frequently invent plausible-looking references.
Citations serve two purposes: they let a reader verify a claim, and they let you audit the system. Both require the citation to be real.
The reliable pattern is to cite what was actually retrieved. Pass identified chunks to the model and require it to reference them by identifier, then resolve those identifiers to real sources in your application. The model never generates the reference itself.
Asking an ungrounded model for citations produces confident fabrications — correctly formatted, entirely fictional. This is one of the most consistently reproducible hallucination behaviours.
Related terms
Grounding
Grounding means tying model output to verifiable sources — retrieved documents, search results, or database records — rather than relying on parametric memory. Grounded answers can be checked.
RAG (Retrieval-Augmented Generation)
RAG retrieves relevant passages from your own documents and inserts them into the prompt before the model answers. It grounds responses in your data, cuts hallucination, and needs no retraining.
Hallucination
A hallucination is model output that is fluent and confident but factually wrong — invented citations, non-existent functions, fabricated statistics. It stems from models being trained to produce plausible text, not verified truth.
Prompt injection
Prompt injection is an attack where malicious instructions hidden in content the model processes override the developer's intended behaviour. It is the most serious unsolved security issue in LLM applications.
Lost in the middle
Lost in the middle is the tendency of models to use information at the start and end of a long context well while under-weighting material in the middle. Position affects whether the model actually uses what you provided.
Model drift
Model drift is when a hosted model's behaviour changes over time as the provider updates it, causing prompts that worked reliably to degrade without any change on your side.
Put this into practice
Understanding the term is step one. Our free courses and tools let you actually use it.