Core Concepts

An AI model is a mathematical function shaped by patterns in its training data, not a set of hand-written rules. A large language model, for instance, has learned statistical relationships between words and concepts from vast amounts of text. Given a prompt, it predicts the most likely next piece of text, one step at a time. It doesn't "know" facts the way a database does — it has learned patterns that often produce accurate-sounding answers, but sometimes produce fluent, confident answers that are simply wrong. That distinction — pattern-matching versus lookup — is the single most useful thing to understand about how these systems work.

Training data is the (usually enormous) collection of text, images, or other content a model learns its patterns from. Whatever biases, gaps, or errors exist in that data can show up in the model's behaviour — if the data underrepresents a group, a topic, or a time period, the model's outputs will reflect that. Training data also has a cut-off: most models don't know about anything that happened after their training data was collected, unless they're explicitly connected to a live retrieval system. That's why a model can be confidently wrong about very recent events, and why it's worth checking anything time-sensitive against a current source rather than trusting the model's internal sense of "now".

Inference is what happens every time you send a prompt and get a response — the model applying what it learned during training to produce an answer for your specific input. It's distinct from training: training is the (very expensive, occasional) process of building the model in the first place; inference is the (comparatively cheap, constant) process of using it. Settings you control at inference time — like "temperature" (how random versus predictable the output is) — change the character of the response without changing the underlying model at all. Two inference runs on the identical prompt can give different answers, especially at higher temperature, which is exactly why treating any single response as definitive is risky.

A hallucination is when a model states something false with the same fluent confidence as something true — a fabricated citation, a plausible-sounding but incorrect statistic, a person or policy that doesn't exist. It happens because the model is optimising for plausible, well-formed text, not for verified truth, and it has no built-in mechanism to flag its own uncertainty unless specifically prompted to. Hallucinations are more likely on narrow, obscure, or very specific factual questions (exact dates, citations, quotes) than on broad conceptual ones. The practical defence isn't trying to spot hallucinations by "vibe" — confident wrong answers read exactly like confident right answers — it's checking anything that matters against a source of record before you rely on it.

An AI agent is a model wired up to take actions, not just produce text — it can call tools, browse, run code, or trigger other systems, often in a loop where it decides what to do next based on the result of its last action. This is powerful and also riskier than a plain chat response: a wrong text answer just sits there until a human reads it, but a wrong action taken by an agent can already have happened (an email sent, a record changed, a purchase made) before anyone reviews it. That's why meaningful agent design always includes a stop condition and a human checkpoint at the point of highest consequence, rather than letting the agent run unconstrained from goal to finished action.

Evaluation is how you find out whether an AI system is actually good enough for a given job — not by trusting a vendor's claim or a single impressive demo, but by testing it against real cases with known correct answers, including edge cases it's likely to get wrong. Good evaluation checks more than raw accuracy: does it fail safely (say "I don't know") or fail confidently (guess anyway)? Does its performance hold up on the messy, unusual inputs your real users will actually send, not just the clean examples in a demo? A system that looks impressive in a five-minute demo can still fail regularly in production — which is exactly why ongoing evaluation, not a one-off demo, is what real oversight looks like.