Cybersecurity glossary
What is a Large Language Model (LLM)?
Learn what a large language model (LLM) is, how it generates text from prompts, where it sits in modern applications, and which security risks appear when models are connected to data, tools, and users.
Definition
A Large Language Model (LLM) is a neural network trained on vast text corpora to predict and generate language. In applications, it turns prompts and retrieved context into answers, plans, or tool calls—without inherent understanding of truth, policy, or authorization.
Why large language models matter in security
A Large Language Model (LLM) is not a database, a search engine, or a policy engine. It is a probability machine for language. That is useful for summarization, coding assistance, and customer support—and dangerous when the same model is trusted to interpret untrusted text, recite private context, or trigger real-world actions.
Security work on LLMs is mostly application security with a new input channel. Attackers do not need to overflow a buffer. They write instructions that the model is eager to follow, hide those instructions in documents the model retrieves, or abuse the cost and latency of inference itself.
How an LLM application actually runs
Collect context
The app assembles a system prompt, user message, chat history, and often retrieved documents or tool results.
Tokenize and infer
Text is split into tokens. The model predicts the next tokens until it stops or hits a limit.
Decode output
Tokens become text, JSON, or a planned tool call. Fluency is not the same as correctness.
Post-process
The application may filter, format, cite sources, or execute tools based on that output.
Return or act
A reply is shown to a user, or a side effect happens in email, tickets, cloud APIs, or databases.
Learn from logs
Prompts and completions are often stored. Those logs become a sensitive store of their own.
What LLMs are good at—and what they are not
Pattern completion
They generate plausible language and code from incomplete instructions. Plausible is not verified.
No built-in authorization
The model does not know who the user is unless the application injects identity and enforces it outside the model.
Context window limits
Only a finite amount of text is visible per request. What does not fit is invisible, and what does fit can be overwritten.
Tool-shaped output
When connected to APIs, the model can request actions. That is agency, not magic—and it needs least privilege.
LLM building blocks versus security failure modes
| Building block | What it does | Typical security failure |
|---|---|---|
| Base model | Generate tokens from context | Jailbreaks, extraction, inversion, and unbounded inference cost |
| System prompt | Steer tone, role, and policy | Prompt leakage and over-reliance on secret instructions |
| Retrieval (RAG) | Ground answers in documents | Retrieval poisoning and indirect prompt injection |
| Tools and MCP servers | Let the model act on systems | Excessive agency and tool poisoning |
| Logs and evals | Debug, train, and monitor | Sensitive information disclosure in telemetry |
- Inventory every LLM: vendor, model ID, region, data classification, and which apps call it.
- Separate public chat features from assistants that can read tickets, mail, or source code.
- Treat all model-visible text (users, web pages, PDFs, tickets) as untrusted input.
- Keep authorization in application code, never only in the system prompt.
- Log prompts and completions with redaction, retention limits, and access control.
- Cap tokens, concurrency, and spend per tenant to contain unbounded consumption.
- Red-team with prompt injection, leakage, and unsafe-tool scenarios before launch.
- Watch for shadow AI: employees pasting production data into unmanaged chatbots.
The practical takeaway
A Large Language Model (LLM) predicts language from context. Value comes from wrapping it with retrieval, tools, and product UX. Risk comes from treating that wrapper as if the model understood policy.
If an LLM can see a secret or call a tool, design the feature as you would any other high-privilege interpreter: untrusted input, least privilege, output handling, and monitoring—not hope that the model “knows better.”
Related security terms
Retrieval-Augmented Generation (RAG)
A common pattern that grounds LLM answers in retrieved documents.
System Prompt
Hidden instructions that steer model behavior before user input.
Prompt Injection
The leading application-layer attack against LLM-powered systems.
Guardrail
Controls that constrain model inputs, outputs, and tool use.
Shadow AI
Unapproved LLM use that bypasses security and data controls.
Frequently asked questions
What is an LLM in simple terms?
An LLM is software that predicts the next words in a sequence. Given a prompt, it produces fluent text, code, or structured output. It does not look up a single stored answer unless the application retrieves that data for it.
Is an LLM the same as a chatbot?
No. A chatbot is a product wrapper. The LLM is the model that generates language. The wrapper adds history, retrieval, tools, logging, and user interface.
Why do security teams care about LLMs?
Once an LLM reads private data or can call tools, it becomes part of the attack surface. Prompt injection, data leakage, unbounded spend, and unsafe tool use are application risks, not science-fiction failures.
Do LLMs store everything they were trained on?
Training data is compressed into weights, not stored as a searchable database. Fragments can still surface (training data leakage), and application context can leak even when the base model is clean.
What is the difference between a base model and an application?
The base model is a general generator. The application supplies system prompts, retrieved documents, tools, memory, and authorization. Most incidents happen in that application layer.
Can you patch an LLM like a traditional CVE?
Sometimes vendors ship model or safety updates, but many failures are integration bugs: missing output encoding, overly powerful tools, or untrusted retrieval. Those require application controls, not only a new model version.
Where should teams start securing LLM features?
Inventory every model, prompt, data source, and tool; treat untrusted text as hostile input; constrain agency; and monitor cost, leakage, and unexpected tool calls.
References
Explore authoritative guidance and frameworks related to large language model (llm).
Explore every security definition
Return to the glossary to search by term, alias, starting letter, or security category.