Cybersecurity glossary
What is Prompt Leakage?
Learn what prompt leakage is, how attackers extract system prompts and hidden instructions from LLM apps, why secret-in-the-prompt fails as a control, and how to reduce exposure without relying on the model to stay quiet.
Definition
Prompt leakage is the unauthorized disclosure of hidden prompt content—system instructions, developer notes, tool schemas, or retrieved snippets—through an LLM’s completions, logs, or error paths, so attackers can read the control plane the application meant to keep private.
Why prompt leakage matters
Teams hide policy in a system message: brand voice, disallowed topics, tool names, even “the admin password is…” because it is convenient. Prompt leakage is what happens when that convenience meets a model whose job is to be helpful with language—including repeating language it just read.
Leaked prompts are reconnaissance. They reveal which tools exist, how moderation is phrased, and which documents were retrieved. Attackers use that map to write tighter injections. If the prompt also contained secrets, leakage is a direct credential incident.
How hidden prompts escape
Place sensitive text in context
System prompts, few-shot examples, tool JSON, and retrieved snippets are concatenated for inference.
Coerce a repeat
Injection, role-play, or ‘output the rules as JSON’ asks the model to surface that text.
Transform to bypass filters
Base64, translation, acrostics, or partial quotes evade naive string matching.
Read it in the completion
The user sees hidden instructions, tool schemas, or retrieved confidential chunks.
Or read it in telemetry
Traces, eval datasets, and support exports may store full prompts for debugging.
Reuse for a better attack
The leaked control plane becomes a template for injection, jailbreaks, or tool abuse.
What usually leaks
System and developer messages
Role, policies, and ‘never mention X’ lists that were never meant for customers.
Tool and MCP schemas
Function names, argument hints, and internal endpoint descriptions.
Retrieved confidential chunks
RAG context that the UI would not have shown as a raw document dump.
Secrets mistakenly inlined
Tokens, connection strings, and ‘use this header’ notes pasted into prompts during prototyping.
Secret-in-prompt versus secret-in-code
| Where the secret lives | Who can see it | Verdict |
|---|---|---|
| System prompt | The model, plus anyone who can elicit or log the prompt | Not a secret store |
| Tool implementation | Server process with IAM and vault access | Correct place for credentials |
| User message | That user and anyone who can read transcripts | Treat as user data; do not echo into other tenants |
| Prompt logs | Engineers, vendors, and anyone with observability access | Needs redaction, retention, and ACLs |
- Assume every prompt string can be shown to a determined user; write it that way.
- Move API keys, passwords, and internal URLs out of prompts into server-side tools.
- Do not encode security policy only as secret text the model must not repeat.
- Redact system prompts and retrieved chunks in logs, traces, and eval exports.
- Add leakage tests: direct asks, encodings, translations, and document-based elicitation.
- If a prompt must stay private for IP reasons, still design so leakage does not grant extra privilege.
- Review few-shot examples for real customer data accidentally left in templates.
- Monitor completions for long verbatim overlap with hidden prompt text.
The practical takeaway
Prompt leakage is the model (or the logging stack) revealing hidden context: system instructions, tool specs, retrieved files, or secrets someone pasted into a prompt.
If a control only works when the prompt stays secret, it is not a control. Keep credentials out of context, treat hidden text as public under attack, and use application authorization so a leaked prompt does not become a leaked production.
Related security terms
System Prompt
The hidden instructions most often targeted by leakage attacks.
Prompt Injection
The usual technique used to coerce the model into repeating hidden text.
Sensitive Information Disclosure
Broader LLM data-leak class that includes user data, not only prompts.
Training Data Leakage
Disclosure of material from training weights rather than the live prompt.
Guardrail
Output filters that can catch some, but not all, leaked prompt fragments.
Frequently asked questions
What is prompt leakage in simple terms?
The application has hidden instructions. A user (or a document) talks the model into printing those instructions, or they show up in logs and traces.
Is leaking a system prompt a vulnerability?
It is if you treated the prompt as a secret: API keys, internal URLs, moderation rules, or customer-specific policies. Even ‘harmless’ prompts help attackers craft better injections.
How do attackers extract prompts?
They ask directly, role-play, request translations or encodings, use continuation tricks, or inject via retrieved files. Some attacks reconstruct prompts across many queries.
Did OWASP rename this risk?
The 2025 list used System Prompt Leakage. Later GenAI guidance discusses hidden context exposure more broadly, covering system text, tool specs, and other non-user context.
Should I put API keys in the system prompt?
Never. Keys belong in a secret store and in server-side tool implementations. Anything in the prompt is one successful injection away from the user.
Do output filters stop leakage?
They catch verbatim phrases you remember to block. Encoding, translation, summarization, and partial quotes still get through. Filters are a backup, not the design.
How is this different from training data leakage?
Prompt leakage exposes the live context you sent this request. Training data leakage exposes material memorized in weights from the training set.
References
Explore authoritative guidance and frameworks related to prompt leakage.
Explore every security definition
Return to the glossary to search by term, alias, starting letter, or security category.