Generative AI (GenAI) and Large Language Models (LLMs) are revolutionizing software across industries, from intelligent chatbots to autonomous systems that help automate business processes. Enterprises are consuming LLMs either off the shelf, through hosted apps, chatbots or building smart apps which leverage LLMs hosted on-prem or in the cloud. The impact on productivity and innovation seems boundless, but with unprecedented capabilities come new security risks that traditional application security frameworks weren’t built to address.
To help practitioners, developers and security teams navigate these emerging threats, the Open Web Application Security Project (OWASP) published its Top 10 Risks and Mitigations for LLMs and GenAI Applications. This guide is rapidly becoming the industry standard, maintained by a global community of experts and updated with evolving attack trends and defensive strategies. OWASP Gen AI Security Project
Below, we briefly summarize each of the Top 10 risks and explain why they matter, especially focusing on the risks that tie directly into data security, access control, and sensitive information leakage.
-
Prompt Injection
LLM applications interpret user-supplied text as instructions. In a prompt injection attack, an adversary crafts input that subtly (or overtly) manipulates the model’s behavior, bypassing intended prompts or policies. This can lead to unauthorized actions, disclosure of internal logic, or harmful outputs.
Why it matters: Prompt injection is consistently ranked as the #1 risk because it blurs the boundary between user input and system logic, much like SQL injection in traditional applications. Without guardrails, attackers can influence decision-making, command execution, or even data access.
Mitigations include sanitizing inputs, isolating user prompts from trusted instructions, and limiting downstream actions models can trigger.
-
Sensitive Information Disclosure
LLMs may unintentionally expose confidential or private information — either from training data, embedded context or across multi-tenant RAG (Retrieval-Augmented Generation) systems.
Examples include:
- Training leakage: Proprietary data inadvertently learned and emitted by the model.
- Context leakage: Sensitive vectors, embeddings or RAG context retrieved for one user but shown to another.
This risk is especially critical because it directly implicates data privacy, regulatory compliance (e.g., GDPR/CCPA), and intellectual property protection. Unlike traditional apps, an LLM can generate sensitive information — not just store or transmit it.
Mitigations include:
- Redacting or sanitizing sensitive content before it enters training or RAG stores.
- Applying fine-grained access controls on who can query what data.
- Using privacy-preserving training techniques like differential privacy where feasible.
The bottom line: treat LLM outputs with awareness that models are statistical generalizers, not secure vaults.
-
Supply Chain Vulnerabilities
Modern GenAI systems depend on a complex supply chain: pre-trained models, frameworks, datasets, plugins, and deployment tooling from third parties. If any of these elements are compromised, for example, through malicious code or poisoned datasets, the entire system’s integrity is at risk.
Mitigations mirror traditional supply chain best practices: vet dependencies, validate sources, sign artifacts, and continuously monitor for anomalies.
-
Data & Model Poisoning
Closely related to supply chain issues, data or model poisoning occurs when attackers insert corrupted or adversarial examples into training/fine-tuning data. The result can be backdoors, biased behavior, or hidden vulnerabilities that trigger under specific conditions.
The dynamic nature of LLM training, where models are periodically retrained or updated, makes this especially insidious. Proven mitigation strategies include:
- Strong data governance throughout the training pipeline. This may mean having stronger data governance for repositories with sensitive content.
- Auditing and analyzing datasets and training logs.
- Applying clean-label techniques and anomaly detection.
-
Improper Output Handling
LLM output is just text, but if downstream systems treat it as safe code, URLs, database queries, or commands, the consequences can be catastrophic. Examples include insecure HTML, SQL, or script injection all originating from unvalidated LLM responses.
This is one of the most security-critical areas tied directly to data security and access control:
- Blind execution of code generated by a model can lead to unauthorized access or privilege escalation.
- Return of sensitive URIs or tokenized content without filtering can expose data or credentials.
Best practices include:
- Treat LLM outputs as untrusted input — just like any user-submitted data.
- Sanitize, validate, and filter before using outputs in security-critical contexts.
Improper output handling is a reminder that AI doesn’t inherently understand security boundaries, systems integrating LLMs must enforce them rigidly.
-
Excessive Agency
LLMs increasingly control workflows and can be connected to tools, APIs, and automation systems. Excessive agency refers to scenarios where models are granted too much authority, such as automatically approving transactions, sending requests to external systems, or initiating sensitive operations without oversight.
This dovetails with traditional access controls: just because a model can make a call doesn’t mean it should. Systems should enforce role-based access, human-in-the-loop checks, and least privilege principles to ensure that model autonomy doesn’t become a liability.
-
System Prompt Leakage
Hidden system prompts, embeddings or instructions intended to govern how the LLM behaves, can themselves contain sensitive logic. If attackers uncover these, they can bypass governance mechanisms, emulate privileged behavior, or escalate attacks.
Mitigations include careful prompt design, separating system instructions from user context, and auditing for leaks.
-
Vector & Embedding Weaknesses
Vector databases and embeddings underpin many RAG systems and semantic search workflows.
Weaknesses here, from poisoned embedding stores to access control failures, can lead to data leakage or manipulated model context that returns faulty outputs.
Segmenting access to vector stores and verifying embedding integrity are emerging controls specific to GenAI security.
-
Misinformation
LLMs can confidently generate false or misleading information, so-called “hallucinations.”
While this might sound like a quality issue, it’s also a real security and business risk: misinformation can lead to poor decisions, reputational damage or even legal liabilities when relied upon in critical workflows.
Mitigations include grounding outputs with citations, confidence scoring, and human review.
-
Unbounded Consumption
Because GenAI services can be resource-intensive, attackers might induce excessive use (e.g., via complex repetitive queries), causing Denial of Service (DoS), runaway costs, or model theft via query volume analysis.
Rate limiting, quotas, and monitoring guard against these economic and operational risks.
Why You Should Care About Data Security in GenAI
While all the Top 10 risks deserve attention, those tied to data security and access controls, particularly Sensitive Information Disclosure, Improper Output Handling, and Excessive Agency, are where GenAI diverges most starkly from traditional application security.
Unlike classic software, GenAI systems:
- Generate content rather than store/serve it, meaning sensitive data can be produced accidentally.
- Blur boundaries between input and logic, making it harder to define trust perimeters.
- Interact with external systems autonomously, making privilege boundaries vital.
These characteristics demand new guardrails and solutions, including privacy-preserving training processes, access control frameworks tailored to AI workflows, output sanitization mechanisms, and governance models that include human oversight.
Organizations that treat GenAI as just another backend service risk exposing proprietary data, compromise compliance, or enable costly exploits.
TL;DR
The OWASP GenAI/LLM Top 10 is more than a checklist, it’s a framework for thinking about security in a world where software can speak, reason and act. By embracing these risks early and embedding strong data-centric security practices, teams can unlock the power of AI without exposing themselves to unnecessary threats. AI Data Security solutions which understand and analyze not just the content getting in and coming out of LLMs and smart applications but also the access controls around the sources of data connected with these models, will be the ones that can provide the most holistic solution.
