AI Security vs. Security of AI

A practitioner research paper on protecting AI systems, securing AI-enabled enterprises, and using AI for defence

Revision: October 2026 · Status: living document · Author: Nachiket Sathaye · Licence: CC BY-SA 4.0

Abstract

"AI security", "security of AI", "secure AI" and "AI for security" are used interchangeably in vendor material, procurement documents and board papers. They describe different problems with different owners, controls and evidence. This paper separates them and gives practitioners a working model for deciding which problem is actually on the table.

The argument has four parts.

Terminology. Security of AI is the protection of AI systems and their lifecycle. AI for security is the use of AI in defensive work. AI security is the enterprise umbrella that contains both, plus the security consequences of AI-enabled business processes and AI-enabled attackers. (Page 1)
A two-axis model. Every AI security question can be placed by what is being protected (the object) and what outcome is required (the mission). The same technology, such as a coding agent, generates obligations in several cells at once. (Page 1)
A design principle. Treat the model as an untrusted component whose output is advisory. Authority to act, access data or spend money must be enforced outside the model by deterministic controls. (Pages 2, 4, 5, 7)
An operating model. Conventional controls, AI-specific controls and governance controls need to be run as one programme, tied to evidence, and mapped to the standards that auditors and regulators already use. (Pages 8 to 11)

The paper is a synthesis. It does not report new experiments. Where a claim rests on published research or a standards document it is cited; where it is the author's analysis it is labelled as such.

How claims are labelled

Label	Meaning
[Documented]	Stated in a cited paper, standard or regulation
[Reported]	Stated in a cited secondary source (vendor, law-firm or community report); verify before relying on it
[Analysis]	The author's reasoning or recommendation; not independently evidenced

Page	Topic
1. Terminology and the Two-Axis Model	Definitions, object × mission matrix, lifecycle
2. The AI System Boundary and Assets	Components, trust boundaries, what must be protected
3. Threat Taxonomy	Training-time, inference-time, model-level and ecosystem threats
4. LLMs, Prompt Injection and RAG	Instruction/data confusion, retrieval trust
5. Agents, Identity and Excessive Agency	Tool use, delegation, approval gates
6. Supply Chain and Infrastructure	Models, datasets, packages, GPUs, cloud
7. Reference Architecture	Layers, policy enforcement, zero trust
8. Assurance: Testing, Monitoring and Response	Threat modelling, red teaming, telemetry, incident response, metrics
9. Governance, Standards and Regulation	NIST, ISO, OWASP, MITRE, SAIF, ENISA, EU AI Act
10. AI for Security and AI-Enabled Threats	Defensive use and offensive use
11. Maturity Model and Roadmap	Five levels, 30-day to 24-month plan
12. Illustrative Scenarios	Four worked, hypothetical scenarios
13. Limitations and Research Gaps	What this paper cannot claim; open problems
Appendices	Questionnaire, worksheets, test procedures, glossary
References	Verified sources

How this paper relates to the rest of the wiki

The paper explains the concepts and the sources. The other sections of the wiki turn them into standards, templates and test cases.

Paper page	Where the wiki puts it into practice
3. Threat Taxonomy	Enterprise AI Risk Register, Model Layer test cases, Training and Fine-Tuning test cases
4. LLMs, Prompt Injection and RAG	Custom AI Application Runtime Security Standard, Prompt and Context test cases, Knowledge and Retrieval test cases
5. Agents, Identity and Excessive Agency	Agentic AI Security and Tool Governance, Agent Orchestration test cases, Agent and Non-Human Identity Governance test cases
6. Supply Chain and Infrastructure	Supply Chain and Third Party test cases, Infrastructure test cases, Vendor Evaluation Master Framework
7. Reference Architecture	AI Security Control Objectives Library
8. Assurance: Testing, Monitoring and Response	AI Red Team Playbook, AI Security Audit and Evidence Checklist, Monitoring, Detection and Response test cases, AI Incident Response and Forensics test cases
9. Governance, Standards and Regulation	AI Governance Operating Model, Sovereign AI, UAE Compliance and Data Residency Requirements, GCC AI Regulatory Hub, EU AI Regulatory Hub
11. Maturity Model and Roadmap	AI Security Maturity Model
Appendices	AI Risk Review Template, AI Security Glossary and Taxonomy

Suggested citation

Sathaye, N. AI Security vs. Security of AI: A Practitioner Research Paper. AI Security Wiki, living document, revision October 2026. https://nachi065.github.io/AI-Security-Wiki/00_Foundations/

Important notice

This is practitioner research, not legal advice, certification guidance or a vendor assessment. Standards and regulations change; confirm the current text and applicability of each instrument before relying on it. AI security is an empirical field, and attack feasibility and mitigation effectiveness change quickly.

Pages in This Section

1. Terminology and the Two-Axis ModelSecurity of AI, AI for security and AI security defined, with a two-axis object and mission model for placing any AI security question. 2. The AI System Boundary and AssetsWhat sits inside an AI system boundary, the trust boundaries to draw, the assets to protect and why the model is treated as an untrusted component. 3. Threat TaxonomyAI threat taxonomy by lifecycle stage: poisoning, evasion, prompt injection, extraction and supply chain, mapped to NIST AI 100-2, OWASP and MITRE ATLAS. 4. LLMs, Prompt Injection and RAGLLM attack surface, direct and indirect prompt injection, RAG access control and hallucination as a security risk, against the OWASP LLM Top 10 2025. 5. Agents, Identity and Excessive AgencyAgentic AI risk: the OWASP Agentic Top 10, advisory output with enforced authority, non-human identity, tool design, approval gates and memory. 6. Supply Chain and InfrastructureAI supply chain and infrastructure security: models, datasets, packages, tool servers, provenance, AI bill of materials, GPUs, cloud and secrets. 7. Reference ArchitectureReference architecture for secure AI systems: layered trust boundaries, model gateway, policy enforcement point, tool gateway and telemetry. 8. Assurance: Testing, Monitoring and ResponseAI assurance: threat modelling, adversarial testing and red teaming, monitoring and detection, AI incident response and control-effectiveness metrics. 9. Governance, Standards and RegulationAI governance and standards compared: NIST AI RMF, NIST AI 100-2, ISO/IEC 42001, OWASP, MITRE ATLAS, Google SAIF, ENISA and the EU AI Act timetable. 10. AI for Security and AI-Enabled ThreatsAI for security and AI-enabled threats: SOC copilots as AI systems needing protection, and how AI changes phishing, reconnaissance and exploit timelines. 11. Maturity Model and RoadmapAI security maturity levels with typical evidence, and a roadmap from the first 30 days to 24 months. 12. Illustrative ScenariosFour hypothetical AI security scenarios: an internal LLM assistant, enterprise RAG search, a SOC copilot and an autonomous operations agent. 13. Limitations and Research GapsLimitations of the AI Security vs. Security of AI paper and open research problems in agent security, evaluation, provenance and detection. AppendicesAI security assessment questionnaire, threat-modelling worksheets for assets, attack paths and agents, and control testing procedures. ReferencesReferences for the AI Security vs. Security of AI paper: standards, regulation, community frameworks and research literature, with verification status.