Problem Statement
PyRIT is widely used as the execution engine for AI red teaming, but currently has no native MITRE ATLAS technique mapping built into its orchestrators, scorers, or reporting layer. As noted in the CSA's Agentic AI Red Teaming Guide (2026): "MITRE ATLAS techniques can be encoded into PyRIT orchestrators and datasets; PyRIT doesn't have its own TTP ontology." This means practitioners must manually map every attack run to ATLAS techniques — an error-prone, time-consuming step that breaks the automated red teaming pipeline.
Why This Matters Now
MITRE ATLAS released v5.4.0 in February 2026, adding 14+ new agentic AI techniques (e.g., AML.T0051 Prompt Injection, AML.T0054 Jailbreak, "Publish Poisoned AI Agent Tool", "Escape to Host"). Azure AI security best practices documentation (learn.microsoft.com/azure/security/fundamentals/ai-security-best-practices) explicitly recommends testing against ATLAS tactics — yet PyRIT produces no ATLAS-tagged output that can be fed directly into Azure Monitor, Microsoft Sentinel, or compliance reports.
Proposed Solution
Add an optional atlas_technique_id metadata field to:
AttackStrategy / orchestrator configuration (user declares intent: "this scenario maps to AML.T0051")
Score output objects (auto-populated from declared mapping)
Memory/logging layer (persisted alongside prompt-response pairs)
Summary report output (ATLAS technique coverage table: which techniques were tested, pass/fail rates per technique)
Minimal Implementation Path
A lightweight YAML-based mapping file (e.g., atlas_mappings.yaml) could define which datasets/converters correspond to which ATLAS technique IDs. This wouldn't require hardcoding ATLAS logic into PyRIT's core — just a tagging layer that flows through the existing memory interface.
Azure Security Impact
This would directly enable:
Azure DevOps CI/CD gates that fail on untested ATLAS technique categories
Sentinel workbooks that visualize ATLAS technique coverage over time
Compliance evidence for Azure AI security benchmark (MCSB v2 AI-5, AI-6)
Happy to draft a reference schema for the mapping file.
Problem Statement
PyRIT is widely used as the execution engine for AI red teaming, but currently has no native MITRE ATLAS technique mapping built into its orchestrators, scorers, or reporting layer. As noted in the CSA's Agentic AI Red Teaming Guide (2026): "MITRE ATLAS techniques can be encoded into PyRIT orchestrators and datasets; PyRIT doesn't have its own TTP ontology." This means practitioners must manually map every attack run to ATLAS techniques — an error-prone, time-consuming step that breaks the automated red teaming pipeline.
Why This Matters Now
MITRE ATLAS released v5.4.0 in February 2026, adding 14+ new agentic AI techniques (e.g., AML.T0051 Prompt Injection, AML.T0054 Jailbreak, "Publish Poisoned AI Agent Tool", "Escape to Host"). Azure AI security best practices documentation (learn.microsoft.com/azure/security/fundamentals/ai-security-best-practices) explicitly recommends testing against ATLAS tactics — yet PyRIT produces no ATLAS-tagged output that can be fed directly into Azure Monitor, Microsoft Sentinel, or compliance reports.
Proposed Solution
Add an optional atlas_technique_id metadata field to:
AttackStrategy / orchestrator configuration (user declares intent: "this scenario maps to AML.T0051")
Score output objects (auto-populated from declared mapping)
Memory/logging layer (persisted alongside prompt-response pairs)
Summary report output (ATLAS technique coverage table: which techniques were tested, pass/fail rates per technique)
Minimal Implementation Path
A lightweight YAML-based mapping file (e.g., atlas_mappings.yaml) could define which datasets/converters correspond to which ATLAS technique IDs. This wouldn't require hardcoding ATLAS logic into PyRIT's core — just a tagging layer that flows through the existing memory interface.
Azure Security Impact
This would directly enable:
Azure DevOps CI/CD gates that fail on untested ATLAS technique categories
Sentinel workbooks that visualize ATLAS technique coverage over time
Compliance evidence for Azure AI security benchmark (MCSB v2 AI-5, AI-6)
Happy to draft a reference schema for the mapping file.