Microsoft Azure
Secure AI deployment on Microsoft Azure
The configuration we expect for Azure OpenAI, Azure AI Foundry and the services around them — and exactly how we verify it before issuing a certificate.
Quick answer
A secure Azure AI deployment uses Microsoft Entra ID and managed identities instead of API keys, keeps Azure OpenAI and Azure AI Search behind private endpoints with public access disabled, enforces content filters and Prompt Shields on every model deployment, routes traffic through an API Management gateway with token limits, trims retrieval results to each user's permissions, and sends diagnostic logs to Log Analytics with Microsoft Defender for Cloud AI threat protection enabled.
Services in scope
- Azure OpenAI in Azure AI Foundry
- Azure AI Foundry hubs & projects
- Azure AI Services
- Azure AI Search
- Azure AI Content Safety
- Azure API Management
- Azure Machine Learning
- Foundry Agent Service
- Azure Key Vault
- Storage accounts holding AI data
Legend: Required must pass for certification. Assessed is tested and reported with recommendations, and becomes required where your data classification demands it.
Identity & access
| Control | Recommended configuration | How we verify |
|---|---|---|
| Keyless authentication Required | disableLocalAuth: true on every Azure OpenAI / AI Services account. Applications authenticate with managed identities and Microsoft Entra ID tokens. | Resource property check across all subscriptions; Azure Policy assignment that denies or audits accounts with local auth enabled. |
| Least-privilege RBAC Required | Workload identities get Cognitive Services OpenAI User (inference only) on the specific resource. No Owner, Contributor or Cognitive Services Contributor for application identities. | Enumeration of role assignments at resource, resource group, subscription and management group scope. |
| Privileged access Required | Admin roles granted just-in-time through Microsoft Entra Privileged Identity Management. Conditional Access requires phishing-resistant MFA for Azure portal and AI Foundry. | PIM eligible vs. active assignment review; Conditional Access policy review. |
| Secrets management Required | Any remaining secrets (third-party APIs, connectors) live in Azure Key Vault with RBAC, soft delete and purge protection, and are rotated. No keys in app settings, code or notebooks. | Review of App Service / Functions / Container Apps settings, Foundry connections and Key Vault configuration. |
Network isolation
| Control | Recommended configuration | How we verify |
|---|---|---|
| Private endpoints Required | publicNetworkAccess: Disabled on Azure OpenAI / AI Services, AI Search, Key Vault and AI storage accounts, reached through private endpoints. | Resource property checks; network ACL review; test connection from the public internet. |
| Private DNS Assessed | Private DNS zones such as privatelink.openai.azure.com, privatelink.cognitiveservices.azure.com and privatelink.search.windows.net linked to workload VNets. | DNS zone and VNet link review. |
| Foundry managed network Assessed | Foundry hubs and Azure ML workspaces use a managed virtual network in Allow only approved outbound mode. | Workspace network configuration and outbound rule review. |
| Outbound restriction Assessed | restrictOutboundNetworkAccess: true with an explicit allowedFqdnList where the service supports it. | Resource property check. |
| Edge protection Assessed | Public AI applications sit behind Azure Front Door or Application Gateway with WAF in prevention mode. | WAF policy and mode review. |
Data protection
| Control | Recommended configuration | How we verify |
|---|---|---|
| Data residency Required | Deployment types (Global, Data Zone or regional Standard) chosen deliberately to match where data may be processed. | Model deployment inventory compared with your documented residency requirements. |
| Customer-managed keys Assessed | CMK in Azure Key Vault for AI Services, AI Search and storage holding fine-tuning or RAG data. Required for regulated data. | Encryption settings on each resource; Key Vault key policy review. |
| Retention & abuse monitoring Assessed | Documented decision on abuse-monitoring data retention; uploaded files, stored completions, threads and fine-tuning data are lifecycle-managed. | Documentation review; inventory of stored files and data in each resource. |
| Sensitive data visibility Assessed | Microsoft Purview Data Security Posture Management for AI monitors sensitive data in AI interactions. | Purview configuration review. |
Guardrails & content safety
| Control | Recommended configuration | How we verify |
|---|---|---|
| Content filters Required | Every model deployment has a content filter policy attached. Any relaxed configuration is documented and approved. | Deployment-by-deployment filter review; adversarial testing of the live application. |
| Prompt Shields Required | Prompt Shields for user prompt attacks enabled, and for document (indirect) attacks wherever the model reads untrusted content: email, web pages, uploads, RAG. | Filter configuration review plus direct and indirect injection testing. |
| Grounding & protected material Assessed | Groundedness detection and protected material detection for customer-facing generation. | Configuration review and output testing. |
AI gateway & consumption
| Control | Recommended configuration | How we verify |
|---|---|---|
| Rate & token limits Required | Applications reach models through Azure API Management (AI gateway) using managed identity to the backend, with per-consumer llm-token-limit policies. | APIM policy review; load testing within agreed limits. |
| Usage telemetry & budgets Assessed | llm-emit-token-metric per consumer; Azure Cost Management budgets and alerts on AI resource groups. | Metrics and budget alert review. |
Retrieval (Azure AI Search)
| Control | Recommended configuration | How we verify |
|---|---|---|
| Keyless search Required | API keys disabled on search services; apps use RBAC roles such as Search Index Data Reader. | Search service authentication settings and role assignments. |
| Security trimming Required | Document-level permissions enforced on every query using the signed-in user's identity and group claims — never values supplied by the model or the prompt. | Index schema and query code review; cross-user and cross-tenant retrieval testing. |
| Secure indexers Assessed | Indexers connect to data sources with managed identity over shared private links. Source permissions are synced. | Indexer and data source configuration review. |
Agents & tools
| Control | Recommended configuration | How we verify |
|---|---|---|
| Least-privilege tools Required | Agent tools (functions, OpenAPI tools, Logic Apps) run as dedicated managed identities scoped to only the actions they perform. | Tool inventory and identity permission review. |
| Human approval Required | Payments, deletions, outbound messages and account changes require user confirmation or server-side authorization. | Code and flow review; agent abuse testing. |
| Grounding sources Assessed | Web grounding, code interpreter and file search are enabled only where justified and documented. | Agent configuration review. |
Logging & threat detection
| Control | Recommended configuration | How we verify |
|---|---|---|
| Diagnostic logging Required | Diagnostic settings on all AI resources send audit and request logs to Log Analytics, with defined retention and restricted access. | Diagnostic settings review on every in-scope resource. |
| AI threat protection Required | Microsoft Defender for Cloud threat protection for AI services enabled on subscriptions running AI. Alerts routed to your SOC or Microsoft Sentinel. | Defender plan status and alert routing review. |
| AI posture & policy Assessed | Defender CSPM AI security posture management for AI inventory; Azure Policy assignments for AI services guardrails. | Recommendation and policy compliance review. |
Reference configuration
A minimal Bicep definition for an Azure OpenAI account that meets the identity and network baseline:
resource aoai 'Microsoft.CognitiveServices/accounts@2024-10-01' = { name: 'contoso-aoai-prod' location: location kind: 'OpenAI' sku: { name: 'S0' } identity: { type: 'SystemAssigned' } properties: { customSubDomainName: 'contoso-aoai-prod' disableLocalAuth: true // Entra ID only — no API keys publicNetworkAccess: 'Disabled' // private endpoints only networkAcls: { defaultAction: 'Deny' } restrictOutboundNetworkAccess: true } }
And an API Management policy that authenticates with managed identity and applies per-subscription token limits:
<inbound> <base /> <authentication-managed-identity resource="https://cognitiveservices.azure.com" /> <llm-token-limit counter-key="@(context.Subscription.Id)" tokens-per-minute="20000" estimate-prompt-tokens="true" /> <llm-emit-token-metric namespace="ai-usage"> <dimension name="Subscription ID" /> </llm-emit-token-metric> </inbound>
Check yourself: run this Azure Resource Graph query to find AI resources that still allow keys or public access:
resources | where type in~ ('microsoft.cognitiveservices/accounts', 'microsoft.search/searchservices') | project name, type, subscriptionId, location, publicNetworkAccess = tostring(properties.publicNetworkAccess), disableLocalAuth = tobool(properties.disableLocalAuth) | where publicNetworkAccess != 'Disabled' or disableLocalAuth != true
How we certify an Azure deployment
- Read-only accessYou grant a time-boxed identity the built-in Reader and Security Reader roles on the in-scope management groups or subscriptions. No keys, no write access.
- Full AI inventoryAzure Resource Graph discovery of every AI Services, Azure OpenAI, AI Search and Machine Learning resource across all subscriptions. This catches "shadow AI" resources nobody listed.
- Configuration auditEvery resource is checked against this baseline, cross-referenced with Azure Policy compliance and Defender for Cloud recommendations.
- Attack testingAdversarial testing of the applications and agents running on the infrastructure: prompt injection, data leakage, retrieval trimming and tool abuse.
- Remediate, verify, certifyYou receive findings with exact settings to change. We re-verify, issue your certificate and evidence pack, and re-audit every quarter.
Frequently asked questions
What is the most important Azure OpenAI security setting?
Disabling local (API key) authentication with disableLocalAuth: true and using Microsoft Entra ID with managed identities instead. Keys are long-lived, easily leaked and can't be tied to a specific workload; managed identities remove secrets entirely and let you apply least-privilege Azure RBAC.
Does Azure OpenAI use our prompts to train models?
According to Microsoft's published data, privacy and security documentation for Azure OpenAI, prompts and completions are not used to train OpenAI or Microsoft foundation models. Data may be retained for a limited period for abuse monitoring unless your organization is approved for modified abuse monitoring. We verify that your configuration and documented decisions match your data-handling obligations.
Should Azure OpenAI have public network access disabled?
For production workloads, yes. Set publicNetworkAccess to Disabled, reach the service through private endpoints, and expose AI to users only through your application or an API Management gateway. This removes the model endpoint from the internet entirely.
What access does CertiSafe AI need to certify our Azure environment?
A time-boxed identity with the built-in Reader and Security Reader roles on the in-scope management groups or subscriptions. We don't need write access, keys, or access to your prompt and completion content.
Certify your Azure AI deployment
Most Azure environments can be audited within two weeks of a scoping call.