New Q1 2027 certification windows are open — lock in your audit before your next release. Schedule a meeting

Microsoft Azure

Secure AI deployment on Microsoft Azure

The configuration we expect for Azure OpenAI, Azure AI Foundry and the services around them — and exactly how we verify it before issuing a certificate.

Quick answer

A secure Azure AI deployment uses Microsoft Entra ID and managed identities instead of API keys, keeps Azure OpenAI and Azure AI Search behind private endpoints with public access disabled, enforces content filters and Prompt Shields on every model deployment, routes traffic through an API Management gateway with token limits, trims retrieval results to each user's permissions, and sends diagnostic logs to Log Analytics with Microsoft Defender for Cloud AI threat protection enabled.

Services in scope

  • Azure OpenAI in Azure AI Foundry
  • Azure AI Foundry hubs & projects
  • Azure AI Services
  • Azure AI Search
  • Azure AI Content Safety
  • Azure API Management
  • Azure Machine Learning
  • Foundry Agent Service
  • Azure Key Vault
  • Storage accounts holding AI data

Legend: Required must pass for certification. Assessed is tested and reported with recommendations, and becomes required where your data classification demands it.

Identity & access

ControlRecommended configurationHow we verify
Keyless authentication
Required
disableLocalAuth: true on every Azure OpenAI / AI Services account. Applications authenticate with managed identities and Microsoft Entra ID tokens.Resource property check across all subscriptions; Azure Policy assignment that denies or audits accounts with local auth enabled.
Least-privilege RBAC
Required
Workload identities get Cognitive Services OpenAI User (inference only) on the specific resource. No Owner, Contributor or Cognitive Services Contributor for application identities.Enumeration of role assignments at resource, resource group, subscription and management group scope.
Privileged access
Required
Admin roles granted just-in-time through Microsoft Entra Privileged Identity Management. Conditional Access requires phishing-resistant MFA for Azure portal and AI Foundry.PIM eligible vs. active assignment review; Conditional Access policy review.
Secrets management
Required
Any remaining secrets (third-party APIs, connectors) live in Azure Key Vault with RBAC, soft delete and purge protection, and are rotated. No keys in app settings, code or notebooks.Review of App Service / Functions / Container Apps settings, Foundry connections and Key Vault configuration.

Network isolation

ControlRecommended configurationHow we verify
Private endpoints
Required
publicNetworkAccess: Disabled on Azure OpenAI / AI Services, AI Search, Key Vault and AI storage accounts, reached through private endpoints.Resource property checks; network ACL review; test connection from the public internet.
Private DNS
Assessed
Private DNS zones such as privatelink.openai.azure.com, privatelink.cognitiveservices.azure.com and privatelink.search.windows.net linked to workload VNets.DNS zone and VNet link review.
Foundry managed network
Assessed
Foundry hubs and Azure ML workspaces use a managed virtual network in Allow only approved outbound mode.Workspace network configuration and outbound rule review.
Outbound restriction
Assessed
restrictOutboundNetworkAccess: true with an explicit allowedFqdnList where the service supports it.Resource property check.
Edge protection
Assessed
Public AI applications sit behind Azure Front Door or Application Gateway with WAF in prevention mode.WAF policy and mode review.

Data protection

ControlRecommended configurationHow we verify
Data residency
Required
Deployment types (Global, Data Zone or regional Standard) chosen deliberately to match where data may be processed.Model deployment inventory compared with your documented residency requirements.
Customer-managed keys
Assessed
CMK in Azure Key Vault for AI Services, AI Search and storage holding fine-tuning or RAG data. Required for regulated data.Encryption settings on each resource; Key Vault key policy review.
Retention & abuse monitoring
Assessed
Documented decision on abuse-monitoring data retention; uploaded files, stored completions, threads and fine-tuning data are lifecycle-managed.Documentation review; inventory of stored files and data in each resource.
Sensitive data visibility
Assessed
Microsoft Purview Data Security Posture Management for AI monitors sensitive data in AI interactions.Purview configuration review.

Guardrails & content safety

ControlRecommended configurationHow we verify
Content filters
Required
Every model deployment has a content filter policy attached. Any relaxed configuration is documented and approved.Deployment-by-deployment filter review; adversarial testing of the live application.
Prompt Shields
Required
Prompt Shields for user prompt attacks enabled, and for document (indirect) attacks wherever the model reads untrusted content: email, web pages, uploads, RAG.Filter configuration review plus direct and indirect injection testing.
Grounding & protected material
Assessed
Groundedness detection and protected material detection for customer-facing generation.Configuration review and output testing.

AI gateway & consumption

ControlRecommended configurationHow we verify
Rate & token limits
Required
Applications reach models through Azure API Management (AI gateway) using managed identity to the backend, with per-consumer llm-token-limit policies.APIM policy review; load testing within agreed limits.
Usage telemetry & budgets
Assessed
llm-emit-token-metric per consumer; Azure Cost Management budgets and alerts on AI resource groups.Metrics and budget alert review.

Retrieval (Azure AI Search)

ControlRecommended configurationHow we verify
Keyless search
Required
API keys disabled on search services; apps use RBAC roles such as Search Index Data Reader.Search service authentication settings and role assignments.
Security trimming
Required
Document-level permissions enforced on every query using the signed-in user's identity and group claims — never values supplied by the model or the prompt.Index schema and query code review; cross-user and cross-tenant retrieval testing.
Secure indexers
Assessed
Indexers connect to data sources with managed identity over shared private links. Source permissions are synced.Indexer and data source configuration review.

Agents & tools

ControlRecommended configurationHow we verify
Least-privilege tools
Required
Agent tools (functions, OpenAPI tools, Logic Apps) run as dedicated managed identities scoped to only the actions they perform.Tool inventory and identity permission review.
Human approval
Required
Payments, deletions, outbound messages and account changes require user confirmation or server-side authorization.Code and flow review; agent abuse testing.
Grounding sources
Assessed
Web grounding, code interpreter and file search are enabled only where justified and documented.Agent configuration review.

Logging & threat detection

ControlRecommended configurationHow we verify
Diagnostic logging
Required
Diagnostic settings on all AI resources send audit and request logs to Log Analytics, with defined retention and restricted access.Diagnostic settings review on every in-scope resource.
AI threat protection
Required
Microsoft Defender for Cloud threat protection for AI services enabled on subscriptions running AI. Alerts routed to your SOC or Microsoft Sentinel.Defender plan status and alert routing review.
AI posture & policy
Assessed
Defender CSPM AI security posture management for AI inventory; Azure Policy assignments for AI services guardrails.Recommendation and policy compliance review.

Reference configuration

A minimal Bicep definition for an Azure OpenAI account that meets the identity and network baseline:

Bicep
resource aoai 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
  name: 'contoso-aoai-prod'
  location: location
  kind: 'OpenAI'
  sku: { name: 'S0' }
  identity: { type: 'SystemAssigned' }
  properties: {
    customSubDomainName: 'contoso-aoai-prod'
    disableLocalAuth: true              // Entra ID only — no API keys
    publicNetworkAccess: 'Disabled'      // private endpoints only
    networkAcls: { defaultAction: 'Deny' }
    restrictOutboundNetworkAccess: true
  }
}

And an API Management policy that authenticates with managed identity and applies per-subscription token limits:

APIM policy
<inbound>
  <base />
  <authentication-managed-identity resource="https://cognitiveservices.azure.com" />
  <llm-token-limit counter-key="@(context.Subscription.Id)"
      tokens-per-minute="20000" estimate-prompt-tokens="true" />
  <llm-emit-token-metric namespace="ai-usage">
    <dimension name="Subscription ID" />
  </llm-emit-token-metric>
</inbound>

Check yourself: run this Azure Resource Graph query to find AI resources that still allow keys or public access:

KQL
resources
| where type in~ ('microsoft.cognitiveservices/accounts', 'microsoft.search/searchservices')
| project name, type, subscriptionId, location,
    publicNetworkAccess = tostring(properties.publicNetworkAccess),
    disableLocalAuth = tobool(properties.disableLocalAuth)
| where publicNetworkAccess != 'Disabled' or disableLocalAuth != true

How we certify an Azure deployment

  1. Read-only accessYou grant a time-boxed identity the built-in Reader and Security Reader roles on the in-scope management groups or subscriptions. No keys, no write access.
  2. Full AI inventoryAzure Resource Graph discovery of every AI Services, Azure OpenAI, AI Search and Machine Learning resource across all subscriptions. This catches "shadow AI" resources nobody listed.
  3. Configuration auditEvery resource is checked against this baseline, cross-referenced with Azure Policy compliance and Defender for Cloud recommendations.
  4. Attack testingAdversarial testing of the applications and agents running on the infrastructure: prompt injection, data leakage, retrieval trimming and tool abuse.
  5. Remediate, verify, certifyYou receive findings with exact settings to change. We re-verify, issue your certificate and evidence pack, and re-audit every quarter.

Frequently asked questions

What is the most important Azure OpenAI security setting?

Disabling local (API key) authentication with disableLocalAuth: true and using Microsoft Entra ID with managed identities instead. Keys are long-lived, easily leaked and can't be tied to a specific workload; managed identities remove secrets entirely and let you apply least-privilege Azure RBAC.

Does Azure OpenAI use our prompts to train models?

According to Microsoft's published data, privacy and security documentation for Azure OpenAI, prompts and completions are not used to train OpenAI or Microsoft foundation models. Data may be retained for a limited period for abuse monitoring unless your organization is approved for modified abuse monitoring. We verify that your configuration and documented decisions match your data-handling obligations.

Should Azure OpenAI have public network access disabled?

For production workloads, yes. Set publicNetworkAccess to Disabled, reach the service through private endpoints, and expose AI to users only through your application or an API Management gateway. This removes the model endpoint from the internet entirely.

What access does CertiSafe AI need to certify our Azure environment?

A time-boxed identity with the built-in Reader and Security Reader roles on the in-scope management groups or subscriptions. We don't need write access, keys, or access to your prompt and completion content.

Certify your Azure AI deployment

Most Azure environments can be audited within two weeks of a scoping call.

Schedule a Meeting