Azure API Management Integration | HiddenLayer Documentation

This guide walks through inserting HiddenLayer AI Runtime Security into an Azure API Management (APIM) pipeline to scan and act on both user inputs and model outputs (including blocking unsafe content and redacting PII) before they reach the LLM or are returned to the user.

Prerequisites

Azure

An Azure APIM instance
An API configured as a passthrough to an Azure OpenAI resource using the /responses endpoint and/or /chat/completions

HiddenLayer

Access to a HiddenLayer SaaS or containerized (hybrid) instance
A ruleset configured with the following policies:
- Prompt Injection: Block
- Code: Block (output at minimum; input optional)
- PII: Redact
A project with that ruleset attached

Note on PII redaction

This guide demonstrates redaction for custom PII entities, which can only be added to a ruleset via API. If you haven’t done that yet, standard entity redaction (e.g., <PHONE_NUMBER>) will still work. See Runtime Security Policy to learn how to configure custom entities.

Background: How APIM Policy Works

Azure APIM policy is written in a combination of XML and C#. Full documentation on policy options is available in the Microsoft APIM policy reference.

To edit policy from the Azure portal, navigate to the operation you want to moderate, then click the angle-bracket icon (</>) in either the Inbound Processing or Outbound Processing pane.

Policy is organized into four sections:

<inbound>: Runs before the request is forwarded to the backend (LLM). Use this to scan user inputs.
<backend>: Controls how requests are forwarded.
<outbound>: Runs after the backend responds. Use this to scan model outputs.
<on-error>: Handles exceptions.

Complete Policy

The policy example below obtains a SaaS auth token, scans input with HiddenLayer in <inbound> (including optional block/redact), forwards the request to Azure OpenAI, then scans the model output in <outbound>. It supports both /chat/completions-style bodies (with system-role messages filtered out before scanning) and /responses-style bodies.

The full policy can be copied into your APIM instance and assumes SaaS detection (https://api.hiddenlayer.ai). Replace placeholder named values ({{hiddenlayer-client-id}}, {{hiddenlayer-client-secret}}, {{hiddenlayer-project-id}}, {{aoai-api-key}}) before deploying. You can define these as named variables in Azure API Management and reference them with {{ }} as shown.

The sections that follow explain each piece in detail.

{/*
    HiddenLayer APIM Policy — Interactions SaaS Integration
*/}
<policies>
    <inbound>
        <base />
        {/* ========================================== */}
        {/* Timing                                     */}
        {/* ========================================== */}
        <set-variable name="request_start" value="@(DateTime.UtcNow)" />
        {/* ========================================== */}
        {/* OAuth 2.0 Client Credentials — with cache  */}
        {/* ========================================== */}
        <cache-lookup-value key="hl-access-token" variable-name="hl_cached_token" />
        <choose>
            <when condition="@(!context.Variables.ContainsKey(&quot;hl_cached_token&quot;))">
                <send-request mode="new" response-variable-name="hl_token_response" timeout="20" ignore-error="false">
                    <set-url>https://auth.hiddenlayer.ai/oauth2/token</set-url>
                    <set-method>POST</set-method>
                    <set-header name="Content-Type" exists-action="override">
                        <value>application/x-www-form-urlencoded</value>
                    </set-header>
                    <set-body>@{
                        string enc(string s) => System.Net.WebUtility.UrlEncode(s);
                        var clientId     = "{{hiddenlayer-client-id}}";
                        var clientSecret = "{{hiddenlayer-client-secret}}";
                        return $"grant_type=client_credentials&client_id={enc(clientId)}&client_secret={enc(clientSecret)}&audience=https://us.auth.hiddenlayer.ai/";
                    }</set-body>
                </send-request>
                <set-variable name="hl_token_json" value="@(((IResponse)context.Variables[&quot;hl_token_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="hl_access_token" value="@((string)((JObject)context.Variables[&quot;hl_token_json&quot;])[&quot;access_token&quot;])" />
                <set-variable name="hl_expires_in" value="@((int?)((JObject)context.Variables[&quot;hl_token_json&quot;])[&quot;expires_in&quot;])" />
                <cache-store-value key="hl-access-token" value="@((string)context.Variables[&quot;hl_access_token&quot;])" duration="@((int)context.Variables[&quot;hl_expires_in&quot;] - 60)" />
            </when>
            <otherwise>
                <set-variable name="hl_access_token" value="@((string)context.Variables[&quot;hl_cached_token&quot;])" />
            </otherwise>
        </choose>
        {/* ========================================== */}
        {/* Extract Request Metadata                   */}
        {/* ========================================== */}
        {/* correlation_id ties APIM trace, HL input event, and HL output event together */}
        <set-variable name="correlation_id" value="@(((System.Guid)context.RequestId).ToString())" />
        {/* requester_id: caller-supplied grouping key, falls back to client IP */}
        <set-variable name="requester_id" value="@{
            string xff = context.Request.Headers.GetValueOrDefault(&quot;X-Forwarded-For&quot;, &quot;&quot;);
            if (!string.IsNullOrEmpty(xff)) { return xff.Split(',')[0].Trim(); }
            return context.Request.IpAddress;
        }" />
        {/* Preserve full request body for reuse in inbound redact and outbound scan */}
        <set-variable name="original_body" value="@(context.Request.Body.As&lt;JObject&gt;(preserveContent: true))" />
        {/* Extract model name from request body for HL metadata */}
        <set-variable name="model" value="@(((JObject)context.Variables[&quot;original_body&quot;])[&quot;model&quot;]?.ToString() ?? context.Api.Name)" />
        {/*
            Capture messages array with system role filtered out at source.
            This ensures neither the input scan nor the output scan ever forwards
            the system prompt to the HiddenLayer API.
        */}
        <set-variable name="messages" value="@{
            var all = (JArray)((JObject)context.Variables[&quot;original_body&quot;])[&quot;messages&quot;];
            if (all == null) { return null; }
            var filtered = new JArray();
            foreach (var msg in all) {
                var m = msg as JObject;
                if (m == null) { continue; }
                var role = (string)m[&quot;role&quot;];
                if (!string.IsNullOrEmpty(role) && role.Equals(&quot;system&quot;, StringComparison.OrdinalIgnoreCase)) { continue; }
                filtered.Add(msg);
            }
            return filtered;
        }" />
        {/* ========================================== */}
        {/* INPUT SCAN                                 */}
        {/* ========================================== */}
        <set-variable name="hl_input_start" value="@(DateTime.UtcNow)" />
        <send-request mode="new" response-variable-name="hl_input_response" timeout="10" ignore-error="true">
            <set-url>https://api.hiddenlayer.ai/detection/v1/interactions</set-url>
            <set-method>POST</set-method>
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="Authorization" exists-action="override">
                <value>@("Bearer " + (string)context.Variables["hl_access_token"])</value>
            </set-header>
            <set-header name="HL-Project-Id" exists-action="override">
                <value>{{hiddenlayer-project-id}}</value>
            </set-header>
            <set-header name="X-Correlation-ID" exists-action="override">
                <value>@((string)context.Variables["correlation_id"])</value>
            </set-header>
            <set-header name="X-Requester-Id" exists-action="override">
                <value>@((string)context.Variables["requester_id"])</value>
            </set-header>
            <set-body>@{
                string inputText = null;
                var original = (JObject)context.Variables["original_body"];
                // /chat/completions: extract last user message from filtered messages array
                // (system messages already excluded from context.Variables["messages"])
                var messages = (JArray)context.Variables["messages"];
                if (messages != null) {
                    for (int i = messages.Count - 1; i >= 0; i--) {
                        var m = messages[i] as JObject;
                        if (m == null) { continue; }
                        var role = (string)m["role"];
                        if (!string.IsNullOrEmpty(role) && role.Equals("user", StringComparison.OrdinalIgnoreCase)) {
                            var tok = m["content"];
                            if (tok != null) {
                                inputText = tok.Type == JTokenType.String
                                    ? (string)tok
                                    : tok.ToString(Newtonsoft.Json.Formatting.None);
                            }
                            break;
                        }
                    }
                }
                // /responses endpoint: input field
                else {
                    var input = original?["input"];
                    if (input?.Type == JTokenType.String) {
                        inputText = (string)input;
                    }
                    else if (input?.Type == JTokenType.Array) {
                        var arr = input as JArray;
                        for (int i = arr.Count - 1; i >= 0; i--) {
                            var item = arr[i] as JObject;
                            if ((string)item?["role"] == "user") {
                                inputText = (string)item?["content"];
                                break;
                            }
                        }
                    }
                }
                var payload = new JObject {
                    ["metadata"] = new JObject {
                        ["model"]        = (string)context.Variables["model"],
                        ["requester_id"] = (string)context.Variables["requester_id"],
                        ["provider"]     = "azure openai"
                    },
                    ["input"] = new JObject {
                        ["messages"] = new JArray()
                    }
                };
                if (!string.IsNullOrEmpty(inputText)) {
                    ((JArray)payload["input"]["messages"]).Add(
                        new JObject { ["role"] = "user", ["content"] = inputText }
                    );
                }
                return payload.ToString(Newtonsoft.Json.Formatting.None);
            }</set-body>
        </send-request>
        <set-variable name="hl_input_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;hl_input_start&quot;]).TotalMilliseconds)" />
        {/* Process input scan result */}
        <choose>
            <when condition="@(
                context.Variables.ContainsKey(&quot;hl_input_response&quot;) &amp;&amp;
                context.Variables[&quot;hl_input_response&quot;] != null &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode >= 200 &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode &lt; 300)">
                <set-variable name="hl_input_result" value="@(((IResponse)context.Variables[&quot;hl_input_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="input_action" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;evaluation.action&quot;)?.ToString() ?? &quot;Allow&quot;)" />
                <set-variable name="input_event_id" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;metadata.event_id&quot;)?.ToString())" />
                <choose>
                    {/* BLOCK */}
                    <when condition="@(((string)context.Variables[&quot;input_action&quot;]).Equals(&quot;Block&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Inbound-SaaS" severity="error">
                            <message>HiddenLayer blocked inbound request</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;input_event_id&quot;])" />
                            <metadata name="threat_level" value="@(((JObject)context.Variables[&quot;hl_input_result&quot;])?.SelectToken(&quot;evaluation.threat_level&quot;)?.ToString())" />
                        </trace>
                        <return-response>
                            <set-status code="403" reason="Blocked by HiddenLayer Policy" />
                            <set-header name="Content-Type" exists-action="override">
                                <value>application/json</value>
                            </set-header>
                            <set-body>@{
                                var result = (JObject)context.Variables["hl_input_result"];
                                var err = new JObject {
                                    ["error"] = new JObject {
                                        ["message"] = "Request blocked by HiddenLayer security policy",
                                        ["type"]    = "blocked_by_policy",
                                        ["code"]    = "content_policy_violation",
                                        ["hiddenlayer"] = new JObject {
                                            ["correlation_id"] = (string)context.Variables["correlation_id"],
                                            ["event_id"]       = (string)context.Variables["input_event_id"],
                                            ["phase"]          = "input",
                                            ["threat_level"]   = result?.SelectToken("evaluation.threat_level")?.ToString(),
                                            ["details"]        = result?.SelectToken("analysis")
                                        }
                                    }
                                };
                                return err.ToString(Newtonsoft.Json.Formatting.None);
                            }</set-body>
                        </return-response>
                    </when>
                    {/* REDACT */}
                    <when condition="@(((string)context.Variables[&quot;input_action&quot;]).Equals(&quot;Redact&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Inbound-SaaS" severity="information">
                            <message>HiddenLayer redacted inbound request</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;input_event_id&quot;])" />
                        </trace>
                        <set-body>@{
                            var body     = (JObject)context.Variables["original_body"];
                            var result   = (JObject)context.Variables["hl_input_result"];
                            var redacted = (string)result?.SelectToken("modified_data.input.messages[0].content") ?? "";
                            // Rewrite last user message in messages array
                            var msgs = body["messages"] as JArray;
                            if (msgs != null) {
                                for (int i = msgs.Count - 1; i >= 0; i--) {
                                    var m = msgs[i] as JObject;
                                    if (m != null && string.Equals((string)m["role"], "user", StringComparison.OrdinalIgnoreCase)) {
                                        m["content"] = redacted;
                                        break;
                                    }
                                }
                            }
                            // /responses format fallback
                            else if (body["input"] != null) {
                                if (body["input"].Type == JTokenType.String) {
                                    body["input"] = redacted;
                                }
                                else if (body["input"] is JArray arr) {
                                    for (int i = arr.Count - 1; i >= 0; i--) {
                                        var item = arr[i] as JObject;
                                        if ((string)item?["role"] == "user") { item["content"] = redacted; break; }
                                    }
                                }
                            }
                            return body.ToString(Newtonsoft.Json.Formatting.None);
                        }</set-body>
                    </when>
                    {/* ALLOW: forward unchanged */}
                    <otherwise />
                </choose>
            </when>
            {/* HL error / timeout: fail-open */}
            <otherwise>
                <trace source="HiddenLayer-Inbound-SaaS" severity="error">
                    <message>HiddenLayer input scan failed or timed out — proceeding fail-open</message>
                    <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                    <metadata name="status_code" value="@(context.Variables.ContainsKey(&quot;hl_input_response&quot;) &amp;&amp; context.Variables[&quot;hl_input_response&quot;] != null ? ((IResponse)context.Variables[&quot;hl_input_response&quot;]).StatusCode.ToString() : &quot;null&quot;)" />
                </trace>
            </otherwise>
        </choose>
        {/* Azure OpenAI auth — set after HL check to avoid sending to HiddenLayer */}
        <set-header name="Authorization" exists-action="override">
            <value>@("Bearer " + "{{aoai-api-key}}")</value>
        </set-header>
        {/* Strip APIM/HL headers before forwarding to backend */}
        <set-header name="Ocp-Apim-Subscription-Key" exists-action="delete" />
        <set-header name="X-Requester-Id" exists-action="delete" />
        <set-variable name="backend_start" value="@(DateTime.UtcNow)" />
    </inbound>
    <backend>
        <base />
    </backend>
    <outbound>
        <base />
        <set-variable name="backend_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;backend_start&quot;]).TotalMilliseconds)" />
        {/* Preserve full OpenAI response for redact rewrite if needed */}
        <set-variable name="openai_response" value="@(context.Response.Body.As&lt;JObject&gt;(preserveContent: true))" />
        <set-variable name="assistant_message" value="@(((JObject)context.Variables[&quot;openai_response&quot;])?.SelectToken(&quot;choices[0].message&quot;))" />
        {/* ========================================== */}
        {/* OUTPUT SCAN                                */}
        {/* Only the latest assistant message is sent  */}
        {/* to HiddenLayer — no history, no system     */}
        {/* prompt, no extra OpenAI fields.            */}
        {/* ========================================== */}
        <set-variable name="hl_output_start" value="@(DateTime.UtcNow)" />
        <send-request mode="new" response-variable-name="hl_output_response" timeout="10" ignore-error="true">
            <set-url>https://api.hiddenlayer.ai/detection/v1/interactions</set-url>
            <set-method>POST</set-method>
            <set-header name="Content-Type" exists-action="override">
                <value>application/json</value>
            </set-header>
            <set-header name="Authorization" exists-action="override">
                <value>@("Bearer " + (string)context.Variables["hl_access_token"])</value>
            </set-header>
            <set-header name="HL-Project-Id" exists-action="override">
                <value>{{hiddenlayer-project-id}}</value>
            </set-header>
            <set-header name="X-Correlation-ID" exists-action="override">
                <value>@((string)context.Variables["correlation_id"])</value>
            </set-header>
            <set-header name="X-Requester-Id" exists-action="override">
                <value>@(context.Variables.ContainsKey("requester_id") ? (string)context.Variables["requester_id"] : context.Request.IpAddress)</value>
            </set-header>
            <set-body>@{
                // Send only the latest assistant message, normalized to {role, content}.
                // Strips extra OpenAI fields (tool_calls, refusal, function_call, etc.)
                // and excludes all conversation history and system prompt.
                var assistantMessage = context.Variables["assistant_message"] as JObject;
                var outputMessages = new JArray();
                if (assistantMessage != null) {
                    outputMessages.Add(new JObject {
                        ["role"]    = "assistant",
                        ["content"] = assistantMessage["content"]?.ToString() ?? ""
                    });
                }
                var payload = new JObject {
                    ["metadata"] = new JObject {
                        ["model"]        = (string)context.Variables["model"],
                        ["requester_id"] = context.Variables.ContainsKey("requester_id")
                                               ? (string)context.Variables["requester_id"]
                                               : context.Request.IpAddress,
                        ["provider"]     = "azure openai"
                    },
                    ["output"] = new JObject {
                        ["messages"] = outputMessages
                    }
                };
                return payload.ToString(Newtonsoft.Json.Formatting.None);
            }</set-body>
        </send-request>
        <set-variable name="hl_output_duration_ms" value="@((DateTime.UtcNow - (DateTime)context.Variables[&quot;hl_output_start&quot;]).TotalMilliseconds)" />
        {/* Process output scan result */}
        <choose>
            <when condition="@(
                context.Variables.ContainsKey(&quot;hl_output_response&quot;) &amp;&amp;
                context.Variables[&quot;hl_output_response&quot;] != null &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode >= 200 &amp;&amp;
                ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode &lt; 300)">
                <set-variable name="hl_output_result" value="@(((IResponse)context.Variables[&quot;hl_output_response&quot;]).Body.As&lt;JObject&gt;())" />
                <set-variable name="output_action" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;evaluation.action&quot;)?.ToString() ?? &quot;Allow&quot;)" />
                <set-variable name="output_event_id" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;metadata.event_id&quot;)?.ToString())" />
                <choose>
                    {/* BLOCK */}
                    <when condition="@(((string)context.Variables[&quot;output_action&quot;]).Equals(&quot;Block&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Outbound-SaaS" severity="error">
                            <message>HiddenLayer blocked outbound response</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;output_event_id&quot;])" />
                            <metadata name="threat_level" value="@(((JObject)context.Variables[&quot;hl_output_result&quot;])?.SelectToken(&quot;evaluation.threat_level&quot;)?.ToString())" />
                        </trace>
                        <return-response>
                            <set-status code="403" reason="Response Blocked by HiddenLayer Policy" />
                            <set-header name="Content-Type" exists-action="override">
                                <value>application/json</value>
                            </set-header>
                            <set-body>@{
                                var result = (JObject)context.Variables["hl_output_result"];
                                var err = new JObject {
                                    ["error"] = new JObject {
                                        ["message"] = "Response blocked by HiddenLayer security policy",
                                        ["type"]    = "response_blocked_by_policy",
                                        ["code"]    = "content_policy_violation",
                                        ["hiddenlayer"] = new JObject {
                                            ["correlation_id"] = (string)context.Variables["correlation_id"],
                                            ["event_id"]       = (string)context.Variables["output_event_id"],
                                            ["phase"]          = "output",
                                            ["threat_level"]   = result?.SelectToken("evaluation.threat_level")?.ToString(),
                                            ["details"]        = result?.SelectToken("analysis")
                                        }
                                    }
                                };
                                return err.ToString(Newtonsoft.Json.Formatting.None);
                            }</set-body>
                        </return-response>
                    </when>
                    {/* REDACT */}
                    <when condition="@(((string)context.Variables[&quot;output_action&quot;]).Equals(&quot;Redact&quot;, StringComparison.OrdinalIgnoreCase))">
                        <trace source="HiddenLayer-Outbound-SaaS" severity="information">
                            <message>HiddenLayer redacted outbound response</message>
                            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                            <metadata name="event_id" value="@((string)context.Variables[&quot;output_event_id&quot;])" />
                        </trace>
                        <set-body>@{
                            var response = (JObject)context.Variables["openai_response"];
                            var result   = (JObject)context.Variables["hl_output_result"];
                            // modified_data.output.messages contains the redacted assistant turn
                            var redactedMsgs = result?.SelectToken("modified_data.output.messages") as JArray;
                            if (redactedMsgs != null && redactedMsgs.Count > 0) {
                                var safeMsg = redactedMsgs[redactedMsgs.Count - 1] as JObject;
                                var choices = response?["choices"] as JArray;
                                if (choices != null && choices.Count > 0) {
                                    var firstChoice = choices[0] as JObject;
                                    if (firstChoice != null && safeMsg != null) {
                                        firstChoice["message"] = safeMsg;
                                    }
                                }
                            }
                            return response.ToString(Newtonsoft.Json.Formatting.None);
                        }</set-body>
                    </when>
                    {/* ALLOW: return response unchanged */}
                    <otherwise />
                </choose>
            </when>
            {/* HL error / timeout: fail-open */}
            <otherwise>
                <trace source="HiddenLayer-Outbound-SaaS" severity="error">
                    <message>HiddenLayer output scan failed or timed out — returning original response fail-open</message>
                    <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
                    <metadata name="status_code" value="@(context.Variables.ContainsKey(&quot;hl_output_response&quot;) &amp;&amp; context.Variables[&quot;hl_output_response&quot;] != null ? ((IResponse)context.Variables[&quot;hl_output_response&quot;]).StatusCode.ToString() : &quot;null&quot;)" />
                </trace>
            </otherwise>
        </choose>
        {/* ========================================== */}
        {/* Timing Headers                             */}
        {/* ========================================== */}
        <set-variable name="total_duration_ms" value="@(context.Elapsed.TotalMilliseconds)" />
        <set-variable name="hl_total_ms" value="@((double)context.Variables[&quot;hl_input_duration_ms&quot;] + (double)context.Variables[&quot;hl_output_duration_ms&quot;])" />
        <set-variable name="apim_overhead_ms" value="@((double)context.Variables[&quot;total_duration_ms&quot;] - (double)context.Variables[&quot;hl_total_ms&quot;] - (double)context.Variables[&quot;backend_duration_ms&quot;])" />
        <trace source="HiddenLayer-Timing" severity="information">
            <message>Request timing summary</message>
            <metadata name="correlation_id" value="@((string)context.Variables[&quot;correlation_id&quot;])" />
            <metadata name="total_ms" value="@(((double)context.Variables[&quot;total_duration_ms&quot;]).ToString())" />
            <metadata name="hl_input_ms" value="@(((double)context.Variables[&quot;hl_input_duration_ms&quot;]).ToString())" />
            <metadata name="hl_output_ms" value="@(((double)context.Variables[&quot;hl_output_duration_ms&quot;]).ToString())" />
            <metadata name="hl_total_ms" value="@(((double)context.Variables[&quot;hl_total_ms&quot;]).ToString())" />
            <metadata name="openai_ms" value="@(((double)context.Variables[&quot;backend_duration_ms&quot;]).ToString())" />
            <metadata name="apim_overhead_ms" value="@(((double)context.Variables[&quot;apim_overhead_ms&quot;]).ToString())" />
            <metadata name="status_code" value="@(context.Response.StatusCode.ToString())" />
        </trace>
        <set-header name="X-Correlation-ID" exists-action="override">
            <value>@((string)context.Variables["correlation_id"])</value>
        </set-header>
        <set-header name="X-Total-Duration-Ms" exists-action="override">
            <value>@(((double)context.Variables["total_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-HiddenLayer-Input-Ms" exists-action="override">
            <value>@(((double)context.Variables["hl_input_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-HiddenLayer-Output-Ms" exists-action="override">
            <value>@(((double)context.Variables["hl_output_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-OpenAI-Ms" exists-action="override">
            <value>@(((double)context.Variables["backend_duration_ms"]).ToString())</value>
        </set-header>
        <set-header name="X-APIM-Overhead-Ms" exists-action="override">
            <value>@(((double)context.Variables["apim_overhead_ms"]).ToString())</value>
        </set-header>
    </outbound>
    <on-error>
        <base />
        <trace source="HiddenLayer-Policy-Error" severity="error">
            <message>@("Unhandled policy error: " + (context.LastError?.Message ?? "unknown"))</message>
            <metadata name="correlation_id" value="@(context.Variables.ContainsKey(&quot;correlation_id&quot;) ? (string)context.Variables[&quot;correlation_id&quot;] : &quot;not-set&quot;)" />
        </trace>
    </on-error>
</policies>

How It Works: Section by Section

Calling HiddenLayer Inbound (User Input)

The inbound section intercepts the user’s request before it reaches the LLM and POSTs to HiddenLayer’s /detection/v1/interactions endpoint.

For /chat/completions requests, the policy builds a filtered copy of messages that omits system role entries (so the system prompt is never sent to HiddenLayer). It scans the latest user message content. For /responses-style bodies, it reads the input field (string or message array) instead.

Requests to HiddenLayer include Authorization (Bearer token from SaaS OAuth), HL-Project-Id, X-Correlation-ID (APIM RequestId), and X-Requester-Id (first hop from X-Forwarded-For, else client IP).

Responding Based on the HiddenLayer Evaluation

The policy reads evaluation.action from the JSON response (Block, Redact, or allow).

If HiddenLayer returns a non-2xx status or times out, the policy fails open: it traces the failure and forwards the original request to Azure OpenAI.

If the evaluation action is Block, the caller receives 403 with a JSON error body that includes correlation and event metadata.

If the evaluation action is Redact, the inbound body is rewritten using modified_data.input.messages[0].content (for chat completions, the last user message is updated; for /responses, the input field is updated accordingly).

After the HiddenLayer inbound check completes, the policy sets the Azure OpenAI Authorization header ({{aoai-api-key}}), removes Ocp-Apim-Subscription-Key and X-Requester-Id, and records timing for the backend call.

Calling HiddenLayer Outbound (Model Output)

The outbound section reads the assistant turn from choices[0].message (chat completions shape), sends only that normalized message to /detection/v1/interactions, and applies the same Block (403) / Redact / fail-open behavior. Redaction merges modified_data.output.messages back into choices[0].message.

Timing / correlation headers (X-Correlation-ID, X-HiddenLayer-*-Ms, etc.) are added on the response for observability.

Appendix

Correlating HiddenLayer Logs with APIM Logs

The complete policy above already stores APIM’s RequestId as correlation_id and sends X-Correlation-ID on HiddenLayer requests and on the client response. That matches gateway logs without extra steps.

If you are building a smaller policy or fragment by hand, you can add the same pattern explicitly:

<set-variable name="corrId" value="@(((System.Guid)context.RequestId).ToString())" />

<set-header name="X-Correlation-ID" exists-action="override">
  <value>@((string)context.Variables["corrId"])</value>
</set-header>

APIM Policy Fragments

Policy fragments are centrally managed, reusable XML code snippets that can enable consistent integration with AI Runtime across the APIM environment. For more information from Microsoft, see Reuse Policy Configurations in API Management.

It is common to maintain the inbound HiddenLayer logic (OAuth, input scan, Azure OpenAI headers) and the outbound HiddenLayer logic (output scan, timing headers) as separate fragments. That lets you reuse the HiddenLayer pieces across APIs while composing them with unrelated APIM policy (rate limiting, caching, JWT validation, transformation steps, or other backends) in the order your operation requires.

The OAuth token cache logic shown in the inbound section above can be extracted into a fragment (for example, named hiddenlayer-saas-auth) and referenced in any policy using:

<include-fragment fragment-id="hiddenlayer-saas-auth" />

Example usage in a policy:

<policies>
  <inbound>
    <include-fragment fragment-id="hiddenlayer-saas-auth" />
    [...]
  </inbound>
  [...]
</policies>