> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://hiddenlayer.ferndocs.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://hiddenlayer.ferndocs.com/_mcp/server. # Deploy AI Runtime Security Container on AWS These installation steps show you how to install locally into a Minikube cluster. Instructions are also valid for Kubernetes provided by Docker Desktop and a full-fledged Kubernetes cluster. Running locally with Minikube introduces extra latency due to the prompt injection classification model. Latency is not present in a full-fledged Kubernetes cluster. Before deploying Model Scanner, make sure you understand: * [Resource Requirements](/docs/products/runtime/resource_requirements) * [Hybrid and Disconnected modes](/docs/products/runtime/hybrid_disconnected) * [Proxy and Prompt Analyzer](/docs/products/runtime/modes_of_operation) > **Select OS** > > Select your operating system to view installation instructions. #### macOS 1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. * For Quay credentials and other requirements, see [Resource Requirements](/docs/products/runtime/resource_requirements). ``` cat << EOF > env HIDDENLAYER_USERNAME=%QUAY_USERNAME% HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% EOF ``` > **Special Characters** > > Special characters will require wrapping the value in a string. 2. Load the environment variables into your shell. ``` source ./env ``` > **Select Deployment Type** > > Select your deployment type to view instructions. #### Hybrid A Hybrid deployment sends metadata to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with your HiddenLayer API key and secret. * Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` #### Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` > **Policy Configuration Environment Variables** > > For other policy configuration environment variables, see [Runtime Security Configuration](/docs/products/runtime/configuration/overview). 4. Make sure Docker Desktop is running. 5. Run the following command to login to the HiddenLayer Quay repository. ``` docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io ``` 6. Run the Runtime Security container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` #### Windows 1. Create an environment variable file with Quay credentials. Manually create an `env.bat` file with the following content. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. ``` @echo off set HIDDENLAYER_USERNAME=%QUAY_USERNAME% set HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% ``` > **Special Characters** > > Special characters will require wrapping the value in a string. 2. Open a command prompt and load the environment variables into your shell. ``` env ``` > **Select Deployment Type** > > Select your deployment type to view instructions. #### Hybrid A Hybrid deployment sends metadata to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`, then copy and paste the following example into the file. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with your HiddenLayer API key and secret. * Replace `` and `` with information for your AWS instance. ```yaml namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled ``` #### Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`, then copy and paste the following example into the file. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with information for your AWS instance. ```yaml namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled ``` > **Policy Configuration Environment Variables** > > For other policy configuration environment variables, see [Runtime Security Configuration](/docs/products/runtime/configuration/overview). 4. Make sure Docker Desktop and the Kubernetes cluster are running. 5. Run the following commands to login to the HiddenLayer Quay repository. ``` docker login --username=%HIDDENLAYER_USERNAME% --password=%HIDDENLAYER_PASSWORD% quay.io ``` 6. Run the Runtime Security container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` #### Ubuntu 1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace `%QUAY_USERNAME%` and `%QUAY_PASSWORD%` with actual values. * For Quay credentials and other requirements, see [Resource Requirements](/docs/products/runtime/resource_requirements). ``` cat << EOF > env HIDDENLAYER_USERNAME=%QUAY_USERNAME% HIDDENLAYER_PASSWORD=%QUAY_PASSWORD% EOF ``` > **Special Characters** > > Special characters will require wrapping the value in a string. 2. Load the environment variables into your shell. ``` source ./env ``` > **Select Deployment Type** > > Select your deployment type to view instructions. #### Hybrid A Hybrid deployment sends metadata to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with your HiddenLayer API key and secret. * Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret HL_LLM_PROXY_CLIENT_ID= HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` #### Disconnected A Disconnected deployment doesn't send data to the HiddenLayer AI Security Platform. See [Hybrid and Disconnected Deployments](/docs/products/runtime/hybrid_disconnected) for information about each deployment type. 3. Create a file named `.env.local`. * Use the comments in the example to find configurations for your deployment needs. * File changes to note: * For the EU region, uncomment `hl_region=eu`. * Replace `` with your Runtime Security license. * Replace `` and `` with information for your AWS instance. ```yaml cat << EOF > .env.local namespace: name=aidr-genai image: ## By default, the image below is deployed; this image is for CPU deployments repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai ## To set up a GPU deployment, uncomment the line below to use the CUDA image # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda ## By default, the latest version is installed ## Uncomment the following to specify a specific version # tag=25.5.2 ## By default, hybrid connection will connect to the 'US' region ## Uncomment the following option to change to a different supported region # hl_region=eu resources: ## For CPU deployments, 8 vCPUs are recommended ## For GPU deployments, 4 vCPUs are recommended requests: cpu=8 ## For GPU deployments, uncomment this to allocate 1 GPU per instance # limits: # nvidia.com/gpu=1 ## Set number of instances for deployment replicas: min=1 max=1 ## Specify a specific node group if necessary ## For instance, GPU deployments may require specific nodes to access a GPU # nodeGroup= config: HL_LICENSE= HL_LLM_PROXY_AWS_ACCESS_KEY_ID_DEFAULT= HL_LLM_PROXY_AWS_SECRET_ACCESS_KEY_DEFAULT= ## This must match the number of vCPUs assigned above OMP_NUM_THREADS=8 ## By default, the hybrid connection type is used and requires a client id and secret # HL_LLM_PROXY_CLIENT_ID= # HL_LLM_PROXY_CLIENT_SECRET= ## Uncomment the following to disable the connection to the AI Security Platform ## in this mode, the client id and secret are not required HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled EOF ``` > **Policy Configuration Environment Variables** > > For other policy configuration environment variables, see [Runtime Security Configuration](/docs/products/runtime/configuration/overview). 4. Make sure Docker Desktop and the Kubernetes cluster are running. 5. Run the following command to login to the HiddenLayer Quay repository. ``` docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io ``` 6. Run the Runtime Security container. ``` docker run --platform linux/amd64 --env-file .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest ``` ## Health Check To check that the proxy is up and running, open a new terminal and use the following command to ping the health monitor. ``` curl http://localhost:8000/health ``` ## Generate Manifest To write the generated manifest to a file (`manifest.yml`), open a terminal and run the following command. * You must run the Runtime Security installer before generating a manifest. * Change `latest` to the Runtime Security version that you use. ``` docker run --env-file ./env -v pwd/config:/src/config quay.io/hiddenlayer/distro-enterprise-aidr-genai-installer:latest > manifest.yml ```