Skip to navigation
Quick Start Guides

Databricks OpenAI

View as Markdown

AI Runtime Security can be configured to route traffic to a Databricks OpenAI instance.

  • Replace <name> with the Databricks OpenAI instance name. Example: databricks-prod.
  • Replace <route_prefix> with the route prefix for the model used. Example: /serving-endpoint/gpt-model.
  • Replace <base_url> with the URL for the instance. Example: https://upstream.provider.com.
  • Set the enrich response to true to return the upstream LLM response enriched with HiddenLayer’s detections and analysis. Set to false to disable the HiddenLayer’s detections and analysis.
  • Replace <api_key> with a Databricks API key.
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS="<name>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ROUTE_PREFIX="<route_prefix>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_BASE_URL="<base_url>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ENRICH_RESPONSE="<true|false>"
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_API_KEY="<api_key>"
Environment KeyDescriptionDefaultRequired
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKSThe name of the Databricks OpenAI instance.False
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ROUTE_PREFIXThe route prefix for the model used.False
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_BASE_URLThe base URL for the instance.False
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_ENRICH_RESPONSESet AI Runtime Security as a forward or reverse proxy.False
HL_LLM_PROXY_OPENAI_ROUTES_DATABRICKS_API_KEYDatabricks API key.False