Integrations / AI models

OpenAI-Compatible Gateway

Updated 5 Nov 20264 min read

Overview

The OpenAI-Compatible Gateway integration connects autobotAI to any endpoint that speaks the OpenAI Chat Completions API. Use it for the LLM gateway or router your organization already runs, such as LiteLLM proxy, Portkey, Envoy AI Gateway, Kong AI Gateway, Routerly or Bifrost. It also works with inference servers (vLLM, TGI, Ollama's /v1 API) and OpenAI-compatible providers. Once connected, the gateway can back Optimus, AI Agent and AI Evaluator nodes, and Memory Spaces.

autobotAI calls POST <Base URL>/chat/completions and, when available, GET <Base URL>/models. Authentication is by API key.

Prerequisites

  • The gateway's base URL, including the version path (usually /v1).
  • An API key for the gateway, if it requires one.
  • Network reachability: the gateway must be reachable from autobotAI. A gateway on a private network must either be exposed (for example allowlisting autobotAI's egress IPs) or used from an autobotAI workspace deployed in your own cloud account.

Steps to Set Up the Integration

Step 1: Navigate to the Integrations page and click + Add Integration.

Step 2: Under the AI section, select OpenAI-Compatible Gateway + Add.

Step 3: Fill in the fields:

Note: Fields marked with an asterisk (*) are mandatory.

  • Alias: A meaningful name for this integration.
  • Base URL*: The API root, e.g. https://llm-gateway.example.com/v1. If you paste a full /chat/completions URL, the suffix is removed.
  • API Key: The gateway key. Leave empty for gateways that need none.
  • Auth Header Name: Keep Authorization to send Authorization: Bearer <key>. If your gateway expects the raw key in another header (for example x-api-key or api-key), enter that name. The key is then sent in that header as well as in the Authorization header.
  • Extra Headers (JSON): Optional headers sent on every request, as a JSON object. Stored encrypted. Example for Portkey: {"x-portkey-provider": "openai"}.
  • Models: Optional. One model ID per line, exactly as the gateway names them. These are merged with the gateway's /models list. They are required when the gateway has no /models endpoint, because the connection test then uses the first one.
  • Model Aliases: Optional. If a gateway model name hides the real model, map it as alias=underlying-model, one per line, for example smart=claude-opus-5-5. autobotAI still sends smart to the gateway, but applies the parameter rules of the underlying model (some models reject temperature, for example).
  • Verify SSL: Leave on. Turn it off only for gateways that use a private or self-signed certificate.
  • Skip Test Integration: Set to Yes to save without testing the connection.

Click Create.

Step 4: Click Test in the integration's menu to confirm autobotAI can reach the gateway.

Additional Information

Gateway Examples

GatewayBase URLAuthNotes
LiteLLM proxyhttps://litellm.example.com/v1Authorization (virtual key sk-…)/models lists the configured model names.
Portkey (self-hosted)https://portkey.example.com/v1Authorization (upstream provider key)Add routing headers such as x-portkey-provider or x-portkey-config in Extra Headers. List models in Models.
Envoy AI GatewayYour gateway route, e.g. https://ai-gw.example.com/v1Whatever API-key header your gateway's security policy usesKey-based auth only. JWT/OIDC and mTLS are not supported.
Routerlyhttps://routerly.example.com/v1Authorization (sk-rt-… router token)List models in Models.
vLLM / TGI / Ollamahttp://host:8000/v1None, or Authorization if started with --api-key

How the Connection Is Tested

  1. GET <Base URL>/models with your auth headers. A 2xx passes, and 401/403 is reported as an authentication failure.
  2. If /models is not available, a one-token chat completion is sent to the first model in Models.

Limitations

  • Only key-based authentication is supported. OAuth2 tokens, JWT exchange, mTLS and request signing are not.
  • Only the Chat Completions API is used, not the OpenAI Responses API.
  • OpenAI prompt-cache hints are not sent through gateways, so there is no provider-side prompt caching for gateway-routed models.
  • Offensive Security scans and skill scans use only the base URL and API key. Custom auth headers, extra headers and Verify SSL = off do not apply to them.
  • Memory Spaces: choose an embedding model the gateway serves. There is no default.

Troubleshooting

  • "Authentication failed (HTTP 401/403)": check the key and the Auth Header Name.
  • "/models returned HTTP 404": the gateway has no /models route. Add your model IDs to Models.
  • "TLS certificate verification failed": the gateway uses a private CA. Turn off Verify SSL.
  • "Could not connect to the gateway": the URL is wrong, or the gateway is not reachable from autobotAI.
  • A model fails on temperature or tool_choice: add a Model Aliases entry mapping the gateway name to the real model ID.
Need a hand with this?Our engineers can walk through it with you on your own stack.