Overview
The OpenAI-Compatible Gateway integration connects autobotAI to any endpoint that speaks the OpenAI Chat Completions API. Use it for the LLM gateway or router your organization already runs, such as LiteLLM proxy, Portkey, Envoy AI Gateway, Kong AI Gateway, Routerly or Bifrost. It also works with inference servers (vLLM, TGI, Ollama's /v1 API) and OpenAI-compatible providers. Once connected, the gateway can back Optimus, AI Agent and AI Evaluator nodes, and Memory Spaces.
autobotAI calls POST <Base URL>/chat/completions and, when available, GET <Base URL>/models. Authentication is by API key.
Prerequisites
- The gateway's base URL, including the version path (usually
/v1). - An API key for the gateway, if it requires one.
- Network reachability: the gateway must be reachable from autobotAI. A gateway on a private network must either be exposed (for example allowlisting autobotAI's egress IPs) or used from an autobotAI workspace deployed in your own cloud account.
Steps to Set Up the Integration
Step 1: Navigate to the Integrations page and click + Add Integration.
Step 2: Under the AI section, select OpenAI-Compatible Gateway + Add.
Step 3: Fill in the fields:
Note: Fields marked with an asterisk (*) are mandatory.
- Alias: A meaningful name for this integration.
- Base URL*: The API root, e.g.
https://llm-gateway.example.com/v1. If you paste a full/chat/completionsURL, the suffix is removed. - API Key: The gateway key. Leave empty for gateways that need none.
- Auth Header Name: Keep
Authorizationto sendAuthorization: Bearer <key>. If your gateway expects the raw key in another header (for examplex-api-keyorapi-key), enter that name. The key is then sent in that header as well as in theAuthorizationheader. - Extra Headers (JSON): Optional headers sent on every request, as a JSON object. Stored encrypted. Example for Portkey:
{"x-portkey-provider": "openai"}. - Models: Optional. One model ID per line, exactly as the gateway names them. These are merged with the gateway's
/modelslist. They are required when the gateway has no/modelsendpoint, because the connection test then uses the first one. - Model Aliases: Optional. If a gateway model name hides the real model, map it as
alias=underlying-model, one per line, for examplesmart=claude-opus-5-5. autobotAI still sendssmartto the gateway, but applies the parameter rules of the underlying model (some models rejecttemperature, for example). - Verify SSL: Leave on. Turn it off only for gateways that use a private or self-signed certificate.
- Skip Test Integration: Set to Yes to save without testing the connection.
Click Create.
Step 4: Click Test in the integration's menu to confirm autobotAI can reach the gateway.
Additional Information
Gateway Examples
| Gateway | Base URL | Auth | Notes |
|---|---|---|---|
| LiteLLM proxy | https://litellm.example.com/v1 | Authorization (virtual key sk-…) | /models lists the configured model names. |
| Portkey (self-hosted) | https://portkey.example.com/v1 | Authorization (upstream provider key) | Add routing headers such as x-portkey-provider or x-portkey-config in Extra Headers. List models in Models. |
| Envoy AI Gateway | Your gateway route, e.g. https://ai-gw.example.com/v1 | Whatever API-key header your gateway's security policy uses | Key-based auth only. JWT/OIDC and mTLS are not supported. |
| Routerly | https://routerly.example.com/v1 | Authorization (sk-rt-… router token) | List models in Models. |
| vLLM / TGI / Ollama | http://host:8000/v1 | None, or Authorization if started with --api-key |
How the Connection Is Tested
GET <Base URL>/modelswith your auth headers. A2xxpasses, and401/403is reported as an authentication failure.- If
/modelsis not available, a one-token chat completion is sent to the first model in Models.
Limitations
- Only key-based authentication is supported. OAuth2 tokens, JWT exchange, mTLS and request signing are not.
- Only the Chat Completions API is used, not the OpenAI Responses API.
- OpenAI prompt-cache hints are not sent through gateways, so there is no provider-side prompt caching for gateway-routed models.
- Offensive Security scans and skill scans use only the base URL and API key. Custom auth headers, extra headers and Verify SSL = off do not apply to them.
- Memory Spaces: choose an embedding model the gateway serves. There is no default.
Troubleshooting
- "Authentication failed (HTTP 401/403)": check the key and the Auth Header Name.
- "/models returned HTTP 404": the gateway has no
/modelsroute. Add your model IDs to Models. - "TLS certificate verification failed": the gateway uses a private CA. Turn off Verify SSL.
- "Could not connect to the gateway": the URL is wrong, or the gateway is not reachable from autobotAI.
- A model fails on
temperatureortool_choice: add a Model Aliases entry mapping the gateway name to the real model ID.