Model gateway (LiteLLM)

A model gateway sits between your agents and the model providers. Routing through one lets you use virtual keys with budgets, map Claude Code's model aliases to the models your proxy serves, and attribute spend to projects and tasks. Configure it once in Settings, then turn it on for every project or opt in per project.

Set up the gateway#

1. Open Settings → Model Gateway

   Fill in:

   • Base URL, for example http://localhost:4000.
   • Virtual key (sk-…), stored in the Keychain.
   • Auth header: Authorization: Bearer (sets ANTHROPIC_AUTH_TOKEN) or x-api-key (sets ANTHROPIC_API_KEY).
2. Test the connection

   Press Test Connection. SI Fabrik calls GET <base>/v1/models and shows the latency and how many models the proxy lists. Those models then appear in the slot pickers.

3. Map the model slots

   Under Claude Code, set the Opus, Sonnet and Haiku (background tasks) slots to model names on your proxy. Under Codex, set the Default model and Wire API (Responses or Chat Completions).

   Set the Haiku slot if your proxy doesn't serve a Haiku model: Claude Code uses it for titles and summaries.

4. Choose the default

   Turn on Route all agents through this gateway by default, or leave it off and opt in per project. Press Save.

Under Advanced:

• Tag requests for spend tracking adds x-litellm-tags: harness,project:<name>,issue:<id> to every request (on by default).
• Disable experimental betas fixes proxies that answer 400 "Extra inputs are not permitted".

Per project#

gateway:
  enabled: true                       # false opts this project out of the default
  base_url: http://localhost:4000     # optional: overrides Settings
  api_key: $LITELLM_API_KEY           # optional: overrides Settings (use $VAR, never an inline key)
  auth_header: bearer                 # bearer → ANTHROPIC_AUTH_TOKEN | x-api-key → ANTHROPIC_API_KEY
  models: { opus: claude-opus-5-5, sonnet: claude-sonnet-5-5, haiku: claude-haiku-5-5, subagent: claude-sonnet-5-5 }
  headers: { X-Team: platform }
  tags: true
  disable_experimental_betas: false
  wire_api: responses                 # Codex: responses | chat

Resolution rules:

• gateway.enabled: false always wins: no gateway for this project.
• Otherwise a gateway: block, or the Settings default, turns it on.
• Each field you set in WORKFLOW.md overrides the Settings value; the rest come from Settings.
• A gateway that's on but has no base URL anywhere blocks dispatch with gateway_unconfigured.

In Workflow → Settings → Model gateway, Route through gateway offers Follow Settings, On and Off, plus the base URL and model slots.

What the agent receives#

On the host, Claude Code gets:

Anthropic credentials are removed from the agent's environment so the two never conflict. Anthropic supports Claude Code with Claude models only.

In a sandbox, ANTHROPIC_BASE_URL points at SI Fabrik's relay instead and the key variable holds a placeholder; the relay adds the gateway key and forwards to the base URL. The slots, headers and tags are the same.

On the host, Codex gets the gateway registered as a provider:

codex app-server \
  -c 'model_providers.litellm={ name = "LiteLLM", base_url = "http://localhost:4000/v1", env_key = "LITELLM_API_KEY", wire_api = "responses" }' \
  -c 'model_provider="litellm"'

In a sandbox, Codex gets a harness provider instead, pointed at the relay with a placeholder key, with the gateway's wire API and headers.

Docker, Cloudflare and private gateways#

• Docker: model traffic goes to the egress proxy's relay, which adds the key and forwards to the gateway; the key never enters the container, and the gateway's host isn't added to the sandbox's allowlist. localhost and 127.0.0.1 in the base URL are rewritten to host.docker.internal, so a gateway on the same machine works.
• Cloudflare: model traffic goes to the relay address in the sandbox and SI Fabrik forwards it from the host, adding the real key there. A LiteLLM proxy on your VPN works without exposing it.
• Remote mode: set the gateway in harnessd.yaml (gateway:) and the key as HARNESS_GATEWAY_API_KEY in secrets.env, or from the app's Server Settings → Model Gateway. The connection test runs from the server.

Cost#

Claude Code's built-in cost estimate may not match what your proxy charges. With tags on, LiteLLM's own spend logs are the source of truth, per project and per task.

See also#

• Agent credentials
• Budgets and limits
• harnessd.yaml reference