CoreWeave

CoreWeave exposes models through W&B Inference, which provides an OpenAI-compatible API. Because CoreWeave is not a first-class provider in the AI Gateway, you connect it using Custom Endpoints — the gateway acts as a transparent proxy that forwards your requests to W&B Inference while keeping the upstream credential configured once in the gateway. This gives your teams centralized authentication, access control, and tracing: clients call the gateway with a single TrueFoundry API key and never see the upstream W&B key.

Prerequisites

Before you start, get a W&B API key — this is the upstream credential the gateway will inject on every request.

Create a W&B API key

Go to your W&B settings page at wandb.ai/settings and open the API keys section under your user account. Click + New key to create a key, then copy it.

Creating a new API key in W&B settings — W&B settings — create a new API key

This key is stored as the upstream credential in TrueFoundry (Step 2 below). Clients calling the gateway never see it.

Adding the Endpoint

This section walks through registering CoreWeave (W&B Inference) as a Custom Endpoint and configuring access control.

Navigate to Custom Endpoints

From the TrueFoundry dashboard, go to AI Gateway → Models, open the Custom Endpoints tab, and click + Add Custom Endpoint.

Custom Endpoints list with Add Custom Endpoint button — AI Gateway — Models — Custom Endpoints tab

Custom Endpoints let you securely proxy any HTTP endpoint through the Model Gateway with centralized authentication, access control, and tracing. Clients use only a TrueFoundry API key, while upstream credentials stay configured in the gateway.

Step 1 — Configure Account

In the Configure Account step of the wizard, set:

Name: coreweave-model — this becomes the providerAccountName segment in your proxy URL.
Endpoint Type: None.
Header Auth: leave OFF — upstream auth is configured at the integration level in the next step.

Click Continue to Endpoints.

Configure Account step with Name set to coreweave-model — Step 1 — Configure Account

Step 2 — Configure the Endpoint Integration

On the Endpoints step, add one Custom Endpoint integration:

Display Name: inference — this becomes the endpointName segment in your proxy URL.
Base URL: https://api.inference.wandb.ai/v1 — the W&B Inference API base. Do not add a trailing slash.
Custom Headers: toggle ON and add the W&B API key as a bearer token:
Header name Header value
Authorization Bearer <YOUR-WANDB-API-KEY>
Header Auth and TLS Settings: leave OFF.

Header name	Header value
`Authorization`	`Bearer <YOUR-WANDB-API-KEY>`

Click Continue.

Endpoints step with inference integration and Authorization header — Step 2 — Endpoint integration with Base URL and Custom Headers

Put the W&B API key in Custom Headers (Authorization: Bearer <key>), not in Header Auth. The gateway injects these custom headers on every upstream request.

Step 3 — Access Control

Configure who can manage and use this endpoint. For example:

Manager (can edit, update, delete): the owning team members.
User (can view and access): everyone, or a specific set of teams.

Learn more in Gateway access control. Click Save Custom Endpoint to finish (or Apply using YAML to save via a manifest).

Access Control step with Manager and User roles — Step 3 — Access Control

Inference

After saving, TrueFoundry shows a Usage code snippet modal for the inference endpoint with the Base URL, Model ID, and a TrueFoundry API key.

Usage code snippet modal showing Base URL, Model ID, and API Key — Usage code snippet for the CoreWeave custom endpoint

The gateway is a transparent proxy, so you call it like the W&B Inference (OpenAI-compatible) API — append the upstream path (chat/completions) to the proxy base URL yourself.

The default code snippet has PROXY_PATH = "". You must append the upstream path — for chat, that is chat/completions — so the full URL becomes:{GATEWAY_BASE_URL}/proxy-api/coreweave-model/inference/chat/completions

Python

import requests

PROXY_BASE_URL = "{GATEWAY_BASE_URL}/proxy-api/coreweave-model/inference"

headers = {
    "Authorization": "Bearer your-truefoundry-api-key",
    "Content-Type": "application/json",
    "Accept-Encoding": "identity",
}

body = {
    "model": "meta-llama/Llama-3.1-8B-Instruct",
    "messages": [
        {"role": "user", "content": "Say hello in one sentence."},
    ],
    "max_tokens": 50,
}

response = requests.post(
    f"{PROXY_BASE_URL}/chat/completions",
    headers=headers,
    json=body,
)
print(response.status_code)
print(response.json()["choices"][0]["message"]["content"])

Add "Accept-Encoding": "identity" to your request headers to avoid gzip decompression issues through the proxy.

Callers only need a TrueFoundry API key for the gateway. The W&B API key stays configured in the gateway and is never passed from client code.

Endpoint structure

Requests use the standard Custom Endpoint URL shape:

{GATEWAY_BASE_URL}/proxy-api/{providerAccountName}/{endpointName}/{upstream-path}

For this setup:

{GATEWAY_BASE_URL}/proxy-api/coreweave-model/inference/chat/completions

Segment	Value	Source
`providerAccountName`	`coreweave-model`	Name field from Step 1 — Configure Account
`endpointName`	`inference`	Display Name field from Step 2 — Endpoints
`upstream-path`	`chat/completions`	W&B Inference API suffix, appended manually

providerAccountName (coreweave-model) and endpointName (inference) must match exactly what you set in the wizard. For more on the proxy URL, authentication, and configuration fields, see Custom Endpoints.

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Prerequisites

Adding the Endpoint

Inference

Endpoint structure

​Prerequisites

​Adding the Endpoint

​Inference

​Endpoint structure

Prerequisites

Adding the Endpoint

Inference

Endpoint structure