> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Completions API (/chat/completions)

> Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface

TrueFoundry AI Gateway provides a universal API for all supported models via the standard OpenAI `/chat/completions` endpoint. This unified interface allows you to seamlessly work with models from different providers through a consistent API.

**API Reference:** [`POST /chat/completions`](/docs/api-reference/chat/chat-completions)

## Provider capabilities

The gateway maps the OpenAI Chat Completions contract to each provider. The table below summarizes feature support by provider for this endpoint.

<Info>
  Legend:

  * **✅** Supported by Provider and Truefoundry
  * <Icon icon="circle-xmark" iconType="regular" color="red" /> Provided by provider, but not by Truefoundry
  * <Icon icon="circle-minus" iconType="regular" /> Provider does not support this feature
</Info>

| Provider      | Stream | Non Stream | Tools                                           | JSON Mode                                       | Schema Mode                                     | Prompt Caching                                  | Reasoning                                       | Structured Output                               |
| ------------- | ------ | ---------- | ----------------------------------------------- | ----------------------------------------------- | ----------------------------------------------- | ----------------------------------------------- | ----------------------------------------------- | ----------------------------------------------- |
| OpenAI        | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Azure OpenAI  | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Anthropic     | ✅      | ✅          | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Bedrock       | ✅      | ✅          | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Vertex        | ✅      | ✅          | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Cohere        | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Gemini        | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Groq          | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| AI21          | ✅      | ✅          | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> |
| Cerebras      | ✅      | ✅          | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Wafer         | ✅      | ✅          | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> |
| SambaNova     | ✅      | ✅          | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Perplexity-AI | ✅      | ✅          | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               |
| Together-AI   | ✅      | ✅          | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               | ✅                                               |
| xAI           | ✅      | ✅          | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               | ✅                                               |
| DeepInfra     | ✅      | ✅          | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |
| Nebius        | ✅      | ✅          | ✅                                               | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> | <Icon icon="circle-minus" iconType="regular" /> | ✅                                               | <Icon icon="circle-minus" iconType="regular" /> |

## Contents

| Section                                                                    | Description                                                |
| -------------------------------------------------------------------------- | ---------------------------------------------------------- |
| [Overview](/docs/ai-gateway/chat-completions-overview)                     | Provider capabilities, getting started, and input controls |
| [Multimodal](/docs/ai-gateway/chat-completions-multimodal)                 | Images, audio, video, PDFs, and vision models              |
| [Tools & signatures](/docs/ai-gateway/chat-completions-tools)              | Function calling and thought signatures                    |
| [Structured outputs](/docs/ai-gateway/chat-completions-structured-outputs) | JSON mode, JSON schema, and Pydantic                       |
| [Caching & reasoning](/docs/ai-gateway/chat-completions-advanced)          | Prompt caching and reasoning models                        |
| [Extended thinking](/docs/ai-gateway/chat-completions-extended-thinking)   | Thinking blocks, multi-turn reasoning, grounding           |

## Getting Started

You can use the standard OpenAI client to send requests to the gateway:

```python theme={"dark"}
from openai import OpenAI

client = OpenAI(
    api_key="your_truefoundry_api_key",
    base_url="{GATEWAY_BASE_URL}"
)

response = client.chat.completions.create(
    model="openai-main/gpt-4o-mini", # this is the truefoundry model id
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response.choices[0].message.content)
```

### Configuration

You will need to configure the following:

1. **base\_url**: The base URL of the TrueFoundry AI Gateway
2. **api\_key**: API key generated from [Personal Access Tokens](/docs/ai-gateway/authentication)
3. **model**: TrueFoundry model ID in the format `provider_account/model_name` (available in the LLM playground UI)

See [Integrate with code](/docs/ai-gateway/quick-start#gateway-base-url) for instructions on obtaining these values.

For using native provider SDKs (OpenAI, Google Gen AI, Anthropic, boto3), see [Native SDK Support](/docs/ai-gateway/native-sdk-support).

## Input Controls

### System Prompts

System prompts set the behavior and context for the model by defining the assistant's role, tone, and constraints:

```python theme={"dark"}
response = client.chat.completions.create(
    model="openai-main/gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant that specializes in Python programming."},
        {"role": "user", "content": "How do I write a function to calculate factorial?"}
    ]
)
```

### Request Parameters

Fine-tune model behavior with these common parameters:

```python theme={"dark"}
response = client.chat.completions.create(
    model="openai-main/gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello, how are you?"}],
    temperature=0.7,       # Controls randomness (0.0 to 1.0)
    max_tokens=100,        # Maximum tokens to generate
    verbosity="high",      # Constrains verbosity: low, medium, high
    top_p=0.9,             # Nucleus sampling parameter
    frequency_penalty=0.0, # Reduces repetition
    presence_penalty=0.0,  # Encourages new topics
    stop=["\n", "Human:"]  # Stop sequences
)
```

<Note>
  Some models don't support all parameters. For example, temperature is not supported by `o` series models like o3-mini.
</Note>