Messages API (/messages)

API Reference: POST /messages

Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

Legend:

✅ Supported by Provider and Truefoundry
Provided by provider, but not by Truefoundry
Provider does not support this feature

Provider	Messages API
Anthropic	✅

For every gateway endpoint and provider, see Supported APIs. Anthropic’s Messages API is a powerful interface for interacting with Claude models. When using TruefFundry as your model gateway, you can access this API through a proxy endpoint that handles authentication and routing to the appropriate model.

Prerequisites

To use the Anthropic Messages API through Truefoundry, you’ll need:

TrueFoundry API Key
Provider account configured in TrueFoundry (Anthropic)
Python environment with anthropic sdk library installed

Using the Anthropic SDK

The Anthropic Python SDK provides a convenient way to interact with Claude models. Here’s how to configure it to work with the Truefoundry proxy:

The gateway accepts both Anthropic SDK auth patterns and translates internally:

api_key=TFY_API_KEY - SDK sends the x-api-key header
auth_token=TFY_API_KEY — SDK sends the Authorization: Bearer header

Either works; the request body is identical. api_key is the idiomatic Anthropic SDK pattern - use it unless you have a reason to send a Bearer token.

from anthropic import Anthropic

BASE_URL = "{GATEWAY_BASE_URL}"
API_KEY = "your-truefoundry-api-key"

# Configure the Anthropic client to use TrueFoundry's Gateway
client = Anthropic(
    api_key=API_KEY,
    base_url=BASE_URL
)

# Make a request to the Messages API
def generate_response():
    response = client.messages.create(
        model="anthropic/claude-3-5",  # The model name configured in Truefoundry
        max_tokens=1024,
        messages=[
            {
                "role": "user",
                "content": "Hello, Claude! Please explain quantum computing in simple terms."
            }
        ]
    )

    print(response.content)

generate_response()

Request Format

When using the Messages API through Truefoundry, your request should follow this format:

{
  "model": "tfy-model-name",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "Hello, Claude! Please explain quantum computing in simple terms."
    }
  ]
}

Response Format

The response from the Messages API will have this structure:

{
  "id": "msg_01XB89YSAA2VGMCF3ZS8ATTA1B",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Quantum computing is like traditional computing but it uses quantum bits or 'qubits' instead of regular bits. While traditional bits can only be in a state of 0 or 1, qubits can exist in multiple states simultaneously thanks to a quantum property called 'superposition.' This allows quantum computers to process certain types of information much faster than regular computers.\n\nAnother key quantum property is 'entanglement,' where qubits become connected and the state of one instantly affects the other, no matter the distance between them.\n\nThese properties give quantum computers the potential to solve certain complex problems much faster than traditional computers, like factoring large numbers (important for encryption) or simulating molecular structures (useful for drug development).\n\nHowever, quantum computers are still in early development stages. They're extremely sensitive to their environment and require special conditions like ultra-cold temperatures to operate. They're not replacements for regular computers but specialized tools for specific types of problems."
    }
  ],
  "model": "claude-3-opus-20240229",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 14,
    "output_tokens": 178
  }
}

Prompt Caching

Prompt caching reuses previously computed prompt prefixes (a large system prompt, a shared context document, or a set of tool definitions) so the provider can skip reprocessing the cached portion on subsequent requests. This lowers both latency and cost. There are two ways to cache with Claude models, and you can combine them:

Explicit caching — you mark cache breakpoints yourself by adding cache_control to individual blocks (a system block, a message content block, or a tool definition).
Automatic caching — you add a single cache_control field at the top level of the request body. The provider then applies the cache breakpoint to the last cacheable block for you and moves it forward as the conversation grows, so you don’t manage breakpoints yourself. Both are provider-side features — see Anthropic’s prompt caching guide.

The Gateway forwards cache_control to native Claude providers unchanged — it does not strip it (the request is already in Anthropic’s format). The Gateway only strips cache_control for providers that don’t accept it, such as OpenAI or Gemini, which aren’t served natively on the Messages endpoint (see the note below).

Provider support

The Messages endpoint is served natively (in Anthropic’s format) by Anthropic and by Claude models on Bedrock, Google Vertex, and Azure AI Foundry. Explicit caching works on all of them; automatic caching is a provider capability that only some of them offer:

Provider	Explicit (block-level)	Automatic (top-level `cache_control`)
Anthropic (direct)	✅	✅
Claude Platform on AWS	✅	✅
Azure AI Foundry (Claude)	✅	✅
AWS Bedrock (Claude)	✅	❌ — the Gateway drops the top-level field
Google Vertex (Claude)	✅	❌

Automatic caching (the single top-level cache_control field) is only supported by the Anthropic API, Claude Platform on AWS, and Microsoft/Azure Foundry. AWS Bedrock and Google Vertex do not support automatic caching — use explicit block-level breakpoints on those providers. For Bedrock specifically, the Gateway drops a top-level cache_control (InvokeModel does not accept it) while forwarding block-level cache_control unchanged.

On Bedrock, prompt caching is enabled by default when the request goes through the InvokeModel API (which the Messages endpoint uses for Claude models). You can set explicit cache checkpoints at any point in your request body — across system blocks, message content blocks, and tool definitions — by attaching cache_control to each block you want to mark. Unlike the Converse-based /chat/completions path, these are forwarded as native Anthropic cache_control rather than rewritten into cachePoint markers.

Non-Claude models (and provider-managed providers such as OpenAI or Gemini) are served on the Messages endpoint by translating the request into Chat Completions format. In that translation cache_control is dropped, but the provider’s own prefix caching still applies transparently, and any cached-token counts are still reported in usage.

Explicit caching

Add "cache_control": {"type": "ephemeral"} to any system block, message content block, or tool definition you want cached. This works on every native Claude provider:

curl -X POST "{GATEWAY_BASE_URL}/messages" \
  -H "Authorization: Bearer $TFY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic-main/claude-sonnet-4-20250514",
    "max_tokens": 1024,
    "system": [
      {
        "type": "text",
        "text": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
        "cache_control": { "type": "ephemeral" }
      }
    ],
    "messages": [
      { "role": "user", "content": "Summarize the key rules from the system prompt." }
    ]
  }'

from anthropic import Anthropic

client = Anthropic(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
)

message = client.messages.create(
    model="anthropic-main/claude-sonnet-4-20250514",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "<LONG_SYSTEM_PROMPT_OVER_THE_MIN_TOKENS>",
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        {"role": "user", "content": "Summarize the key rules from the system prompt."}
    ],
)

print(message.usage)

Automatic caching

Add a single cache_control field at the top level of the request body instead of marking individual blocks. The provider applies the breakpoint to the last cacheable block and advances it as the conversation grows — useful for multi-turn chats. This is only supported on Anthropic (direct), Claude Platform on AWS, and Azure AI Foundry:

curl -X POST "{GATEWAY_BASE_URL}/messages" \
  -H "Authorization: Bearer $TFY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic-main/claude-opus-4-8",
    "max_tokens": 1024,
    "cache_control": { "type": "ephemeral" },
    "system": "You are a helpful assistant that remembers our conversation.",
    "messages": [
      { "role": "user", "content": "My name is Alex. I work on machine learning." },
      { "role": "assistant", "content": "Nice to meet you, Alex!" },
      { "role": "user", "content": "What did I say I work on?" }
    ]
  }'

from anthropic import Anthropic

client = Anthropic(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
)

message = client.messages.create(
    model="anthropic-main/claude-opus-4-8",
    max_tokens=1024,
    cache_control={"type": "ephemeral"},
    system="You are a helpful assistant that remembers our conversation.",
    messages=[
        {"role": "user", "content": "My name is Alex. I work on machine learning."},
        {"role": "assistant", "content": "Nice to meet you, Alex!"},
        {"role": "user", "content": "What did I say I work on?"},
    ],
)

print(message.usage)

On AWS Bedrock and Google Vertex, automatic caching is not supported. A top-level cache_control is dropped by the Gateway for Bedrock — use explicit block-level breakpoints there instead.

Anthropic enforces a minimum cacheable prefix length; shorter prompts accept the cache_control hint but are not actually cached:

Minimum tokens	Models
`4096`	Claude Mythos Preview, Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5
`2048`	Sonnet 4.6, Haiku 3.5, Haiku 3
`1024`	Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7

Cache usage in the response

When caching is active, the response usage object reports the cached token counts:

{
  "usage": {
    "input_tokens": 120,
    "output_tokens": 210,
    "cache_creation_input_tokens": 4096,
    "cache_read_input_tokens": 0
  }
}

cache_creation_input_tokens: tokens written to the cache (first call).
cache_read_input_tokens: tokens served from the cache (subsequent calls).

Advanced Features

The Messages API supports several advanced features:

System Prompts

You can include a system prompt to guide Claude’s behavior:

client.messages.create(
    model="anthropic/claude-3-5",
    system="You are a helpful AI assistant that specializes in explaining complex topics simply.",
    messages=[
        {"role": "user", "content": "Explain quantum entanglement."}
    ]
)

Multi-turn Conversations

For multi-turn conversations, include previous messages:

client.messages.create(
    model="anthropic/claude-3-5",
    messages=[
        {"role": "user", "content": "What is machine learning?"},
        {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence..."},
        {"role": "user", "content": "Can you explain supervised vs unsupervised learning?"}
    ]
)

Streaming Responses

For streaming responses, use the streaming parameter:

with client.messages.stream(
    model="anthropic/claude-3-5",
    messages=[{"role": "user", "content": "Write a short poem about AI."}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

    # Access the final message at the end
    print("\nFinal message:", stream.get_final_message())

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Provider capabilities

Prerequisites

Using the Anthropic SDK

Request Format

Response Format

Prompt Caching

Provider support

Explicit caching

Automatic caching

Cache usage in the response

Advanced Features

System Prompts

Multi-turn Conversations

Streaming Responses

​Provider capabilities

​Prerequisites

​Using the Anthropic SDK

​Request Format

​Response Format

​Prompt Caching

​Provider support

​Explicit caching

​Automatic caching

​Cache usage in the response

​Advanced Features

​System Prompts

​Multi-turn Conversations

​Streaming Responses

Provider capabilities

Prerequisites

Using the Anthropic SDK

Request Format

Response Format

Prompt Caching

Provider support

Explicit caching

Automatic caching

Cache usage in the response

Advanced Features

System Prompts

Multi-turn Conversations

Streaming Responses