> ## Documentation Index > Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt > Use this file to discover all available pages before exploring further. # Messages API (/messages) > Learn how to use Messages API through TrueFoundry's AI Gateway for interacting with Claude models. **API Reference:** [`POST /messages`](/docs/api-reference/messages/messages) ## Provider capabilities The table below summarizes gateway support for this endpoint by provider. Legend: * **✅** Supported by Provider and Truefoundry * Provided by provider, but not by Truefoundry * Provider does not support this feature | Provider | Messages API | | --------- | ------------ | | Anthropic | ✅ | For every gateway endpoint and provider, see [Supported APIs](/docs/ai-gateway/intro-to-llm-gateway#supported-apis). [Anthropic's Messages API](https://docs.anthropic.com/en/api/messages) is a powerful interface for interacting with Claude models. When using TruefFundry as your model gateway, you can access this API through a proxy endpoint that handles authentication and routing to the appropriate model. ## **Prerequisites** To use the Anthropic Messages API through Truefoundry, you'll need: 1. TrueFoundry API Key 2. Provider account configured in TrueFoundry (Anthropic) 3. Python environment with `anthropic sdk` library installed ## Using the Anthropic SDK The Anthropic Python SDK provides a convenient way to interact with Claude models. Here's how to configure it to work with the Truefoundry proxy: The gateway accepts both Anthropic SDK auth patterns and translates internally: * `api_key=TFY_API_KEY` - SDK sends the `x-api-key` header * `auth_token=TFY_API_KEY` — SDK sends the `Authorization: Bearer` header Either works; the request body is identical. `api_key` is the idiomatic Anthropic SDK pattern - use it unless you have a reason to send a Bearer token. ```python lines theme={"dark"} from anthropic import Anthropic BASE_URL = "{GATEWAY_BASE_URL}" API_KEY = "your-truefoundry-api-key" # Configure the Anthropic client to use TrueFoundry's Gateway client = Anthropic( api_key=API_KEY, base_url=BASE_URL ) # Make a request to the Messages API def generate_response(): response = client.messages.create( model="anthropic/claude-3-5", # The model name configured in Truefoundry max_tokens=1024, messages=[ { "role": "user", "content": "Hello, Claude! Please explain quantum computing in simple terms." } ] ) print(response.content) generate_response() ``` ## Request Format When using the Messages API through Truefoundry, your request should follow this format: ```json lines theme={"dark"} { "model": "tfy-model-name", "max_tokens": 1024, "messages": [ { "role": "user", "content": "Hello, Claude! Please explain quantum computing in simple terms." } ] } ``` ## Response Format The response from the Messages API will have this structure: ```json lines theme={"dark"} { "id": "msg_01XB89YSAA2VGMCF3ZS8ATTA1B", "type": "message", "role": "assistant", "content": [ { "type": "text", "text": "Quantum computing is like traditional computing but it uses quantum bits or 'qubits' instead of regular bits. While traditional bits can only be in a state of 0 or 1, qubits can exist in multiple states simultaneously thanks to a quantum property called 'superposition.' This allows quantum computers to process certain types of information much faster than regular computers.\n\nAnother key quantum property is 'entanglement,' where qubits become connected and the state of one instantly affects the other, no matter the distance between them.\n\nThese properties give quantum computers the potential to solve certain complex problems much faster than traditional computers, like factoring large numbers (important for encryption) or simulating molecular structures (useful for drug development).\n\nHowever, quantum computers are still in early development stages. They're extremely sensitive to their environment and require special conditions like ultra-cold temperatures to operate. They're not replacements for regular computers but specialized tools for specific types of problems." } ], "model": "claude-3-opus-20240229", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 14, "output_tokens": 178 } } ``` ## Prompt Caching Prompt caching reuses previously computed prompt prefixes (a large `system` prompt, a shared context document, or a set of tool definitions) so the provider can skip reprocessing the cached portion on subsequent requests. This lowers both latency and cost. There are two ways to cache with Claude models, and you can combine them: * **Explicit caching** — you mark cache breakpoints yourself by adding `cache_control` to individual blocks (a `system` block, a message content block, or a `tool` definition). * **Automatic caching** — you add a single `cache_control` field at the **top level** of the request body. The provider then applies the cache breakpoint to the last cacheable block for you and moves it forward as the conversation grows, so you don't manage breakpoints yourself. Both are provider-side features — see [Anthropic's prompt caching guide](https://platform.claude.com/docs/en/build-with-claude/prompt-caching). The Gateway forwards `cache_control` to native Claude providers **unchanged — it does not strip it** (the request is already in Anthropic's format). The Gateway only strips `cache_control` for providers that don't accept it, such as OpenAI or Gemini, which aren't served natively on the Messages endpoint (see the note below). ### Provider support The Messages endpoint is served natively (in Anthropic's format) by Anthropic and by Claude models on Bedrock, Google Vertex, and Azure AI Foundry. Explicit caching works on all of them; automatic caching is a provider capability that only some of them offer: | Provider | Explicit (block-level) | Automatic (top-level `cache_control`) | | ------------------------- | ---------------------- | ----------------------------------------- | | Anthropic (direct) | ✅ | ✅ | | Claude Platform on AWS | ✅ | ✅ | | Azure AI Foundry (Claude) | ✅ | ✅ | | AWS Bedrock (Claude) | ✅ | ❌ — the Gateway drops the top-level field | | Google Vertex (Claude) | ✅ | ❌ | Automatic caching (the single top-level `cache_control` field) is only supported by the Anthropic API, Claude Platform on AWS, and Microsoft/Azure Foundry. **AWS Bedrock and Google Vertex do not support automatic caching** — use explicit block-level breakpoints on those providers. For Bedrock specifically, the Gateway drops a top-level `cache_control` (InvokeModel does not accept it) while forwarding block-level `cache_control` unchanged. On Bedrock, prompt caching is **enabled by default** when the request goes through the InvokeModel API (which the Messages endpoint uses for Claude models). You can set explicit cache checkpoints at any point in your request body — across `system` blocks, message content blocks, and `tool` definitions — by attaching `cache_control` to each block you want to mark. Unlike the Converse-based `/chat/completions` path, these are forwarded as native Anthropic `cache_control` rather than rewritten into `cachePoint` markers. Non-Claude models (and provider-managed providers such as OpenAI or Gemini) are served on the Messages endpoint by translating the request into Chat Completions format. In that translation `cache_control` is dropped, but the provider's own prefix caching still applies transparently, and any cached-token counts are still reported in `usage`. ### Explicit caching Add `"cache_control": {"type": "ephemeral"}` to any `system` block, message content block, or `tool` definition you want cached. This works on every native Claude provider: ```bash cURL theme={"dark"} curl -X POST "{GATEWAY_BASE_URL}/messages" \ -H "Authorization: Bearer $TFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic-main/claude-sonnet-4-20250514", "max_tokens": 1024, "system": [ { "type": "text", "text": "", "cache_control": { "type": "ephemeral" } } ], "messages": [ { "role": "user", "content": "Summarize the key rules from the system prompt." } ] }' ``` ```python Anthropic SDK theme={"dark"} from anthropic import Anthropic client = Anthropic( api_key="your-truefoundry-api-key", base_url="{GATEWAY_BASE_URL}", ) message = client.messages.create( model="anthropic-main/claude-sonnet-4-20250514", max_tokens=1024, system=[ { "type": "text", "text": "", "cache_control": {"type": "ephemeral"}, } ], messages=[ {"role": "user", "content": "Summarize the key rules from the system prompt."} ], ) print(message.usage) ``` ### Automatic caching Add a single `cache_control` field at the **top level** of the request body instead of marking individual blocks. The provider applies the breakpoint to the last cacheable block and advances it as the conversation grows — useful for multi-turn chats. This is only supported on Anthropic (direct), Claude Platform on AWS, and Azure AI Foundry: ```bash cURL theme={"dark"} curl -X POST "{GATEWAY_BASE_URL}/messages" \ -H "Authorization: Bearer $TFY_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "anthropic-main/claude-opus-4-8", "max_tokens": 1024, "cache_control": { "type": "ephemeral" }, "system": "You are a helpful assistant that remembers our conversation.", "messages": [ { "role": "user", "content": "My name is Alex. I work on machine learning." }, { "role": "assistant", "content": "Nice to meet you, Alex!" }, { "role": "user", "content": "What did I say I work on?" } ] }' ``` ```python Anthropic SDK theme={"dark"} from anthropic import Anthropic client = Anthropic( api_key="your-truefoundry-api-key", base_url="{GATEWAY_BASE_URL}", ) message = client.messages.create( model="anthropic-main/claude-opus-4-8", max_tokens=1024, cache_control={"type": "ephemeral"}, system="You are a helpful assistant that remembers our conversation.", messages=[ {"role": "user", "content": "My name is Alex. I work on machine learning."}, {"role": "assistant", "content": "Nice to meet you, Alex!"}, {"role": "user", "content": "What did I say I work on?"}, ], ) print(message.usage) ``` On **AWS Bedrock** and **Google Vertex**, automatic caching is not supported. A top-level `cache_control` is dropped by the Gateway for Bedrock — use explicit block-level breakpoints there instead. Anthropic enforces a minimum cacheable prefix length; shorter prompts accept the `cache_control` hint but are not actually cached: | Minimum tokens | Models | | -------------- | -------------------------------------------------------------- | | `4096` | Claude Mythos Preview, Opus 4.7, Opus 4.6, Opus 4.5, Haiku 4.5 | | `2048` | Sonnet 4.6, Haiku 3.5, Haiku 3 | | `1024` | Sonnet 4.5, Opus 4.1, Opus 4, Sonnet 4, Sonnet 3.7 | ### Cache usage in the response When caching is active, the response `usage` object reports the cached token counts: ```json lines theme={"dark"} { "usage": { "input_tokens": 120, "output_tokens": 210, "cache_creation_input_tokens": 4096, "cache_read_input_tokens": 0 } } ``` * `cache_creation_input_tokens`: tokens written to the cache (first call). * `cache_read_input_tokens`: tokens served from the cache (subsequent calls). ## Advanced Features The Messages API supports several advanced features: ### System Prompts You can include a system prompt to guide Claude's behavior: ```python lines theme={"dark"} client.messages.create( model="anthropic/claude-3-5", system="You are a helpful AI assistant that specializes in explaining complex topics simply.", messages=[ {"role": "user", "content": "Explain quantum entanglement."} ] ) ``` ### Multi-turn Conversations For multi-turn conversations, include previous messages: ```python lines theme={"dark"} client.messages.create( model="anthropic/claude-3-5", messages=[ {"role": "user", "content": "What is machine learning?"}, {"role": "assistant", "content": "Machine learning is a subset of artificial intelligence..."}, {"role": "user", "content": "Can you explain supervised vs unsupervised learning?"} ] ) ``` ### Streaming Responses For streaming responses, use the streaming parameter: ```python lines theme={"dark"} with client.messages.stream( model="anthropic/claude-3-5", messages=[{"role": "user", "content": "Write a short poem about AI."}] ) as stream: for text in stream.text_stream: print(text, end="", flush=True) # Access the final message at the end print("\nFinal message:", stream.get_final_message()) ```