Responses API (/responses)

API Reference: POST /responses

Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

Legend:

✅ Supported by Provider and Truefoundry
Provided by provider, but not by Truefoundry
Provider does not support this feature

Provider	Model Response
OpenAI	✅
Azure OpenAI	✅
Anthropic
Bedrock
Vertex
Cohere
Gemini
Groq
Cerebras
Together-AI
xAI
DeepInfra

For every gateway endpoint and provider, see Supported APIs. This guide explains how to use the OpenAI / Azure OpenAI client to interact with TrueFoundry’s responses endpoint for making inference requests.

Authentication

You’ll need a TrueFoundry API key to authenticate your requests. You can find authentication details from here. You’ll also need to set the x-tfy-provider-name header to the name of the provider integration you’re using. For example, if you’re using an OpenAI provider integration named my-openai-provider, you’ll set the x-tfy-provider-name header to my-openai-provider.

from openai import OpenAI

client = OpenAI(
    base_url="{GATEWAY_BASE_URL}",
    api_key="your_truefoundry_api_key",
    default_headers={"x-tfy-provider-name": "my-openai-provider"}
)

Text Completion

To get a text completion response:

response = client.responses.create(
    model="your_truefoundry_model_name", //  OpenAI or Azure OpenAI tfy model name
    input=[{"role": "user", "content": "Your prompt here"}]
)

print(response)

Image Inputs

For tasks involving images:

response = client.responses.create(
    model="your_truefoundry_model_name", // OpenAI or Azure OpenAI tfy model name
    input=[
        {"role": "user", "content": "Your prompt here"},
        {
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "your_image_url"
                }
            ]
        }
    ]
)
response_id = response.id

Exprected Response

{
  "id": "resp_6847fa2670f8819887e2d14c08bdc5a305d8dc9b22b6f0dd",
  "created_at": 1749547558,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "metadata": {},
  "model": "gpt-4o-mini-2024-07-18",
  "object": "response",
  "output": [
    {
      "id": "msg_6847fa35e8c48198a5120ee3d38c353105d8dc9b22b6f0dd",
      "content": [
        {
          "annotations": [],
          "text": "It seems you might be looking to start a discussion or ask a question! How can I assist you today?",
          "type": "output_text"
        }
      ],
      "role": "assistant",
      "status": "completed",
      "type": "message"
    }
  ],
  "parallel_tool_calls": true,
  "temperature": 1,
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1,
  "max_output_tokens": null,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "generate_summary": null,
    "summary": null
  },
  "status": "completed",
  "text": {
    "format": {
      "type": "text"
    }
  },
  "truncation": "disabled",
  "usage": {
    "input_tokens": 10,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 23,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 33
  },
  "user": null,
  "background": false,
  "service_tier": "default",
  "store": true,
  "provider": "openai"
}

Managing Responses

Retrieve Response

Retrieve a response by id.

response = client.responses.retrieve(
	response_id=response_id
)

print(response)

Expected Output

{
  "id": "resp_6847fa2670f8819887e2d14c08bdc5a305d8dc9b22b6f0dd",
  "created_at": 1749547558,
  "error": null,
  "incomplete_details": null,
  "instructions": null,
  "metadata": {},
  "model": "gpt-4o-mini-2024-07-18",
  "object": "response",
  "output": [
    {
      "id": "msg_6847fa35e8c48198a5120ee3d38c353105d8dc9b22b6f0dd",
      "content": [
        {
          "annotations": [],
          "text": "It seems you might be looking to start a discussion or ask a question! How can I assist you today?",
          "type": "output_text"
        }
      ],
      "role": "assistant",
      "status": "completed",
      "type": "message"
    }
  ],
  "parallel_tool_calls": true,
  "temperature": 1,
  "tool_choice": "auto",
  "tools": [],
  "top_p": 1,
  "max_output_tokens": null,
  "previous_response_id": null,
  "reasoning": {
    "effort": null,
    "generate_summary": null,
    "summary": null
  },
  "status": "completed",
  "text": {
    "format": {
      "type": "text"
    }
  },
  "truncation": "disabled",
  "usage": {
    "input_tokens": 10,
    "input_tokens_details": {
      "cached_tokens": 0
    },
    "output_tokens": 23,
    "output_tokens_details": {
      "reasoning_tokens": 0
    },
    "total_tokens": 33
  },
  "user": null,
  "background": false,
  "service_tier": "default",
  "store": true,
  "provider": "openai"
}

Delete Response

Delete a response permanently

delete_result = client.responses.delete(
	response_id=response_id
)

Finetuning Image Generation

⌘I

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Provider capabilities

Authentication

Text Completion

Image Inputs

Exprected Response

Managing Responses

Retrieve Response

Expected Output

Delete Response

​Provider capabilities

​Authentication

​Text Completion

​Image Inputs

​Exprected Response

​Managing Responses

​Retrieve Response

​Expected Output

​Delete Response

Provider capabilities

Authentication

Text Completion

Image Inputs

Exprected Response

Managing Responses

Retrieve Response

Expected Output

Delete Response