Finetune API (/fine_tuning)

API Reference: POST /fine_tuning/jobs

Provider capabilities

The table below summarizes gateway support for this endpoint by provider.

Legend:

✅ Supported by Provider and Truefoundry
Provided by provider, but not by Truefoundry
Provider does not support this feature

Provider	Fine Tune
OpenAI	✅
Azure OpenAI
Anthropic
Bedrock
Vertex	✅
Cohere
Gemini
Groq
Cerebras
Together-AI
xAI
DeepInfra

For every gateway endpoint and provider, see Supported APIs. This guide explains how to fine-tune models using TrueFoundry’s AI Gateway with OpenAI or Vertex AI providers.

Client Setup

All providers use the OpenAI SDK with provider-specific headers. Choose your provider to get started:

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "openai-main"  # truefoundry provider integration name
    }
)

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "vertex-main",  # truefoundry provider integration name
        "x-tfy-vertex-storage-bucket-name": "your-bucket-name",
        "x-tfy-provider-model": "gemini-2-0-flash-001",
        "x-tfy-file-purpose": "fine-tune"  # Required for Vertex AI file uploads
    }
)

Training File Format

Create a JSONL file with one JSON object per line. Each line represents a conversation pair for training:

{"messages":[{"role":"user","content":"What is the weather in San Francisco?"},{"role":"assistant","content":"The weather in San Francisco is currently 18°C with partly cloudy skies. It's a mild day with light winds."}]}
{"messages":[{"role":"user","content":"What is the weather in Minneapolis?"},{"role":"assistant","content":"The weather in Minneapolis is currently 5°C with overcast skies. It's a chilly day, so you might want to dress warmly."}]}
{"messages":[{"role":"user","content":"What is the weather in San Diego?"},{"role":"assistant","content":"The weather in San Diego is currently 22°C with sunny skies. It's a beautiful day with perfect beach weather."}]}

Workflow Steps

The finetuning process follows these steps for all providers:

Upload: Upload training file → Get file ID
Create: Create finetune job → Get job ID
Monitor: Check status until complete
Use: Use the fine-tuned model for inference

Step-by-Step Examples

1. Upload Training File

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "openai-main"
    }
)

# Upload the training file
file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune"
)

print(file.id)  # Example: file-PnFGrFLN5LjjcWr4eFsStK

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "vertex-main",
        "x-tfy-vertex-storage-bucket-name": "your-bucket-name",
        "x-tfy-provider-model": "gemini-2-0-flash-001",
        "x-tfy-file-purpose": "fine-tune"  # Required header
    }
)

# Upload the training file
# File is automatically transformed to Vertex AI format
file = client.files.create(
    file=open("training.jsonl", "rb"),
    purpose="fine-tune"
)

print(file.id)  # Example: gs%3A%2F%2Fbucket-name%2Ffile.jsonl
# This is a GCS URI that you'll use in the next step

2. Create Finetune Job

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "openai-main"
    }
)

job = client.fine_tuning.jobs.create(
    training_file=file.id,
    model="gpt-3.5-turbo",
    hyperparameters={
        "n_epochs": 2
    },
    suffix="my-custom-model"
)

print(job.id)  # Example: ftjob-abc123xyz

from openai import OpenAI

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "vertex-main",
        "x-tfy-vertex-storage-bucket-name": "your-bucket-name",
        "x-tfy-provider-model": "gemini-2-0-flash-001"
    }
)

# Use the GCS URI from file upload
job = client.fine_tuning.jobs.create(
    training_file=file.id,  # GCS URI from step 1
    model="vertex-main/gemini-2-5-flash",  # model to finetune (truefoundry model format)
    suffix="finetune-model",  # Display name for the fine-tuned model
    extra_body={
        "hyperparameters": {
            "n_epochs": 2
        }
    }
)

print(job.id)  # Example: 7170920278355083264
print(job.status)  # Example: queued, running, succeeded, failed

3. Monitor Job Status

from openai import OpenAI
import time

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "openai-main"
    }
)

# Retrieve job status
job = client.fine_tuning.jobs.retrieve(job.id)
print(f"Status: {job.status}")  # queued, running, succeeded, failed, cancelled

from openai import OpenAI
import time

client = OpenAI(
    api_key="your-truefoundry-api-key",
    base_url="{GATEWAY_BASE_URL}",
    default_headers={
        "x-tfy-provider-name": "vertex-main",
        "x-tfy-vertex-storage-bucket-name": "your-bucket-name",
        "x-tfy-provider-model": "gemini-2-0-flash-001"
    }
)

# Retrieve job status
job = client.fine_tuning.jobs.retrieve(job.id)
print(f"Status: {job.status}")  # queued, running, succeeded, failed, cancelled

Status Reference

queued: Job is queued and waiting to start
running: Fine-tuning is in progress
succeeded: Fine-tuning completed successfully
failed: Fine-tuning failed

Hyperparameters

You can configure the following hyperparameters in the hyperparameters object:

n_epochs: Number of training epochs
batch_size: Batch size
learning_rate_multiplier: Learning rate multiplier

Get Started

LLM Gateway

MCP Registry and Gateway

Skills Registry

Prompt Registry

Guardrails and Security

Observability

Deployment

Admin Guide

Chat

Messages

Embeddings

Rerank

Responses

Image

Audio

Batch

Files

Fine-tuning

Moderations

Models

Provider capabilities

Client Setup

Training File Format

Workflow Steps

Step-by-Step Examples

Status Reference

Hyperparameters

​Provider capabilities

​Client Setup

​Training File Format

​Workflow Steps

​Step-by-Step Examples

​Status Reference

​Hyperparameters

Provider capabilities

Client Setup

Training File Format

Workflow Steps

Step-by-Step Examples

Status Reference

Hyperparameters