> ## Documentation Index
> Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Google Gemini
> Add and configure Google Gemini models in TrueFoundry's AI Gateway
### Adding Models
This section explains the steps to add Google Gemini models and configure the required access controls.
From the TrueFoundry dashboard, navigate to `AI Gateway` > `Models` and select `Google Gemini`.
Click `Add Google Gemini Account`. Give a unique name to your Gemini account and complete the form with your Gemini authentication details (API Key). Add collaborators to your account, this will give access to the account to other users/teams. Learn more about access control [here](/docs/ai-gateway/gateway-access-control).
Select the model from the list. If you see the model you want to add in the list of checkboxes, we support public model cost for these models.
(Optional) If the model you are looking for is not present in the options, you can add it using `+ Add Model` at the end of list (scroll down to see the option) by filling the form.
### Inference
After adding the models, you can perform inference using an OpenAI-compatible API via the Playground or by integrating with your own application.
### Supported APIs
Once your Gemini provider account is configured, the following API surfaces are available through the gateway.
Legend:
* **✅** Supported by Provider and Truefoundry
* Supported by Provider, but not by Truefoundry
* Provider does not support this feature
| API | Endpoint | Tracing | Cost Tracking |
| ------------------------------------- | --------------------- | ------- | ----------------------------------------------------------- |
| [Chat Completions](#chat-completions) | `/chat/completions` | **✅** | **✅** |
| [Text-to-Speech](#text-to-speech) | `/audio/speech` | **✅** | |
| [Embeddings](#embeddings) | `/embeddings` | **✅** | **✅** |
| [Image Generation](#image-generation) | `/images/generations` | **✅** | **✅** |
| [Image Edit](#image-edit) | `/images/edits` | **✅** | |
| [Fine-tuning](#fine-tuning) | `/fine_tuning/jobs` | **✅** | |
**Not supported for Gemini:** Batch API, Files API, Messages API (Anthropic-only), Responses API, Speech-to-Text, Audio Translation, Moderation, Image Variation, and Realtime API. For Batch and Files workflows, use [Google Vertex AI](/docs/ai-gateway/google-vertex) — same Gemini models, GCS-backed batch/files.
Gemini's chat completions endpoint supports streaming, tools, multimodal input (images, **audio, video**, PDF), structured JSON outputs, extended thinking, and **Google Search grounding**. The gateway translates OpenAI-compatible requests into Gemini's native `generateContent` API.
Full provider capability matrix: [Chat Completions API](/docs/ai-gateway/chat-completions-overview).
```python Python lines theme={"dark"}
from openai import OpenAI
client = OpenAI(
api_key="your-truefoundry-api-key",
base_url="{GATEWAY_BASE_URL}",
)
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{"role": "user", "content": "What is TrueFoundry in one line?"}],
)
print(response.choices[0].message.content)
```
Set `stream=True` and iterate over delta chunks. Defensively check that `chunk.choices` is non-empty and `delta.content` is not `None`.
```python Python lines theme={"dark"}
stream = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{"role": "user", "content": "Count from 1 to 5, one number per line."}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content is not None:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
```
Advertise a tool, hand the model's `tool_calls` back as a `tool` role message, then request the final response.
```python Python lines theme={"dark"}
import json
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
messages = [{"role": "user", "content": "What's the weather in Bengaluru?"}]
first = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=messages,
tools=tools,
tool_choice={"type": "function", "function": {"name": "get_weather"}},
)
tool_call = first.choices[0].message.tool_calls[0]
messages.append(first.choices[0].message)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps({"city": "Bengaluru", "temp_c": 28}),
})
second = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=messages,
)
print(second.choices[0].message.content)
```
Gemini accepts images, audio, video, and PDFs via multimodal content parts. Images use the `image_url` content type; audio and video also use `image_url` with a `mime_type` hint; PDFs use the `file` content type with a base64 data URI.
```python Python lines theme={"dark"}
# Image input
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}},
],
}],
)
# Audio input (Gemini-specific)
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Describe this audio."},
{"type": "image_url", "image_url": {"url": "data:audio/wav;base64,...", "mime_type": "audio/wav"}},
],
}],
)
# Video input (Gemini-specific) — gateway fetches the URL server-side
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Describe this video."},
{"type": "image_url", "image_url": {"url": "https://example.com/video.mp4", "mime_type": "video/mp4"}},
],
}],
)
```
Gemini supports two structured-output modes via `response_format`:
* **JSON object** — `{"type": "json_object"}` — guarantees valid JSON, no schema
* **JSON schema** — `{"type": "json_schema", "json_schema": {...}}` — enforces a schema
```python Python lines theme={"dark"}
schema = {
"name": "person",
"schema": {
"type": "object",
"properties": {
"name": {"type": "string"},
"age": {"type": "integer"},
"hobbies": {"type": "array", "items": {"type": "string"}},
},
"required": ["name", "age", "hobbies"],
"additionalProperties": False,
},
"strict": True,
}
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{"role": "user", "content": "Invent a fictional person."}],
response_format={"type": "json_schema", "json_schema": schema},
)
```
Gemini 2.5 Pro and 2.5 Flash support extended thinking, on by default. Use `reasoning_effort` (`low`/`medium`/`high`) — the gateway translates it to Gemini's native thinking-budget parameter.
```python Python lines theme={"dark"}
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-pro",
messages=[{"role": "user", "content": "A bat and a ball cost $1.10. The bat costs $1.00 more than the ball. How much is the ball?"}],
reasoning_effort="high",
max_tokens=8000,
)
print(response.choices[0].message.content)
```
Gemini can augment responses with live Google Search results via a special `google_search` tool. The model decides when to invoke search during generation.
```python Python lines theme={"dark"}
response = client.chat.completions.create(
model="gemini-main/gemini-2.5-flash",
messages=[{"role": "user", "content": "What is today's date? Include the year."}],
tools=[{"type": "function", "function": {"name": "google_search"}}],
)
print(response.choices[0].message.content)
```
Gemini's TTS-preview models generate audio from text via the OpenAI-compatible `/audio/speech` endpoint. The gateway routes this to Google Cloud Text-to-Speech upstream.
Full docs: [Text-to-Speech](/docs/ai-gateway/text-to-speech).
```python Python lines theme={"dark"}
response = client.audio.speech.create(
model="gemini-main/gemini-2.5-pro-preview-tts",
voice="alloy",
input="Hello from TrueFoundry. The AI gateway makes multi-provider routing simple.",
)
with open("tts.wav", "wb") as f:
f.write(response.read())
```
The gateway accepts OpenAI voice aliases (`alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer`) and translates them to default Cloud TTS voices. For specific Cloud TTS voices, pass the fully-qualified name (e.g. `en-US-Chirp3-HD-Kore`).
Gemini's `gemini-embedding-2` text embedding model via the OpenAI-compatible `/embeddings` endpoint. Accepts a `task_type` parameter via `extra_body` that tunes the vector for the downstream task (`RETRIEVAL_DOCUMENT`, `RETRIEVAL_QUERY`, `SEMANTIC_SIMILARITY`, `CLASSIFICATION`, `CLUSTERING`).
Full docs: [Embed API](/docs/ai-gateway/embed).
```python Python lines theme={"dark"}
doc = client.embeddings.create(
model="gemini-main/gemini-embedding-2",
input="TrueFoundry is an AI gateway that unifies access to multiple LLM providers.",
extra_body={"task_type": "RETRIEVAL_DOCUMENT"},
)
query = client.embeddings.create(
model="gemini-main/gemini-embedding-2",
input="What does TrueFoundry do?",
extra_body={"task_type": "RETRIEVAL_QUERY"},
)
print("dim:", len(doc.data[0].embedding))
```
Gemini's `gemini-3.1-flash-image` generates images from text via the OpenAI-compatible `/images/generations` endpoint.
Full docs: [Image Generation](/docs/ai-gateway/image-generation).
```python Python lines theme={"dark"}
import base64
response = client.images.generate(
model="gemini-main/gemini-3.1-flash-image",
prompt="A minimalist isometric illustration of a cloud with a lightning bolt, flat colors.",
size="1024x1024",
n=1,
)
item = response.data[0]
if getattr(item, "b64_json", None):
image_bytes = base64.b64decode(item.b64_json)
else:
import requests
image_bytes = requests.get(item.url, timeout=60).content
with open("generated.png", "wb") as f:
f.write(image_bytes)
```
**Pricing varies by model family.** Imagen models are billed **per image** at a flat rate. Gemini image models are billed **per token**, where higher-resolution / HD outputs consume more tokens.
Image edit uses the same `gemini-3.1-flash-image` model via the OpenAI-compatible `/images/edits` endpoint. Pass the source image and a mask.
```python Python lines theme={"dark"}
import base64
from PIL import Image as PILImage, ImageDraw
mask_img = PILImage.new("L", (1024, 1024), 0)
ImageDraw.Draw(mask_img).rectangle((700, 0, 1024, 300), fill=255)
mask_img.save("mask.png", format="PNG")
with open("generated.png", "rb") as img_f, open("mask.png", "rb") as mask_f:
response = client.images.edit(
model="gemini-main/gemini-3.1-flash-image",
image=img_f,
mask=mask_f,
prompt="Paint a bright yellow sun in the top-right corner.",
n=1,
)
item = response.data[0]
if getattr(item, "b64_json", None):
image_bytes = base64.b64decode(item.b64_json)
else:
import requests
image_bytes = requests.get(item.url, timeout=60).content
with open("edited.png", "wb") as f:
f.write(image_bytes)
```
Gemini supports supervised fine-tuning of Gemini models. Upload JSONL via the Files API with `purpose="fine-tune"`, then submit a tuning job through `/fine_tuning/jobs`. Lifecycle mirrors OpenAI fine-tuning.
Full docs: [Fine-tuning](/docs/ai-gateway/finetune).
Fine-tuning incurs real GCP charges. See the [Vertex tunable model list](https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini-supervised-tuning).
```python Python lines theme={"dark"}
import json, uuid
from openai import OpenAI
FINETUNE_BASE_MODEL = "gemini-2.5-flash"
with open("finetune_training.jsonl", "w") as f:
for topic, haiku in [
("the sun", "Bright orb in the sky"),
("a river", "Silver thread of life"),
("the moon", "Pale night-watcher gleams"),
("a forest", "Tall trees stand in green"),
("the ocean", "Vast blue stretching wide"),
]:
f.write(json.dumps({"messages": [
{"role": "user", "content": f"What is {topic}?"},
{"role": "assistant", "content": haiku},
]}) + "\n")
ft_client = OpenAI(
api_key="your-truefoundry-api-key",
base_url="{GATEWAY_BASE_URL}",
default_headers={
"x-tfy-provider-name": "gemini-main",
"x-tfy-provider-model": FINETUNE_BASE_MODEL,
"x-tfy-file-purpose": "fine-tune",
},
)
with open("finetune_training.jsonl", "rb") as f:
ft_file = ft_client.files.create(file=f, purpose="fine-tune")
ft_job = ft_client.fine_tuning.jobs.create(
training_file=ft_file.id,
model=f"gemini-main/{FINETUNE_BASE_MODEL}",
suffix=f"gemini-ft-{uuid.uuid4().hex[:6]}",
extra_body={"hyperparameters": {"n_epochs": 2}},
)
print(f"created: {ft_job.id} status={ft_job.status}")
# Retrieve status (queued, running, succeeded, failed, cancelled)
ft_job = ft_client.fine_tuning.jobs.retrieve(ft_job.id)
print(f"status: {ft_job.status}")
```