Point any agent framework or LLM app at the gateway by overriding the base URL and API key — no code rewrite needed. Pick your integration below, or expand **How to proxy any request** for the general pattern.

Agno

CrewAI

DSPy

Instructor

LangChain

Langroid

OpenAI Agents SDK

OpenAI Swarm

Phidata

Pydantic AI

Strands Agents

AnythingLLM

Dify

Flowise

Jan

Langflow

LibreChat

n8n

Open WebUI

Agent frameworks and LLM applications talk to models over an OpenAI-compatible API. To route them through TrueFoundry, you don't rewrite any logic — you point the client at the gateway by overriding two values: * **Base URL** → your TrueFoundry AI Gateway endpoint * **API key** → your TrueFoundry API key Grab your base URL, model ID, and API key from the unified code snippet in the gateway playground: TrueFoundry playground usage code snippet showing the Base URL, Model ID, and API Key

TrueFoundry playground usage code snippet showing the Base URL, Model ID, and API Key

Once traffic flows through the gateway, every framework and app inherits unified model access, rate limiting, budgets, guardrails, logging, and observability. ```python OpenAI SDK (Python) theme={"dark"} from openai import OpenAI client = OpenAI( base_url="https://gateway.truefoundry.ai", # your TrueFoundry Gateway URL api_key="your-truefoundry-api-key", ) response = client.chat.completions.create( model="openai-main/gpt-4o", messages=[{"role": "user", "content": "Hello from TrueFoundry!"}], ) print(response.choices[0].message.content) ``` ```typescript OpenAI SDK (TypeScript) theme={"dark"} import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://gateway.truefoundry.ai", // your TrueFoundry Gateway URL apiKey: "your-truefoundry-api-key", }); const response = await client.chat.completions.create({ model: "openai-main/gpt-4o", messages: [{ role: "user", content: "Hello from TrueFoundry!" }], }); console.log(response.choices[0].message.content); ``` ```bash Environment variables theme={"dark"} # Most apps read these standard variables — set them and you're done export OPENAI_BASE_URL="https://gateway.truefoundry.ai" export OPENAI_API_KEY="your-truefoundry-api-key" ``` Any framework or app that lets you set a custom base URL and API key can route through the gateway using this same pattern.

Attach policy checks — PII redaction, moderation, prompt-injection defense, and more — to requests flowing through the gateway.

TrueFoundry Guardrails

OpenAI Moderation

Bedrock Guardrails

Azure PII

Azure Content Safety

Azure Prompt Shield

Enkrypt AI

Palo Alto Prisma AIRS

CrowdStrike AIDR

Patronus AI

Fiddler AI

Google Model Armor

GraySwan Cygnal

Akto

TrojAI

Noma Security

Pillar Security

NVIDIA NeMo

Guardrails AI

CoreWeave Guardrails

Lasso Security

Arthur AI

Verra

HiddenLayer

Custom Guardrails

Tavily

Microsoft 365

Exa

Linear

Notion

Slack

Looker

HubSpot

Databricks

Neo4j

GitHub

Smartsheet

Atlassian Rovo

Google Workspace

CrowdStrike Falcon

Salesforce

Zoom

Honeycomb

dbt

Gong

PagerDuty

Zendesk

QuickBooks

Snowflake

Firebase Crashlytics

Airtable

incident.io

MongoDB

Omni Analytics

NetSuite

Datadog

Ironclad

Ramp

TrueFoundry builds and maintains open-source infrastructure for running AI and Kubernetes workloads efficiently. Each project runs entirely in your own environment — explore the ones below and dive into their docs. Kubernetes-native scale-to-zero for HTTP services — cut cost when idle without dropping traffic. A CNCF Sandbox project. Kubernetes-native rightsizing that lowers request overhead per workload using the Prometheus metrics you already collect. An on-device agent that routes AI (LLM and MCP) traffic through your gateway for logging and policy — no app changes.