Globally Distributed SAAS Gateway

​

Features

1. Globally Distributed: Deployed across more than 12 regions around the globe and across 3 multiple cloud providers for maximum availability while minimizing latency.
2. Automated Failover: All traffic is routed to the nearest gateway for minimum latency. In case of regional downtime, traffic is automatically routed to closest healthy regions ensuring uninterrupted service.
3. Multi-Cloud Deployment: Distributed across multiple cloud providers to be tolerant to cloud provider-specific disruptions.
4. Data Encryption: Data is encrypted at rest and in transit.
5. Compliance: Truefoundry Infrastructure is SOC2, ISO27001, GDPR, and HIPAA compliant

​

Architecture

• Control Plane — Manages all gateway configuration including models, users, teams, virtual accounts, rate-limiting, and routing configs. The SAAS control plane is hosted in Ireland (Europe).
• Gateway Planes — Stateless, horizontally scalable gateway instances that handle all production traffic (LLM requests, MCP requests, etc.). These are deployed across the regions listed in the Regional Deployments section below.

​

Global Deployment

​

Regional Deployments

​

Gateway Metadata

• tfy_gateway_region
• tfy_gateway_zone

​

Multi-regional Deployments

​

Gateway Status Monitoring

​

Check Gateway Status

​

Subscribe to Status Updates

1. Visit the Gateway Status Page
2. Click the Get Updates button in the top right
3. Choose your preferred notification method:
   • Email notifications
   • RSS Feed
   • On a custom webhook

​

Connecting Your Models or MCP Servers to the Gateway

​

Option 1: Reverse Tunnel Agent

1. Raise a request with the Truefoundry team with the following details:
   • private_service_url: The URL of the private service that hosts your MCP server or model. You can also setup a load balancer in front of multiple MCP servers or models to route traffic as needed. Eg - For an mcp server running within the same kubernetes cluster as cloudflared, it would be something like http://mcp-server.default.svc.cluster.local:8080.
2. Truefoundry shares the following with you:
   • public_domain - The public domain that you can use to access your private service. For example <tenant-name>.private-endpoints.truefoundry.ai.
   • auth_token - The token that you can use to run cloudflared in your environment.
3. Run cloudflared in your environment:
   • VM
   • Kubernetes (Helm)

   cloudflared tunnel run --token <auth_token>
   

   Create a values.yaml file:

   tunnel:
     token: <auth_token>
   

   Install tfy-cloudflared:

   helm repo add truefoundry https://truefoundry.github.io/infra-charts/
   helm repo update
   helm install tfy-cloudflared truefoundry/tfy-cloudflared -f values.yaml -n cloudflared --create-namespace
   

   Verify the installation that the pods are running:

   kubectl get pods -n cloudflared
   
4. You can now create the remote MCP server using https://<tenant-name>.private-endpoints.truefoundry.ai/mcp. For detailed MCP server setup instructions, see Getting started with MCP servers.

​

Option 2: Private Link

# Verify connectivity from within your VPC
curl -X GET "https://<vpc-endpoint-dns>/health"

# Verify connectivity from within your VPC
curl -X GET "http://<internal-ip>/health"

# Verify connectivity from within your VNet
curl -X GET "http://<private-endpoint-ip>/health"

• Most secure option — traffic never leaves the cloud provider’s private network.
• Requires setup in each region where you need connectivity.
• Involves coordination with the Truefoundry team for service endpoint provisioning.
• May incur additional cloud provider charges for Private Link / Private Service Connect endpoints and data transfer.

​

Option 3: Self-Host the Gateway