Deploy Control Plane and Compute Plane

Truefoundry Control-Plane ships as a helm chart (https://github.com/truefoundry/infra-charts/tree/main/charts/truefoundry) that can be deployed on any Kubernetes cluster. The installation process is different based on if you are spinning up a Kubernetes cluster from scratch or using an existing cluster to install the control-plane.

Please read the control-plane architecture to understand the key components of the control plane.

Control-plane Dependencies

The TrueFoundry control-plane requires a Kubernetes cluster, a Postgres database, and Blob Storage. Below is a table outlining popular options for each, along with typical cost ranges. Actual cost may vary depending on region, resource choice, scale, and provider pricing. All costs are rough monthly estimates.

Component	Supported Options	Typical Cost Range (USD/month)	Remarks
Kubernetes Cluster K8s version >= 1.28	AWS EKS GCP GKE Azure AKS Self-Managed K8s	$400-1200 (depending on if we want AI deployment or AI Gateway or both features)	Kubernetes cluster is required to host the control-plane helm chart which brings up all the components.
Postgres Database Postgres version >= 13, Storage size: 30GB	-AWS RDS/Postgres (db.t4g.medium) - GCP CloudSQL (db-custom-1-3840) -Azure Postgres (GP_Standard_D4ds_v5) -Self-hosted Postgres	$200-400 (Depending on the cloud provider and the instance type)	The database stores all the data including deployment configuration, model registry metadata, user information, gateway configuration, etc.
Blob Storage	AWS S3 GCP Cloud Storage Azure Blob Storage Any S3-compatible storage like Minio	Depends on storage (Provision for 200GB min)	Stores ML models and artifacts, Gateway request and response logs
Domain Name	Any provider (Route53, Cloudflare, etc.)	Typically free with domain or varies per provider	We need one domain to be pointed to the control-plane dashboard so that the UI can be accessed over the browser.

Control-Plane Compute Requirements

The control-plane has different components which can be disabled or enabled based on your needs. The control-plane helm chart also has the Gateway plane bundled with it in case you want to deploy it. Here’s the compute requirements based on the features you want to enable in the control-plane.

Small (Dev)
Medium (Prod)
Large (Prod)

The small tier is recommended for POC/dev purposes. Here all the components are deployed on Kubernetes and in non HA mode (single replica). This is suitable if you are just testing out the different features of Truefoundry.

This setup brings up 1 replica of the services and is not highly-available. It can enable you to test the features but we do not recommend this for production mode.

Configuration	CPU / Memory	Block Storage PV	Min Nodes	Cost
AI Deployment + AI Gateway	2 vCPU / 8GB	60GB	2 _{Pods should be spread over min 2 nodes}	~ $120 pm
AI Deployment Only	1 vCPU / 4GB	50GB	2 _{Pods should be spread over min 2 nodes}	~ $60 pm
AI Gateway Only	2 vCPU / 8GB	60GB	2 _{Pods should be spread over min 2 nodes}	~ $120 pm
AI Gateway + Gateway Plane	3 vCPU / 10GB	60GB	2 _{Pods should be spread over min 2 nodes}	~ $120 pm
Postgres (Deployed on Kubernetes)	0.5 vCPU / 0.5GB	5GB
Blob Storage (S3 Compatible)		20GB

The medium tier is configured for production and will suffice teams of 10-500 members.

This configuration can handle upto 10,000 deployments daily and around 50 Kubernetes compute-plane clusters. The AI Gateway is configured for a minimum 3 replicas which can handle around 500 requests/second to LLMs.It’s configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.

Configuration	CPU / Memory	Block Storage PV	Min Nodes	Cost
AI Deployment + AI Gateway	16 vCPU / 56GB	450GB	3 _{Pods should be spread over min 3 nodes}	~ $1000 pm
AI Deployment Only	8 vCPU / 24GB	250GB	3 _{Pods should be spread over min 3 nodes}	~ $400 pm
AI Gateway Only	16 vCPU / 48GB	250GB	3 _{Pods should be spread over min 3 nodes}	~ $800 pm
AI Gateway + Gateway Plane	20 vCPU / 60GB	250GB	3 _{Pods should be spread over min 3 nodes}	~ $1200 pm
Postgres (Managed Database)	2 vCPU / 4GB	30GB
Blob Storage (S3 Compatible)		500GB

The large tier is configured for production and will suffice organizations of 500-50000 members.

This configuration can handle upto 1 million deployments daily and over 500 Kubernetes compute-plane clusters. The AI Gateway is configured for a minimum 10 replicas which can handle around 2000 requests/second to LLMs.It’s configured to be horizontally scalable and autoscale when the load increases. The Block Storage and S3 are used to store LLM request logs. The size is dependent on the size and number of requests and should be set as per the expected usage.

Configuration	CPU / Memory	Block Storage PV	Min Nodes	Cost
AI Deployment + AI Gateway	32 vCPU / 72GB	750GB	10 _{Pods should be spread over min 5 nodes}	~ $1600 pm
AI Deployment Only	16 vCPU / 32GB	300GB	5 _{Pods should be spread over min 5 nodes}	~ $1000 pm
AI Gateway Only	32 vCPU / 64GB	400GB	10 _{Pods should be spread over min 10 nodes}	~ $1400 pm
AI Gateway + Gateway Plane	40 vCPU / 80GB	400GB	10 _{Pods should be spread over min 10 nodes}	~ $1800 pm
Postgres (Managed Database)	8 vCPU / 16GB	100GB
Blob Storage (S3 Compatible)		1000GB

Deploying Control-Plane

There are different scenarios to install the control-plane depending on if you are spinning up a new Kubernetes cluster or using an existing cluster. 1. Spin up a new Kubernetes cluster and deploy control-plane Truefoundry provides OpenTofu/Terraform code to spin up a new Kubernetes cluster and deploy the control-plane on it. You can customize the network configuration and Kubernetes configuration as per your needs. Please follow the guides below:

2. Deploy control-plane on an existing Kubernetes cluster Install the control-plane helm chart using helm install. Follow this guide to install the control-plane helm chart.

If your environment is air-gapped or has restricted network access, refer to the air-gapped FAQ in the control plane or compute plane installation guides for additional setup steps.

​Control-plane Dependencies

​Control-Plane Compute Requirements

​Deploying Control-Plane

Control-plane Dependencies

Control-Plane Compute Requirements

Deploying Control-Plane