Azure - TrueFoundry Docs

Architecture

The diagram below shows the architecture of the TrueFoundry compute plane.

Access Policies

The compute plane requires the following IAM policies to operate. These are created automatically during platform deployment.

Required IAM policies

Policy	Description
Access required for Azure container registry, storage account	An azure container registry is used to store the docker images for the platform. A storage account is used to store the model artifacts.
Azure AD application with `Reader` and `Monitoring Reader` on AKS	Reader and monitoring reader permission on AKS is used to access the cluster autoscaler logs in Log Analytics and read azure node pools. User should have access to create Azure AD application.

Requirements

Common requirements for setting up the compute plane:

Billing must be enabled for the Azure subscription.
The Microsoft.Storage resource provider must be registered. See the Azure host-based encryption documentation for details.
Egress access to container registries — public.ecr.aws, quay.io, ghcr.io, tfy.jfrog.io, docker.io/natsio, nvcr.io, registry.k8s.io — to pull images for ArgoCD, NATS, GPU Operator, Argo Rollouts, Argo Workflows, Istio, and Keda.
A domain mapped to the service endpoints, plus a certificate to encrypt traffic. A wildcard domain such as *.services.example.com is preferred. TrueFoundry supports path-based routing (e.g. services.example.com/tfy/*), but many frontend applications do not.
Sufficient CPU/GPU quotas for your use case. Check and increase them at Azure compute quotas.
Host encryption must be enabled.
Sufficient permissions on the Azure subscription to create the compute-plane resources. See Permissions required to create the infrastructure for details.

New network and New AKS Cluster
Existing network and New AKS Cluster
Existing AKS Cluster

The new VPC subnet should have a CIDR range of /24 or larger. Secondary ranges for pods (min /20) and services (min /24) are required. Secondary range can be from a non-routable range. This is to ensure capacity for ~250 instances and 4096 pods.
A user/service account to provision the infrastructure.

AKS Version should be 1.30 or later.
Following node pools must be created:

Critical CPU on-demand node pool (2vCPU, 8GB RAM, min 2 nodes)

Create one node pool for running TrueFoundry critical workloads like Prometheus, Loki, and tfy-agent. Add the taint class.truefoundry.com/component=critical:NoSchedule and label class.truefoundry.com/component=critical on this node pool. The min-instance count should be 2 and max-instance count should be 5.

CPU on-demand node pools

These node pools are for running the user-deployed applications. Create 2-3 on-demand CPU node pools with varying configuration depending on the requirement of the workloads. The min instance count can be configured to 0 and max-instance count can be configured to 10. A few sample instance types to choose from: Standard_D4ds_v5, Standard_D8ds_v5, Standard_D16ds_v5. This depends on the expected usage of the cluster.

CPU spot node pools

These node pools are for running the user-deployed applications. Create 2-3 spot CPU node pools with varying configuration depending on the requirement of the workloads. The min instance count can be configured to 0 and max-instance count can be configured to 10. A few sample instance types to choose from: Standard_D4ds_v5, Standard_D8ds_v5, Standard_D16ds_v5. This depends on the expected usage of the cluster.

GPU on-demand node pools

These node pools are for running the user-deployed applications. If you are planning to use GPU instances, create 2-3 on-demand GPU node pools with different types of GPUs — for example, instance types Standard_NC4as_T4_v3, Standard_NC24ads_A100_v4, Standard_NV6ads_A10_v5. The min instance count should be configured to 0 and max-instance count can be configured to 10.

GPU spot node pools

These node pools are for running the user-deployed applications. If you are planning to use GPU instances, create 2-3 spot GPU node pools with different types of GPUs — for example, instance types Standard_NC4as_T4_v3, Standard_NC24ads_A100_v4, Standard_NV6ads_A10_v5. The min instance count should be configured to 0 and max-instance count can be configured to 10.

Compute quotas should be increased for the cluster. Required to ensure sufficient resources are available. Spot instances can help optimize costs for interruptible workloads.

Permissions required to create the infrastructure

Contributor + User Access Administrator on the subscription (or Owner) is the simplest. For the exact least-privilege set, the custom role definition below is the base cluster role plus the compute-plane (blob + registry) delta. Set AssignableScopes before running az role definition create — wildcards in Actions do not expand the scope. The role includes Microsoft.Resources/subscriptions/resourceGroups/*, so you do not need to create the resource group first — the deploy creates it. Scope depends on which you use:

New resource group (deploy creates it) — scope the role at the subscription (/subscriptions/<subscription-id>), because resourceGroups/write is evaluated at subscription scope before the group exists.
Existing resource group — scope to that group (/subscriptions/<subscription-id>/resourceGroups/<resource-group>) for tighter least privilege.

{
  "Name": "TrueFoundry Deploy - Azure Compute Plane",
  "IsCustom": true,
  "Description": "Least-privilege role to provision the TrueFoundry compute plane (base cluster + blob + registry).",
  "Actions": [
    "Microsoft.Resources/subscriptions/resourceGroups/*",
    "Microsoft.Resources/deployments/*",
    "Microsoft.Resources/tags/*",
    "Microsoft.Authorization/roleAssignments/read",
    "Microsoft.Authorization/roleAssignments/write",
    "Microsoft.Authorization/roleAssignments/delete",
    "Microsoft.Authorization/roleDefinitions/read",
    "Microsoft.ManagedIdentity/userAssignedIdentities/*",
    "Microsoft.Network/virtualNetworks/*",
    "Microsoft.Network/networkSecurityGroups/*",
    "Microsoft.Network/routeTables/*",
    "Microsoft.Network/publicIPAddresses/*",
    "Microsoft.Network/natGateways/*",
    "Microsoft.Network/privateDnsZones/*",
    "Microsoft.ContainerService/managedClusters/*",
    "Microsoft.OperationalInsights/workspaces/*",
    "Microsoft.OperationsManagement/solutions/*",
    "Microsoft.Insights/dataCollectionRules/*",
    "Microsoft.Insights/dataCollectionRuleAssociations/*",
    "Microsoft.Insights/diagnosticSettings/*",
    "Microsoft.Storage/storageAccounts/*",
    "Microsoft.ContainerRegistry/registries/*"
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": [],
  "AssignableScopes": [
    "/subscriptions/<subscription-id>/resourceGroups/<resource-group>"
  ]
}

The base role includes Microsoft.Authorization/roleAssignments/write because the AKS module assigns roles (e.g. Network Contributor) to managed identities. If you use the built-in shortcut instead of this custom role, plain Contributor is not enough — use Contributor + User Access Administrator.

Microsoft Graph permission

When Cluster Integration is on (the default), the compute plane creates a Microsoft Entra application and service principal. This is not Azure RBAC — it is a directory write behind the Microsoft Graph API, so no ARM role above covers it. Grant the deploy service principal the Microsoft Graph application permission Application.ReadWrite.OwnedBy (with admin consent):

This is a separate grant from the Azure RBAC role above, and the two are independent:

Grant	System	Governs	Scope
Custom role (above)	Azure RBAC	Creating Azure resources (cluster, network, storage)	Subscription or resource group — depends on new vs. existing RG
`Application.ReadWrite.OwnedBy` (here)	Microsoft Graph	Creating the Entra app + service principal in the directory	None — it is a directory-level permission, not scoped to any subscription or resource group

So Application.ReadWrite.OwnedBy is always the same regardless of whether the resource group is created before or after — the new-vs-existing-RG choice only affects where you assign the RBAC role, never this Graph permission.

# Microsoft Graph app id is the well-known 00000003-0000-0000-c000-000000000000
# Application.ReadWrite.OwnedBy app-role id: 18a4783c-866b-4cc7-a460-3d5e5662c884
az ad app permission add \
  --id "<deploy-sp-app-id>" \
  --api 00000003-0000-0000-c000-000000000000 \
  --api-permissions 18a4783c-866b-4cc7-a460-3d5e5662c884=Role

az ad app permission admin-consent --id "<deploy-sp-app-id>"

Granting Graph permissions is itself a privileged Entra operation performed by a tenant admin (Privileged Role Administrator / Global Administrator), separate from subscription RBAC. If you disable Cluster Integration, no Graph permission is required and the role above alone suffices.

Setting up compute plane

TrueFoundry compute plane infrastructure is provisioned using OpenTofu/Terraform. You can download the OpenTofu/Terraform code for your exact account by filling up your account details and downloading a script that can be executed on your local machine.

Enable Deployment Feature in the Platform (Optional)

To enable the deployment feature which allows you to deploy services through the platform, you need to enable it;

In the left hand navigation, go to Settings then Platform Feature Visibility under Preferences
Click on Edit button. Then enable the toggle for Enable Deployment

Click on Save button.

This will enable the deployment feature in the platform and allow you to create either a control plane and compute plane.

Choose to create a new cluster or attach an existing cluster

Go to the platform section in the left panel and click on Clusters. You can click on Create New Cluster or Attach Existing Cluster depending on your use case. Read the requirements and if everything is satisfied, click on Continue.

Fill up the form to generate the OpenTofu/Terraform code

A form will be presented with the details for the new cluster to be created. Fill in with your cluster details. Click Submit when done.

Create New Cluster
Attach Existing Cluster

The key fields to fill up here are:

Region - The region and availability zones where you want to create the cluster.
Resource Group - The resource group where you want to create the cluster. Chose between New Resource Group or Existing Resource Group depending on your use case.
Cluster Name - A name for your cluster.
Kubernetes Version - The Kubernetes version for the cluster (e.g. 1.34).

Node Pools - Configure CPU and GPU node pools for the cluster. The form comes with sensible defaults (see below) which you can adjust based on your workload requirements.

The default node pool configuration is:

Pool	Type	Instance Type	Capacity	Min	Max
initial (system)	On-Demand	`Standard_D4ds_v5`	CPU	2	2
cpu	On-Demand	`Standard_D4ds_v5`	CPU	0	2
cpu2x	On-Demand	`Standard_D8ds_v5`	CPU	0	2
a10	On-Demand	`Standard_NV6ads_A10_v5`	GPU	0	2
t4	On-Demand	`Standard_NC4as_T4_v3`	GPU	0	2

The initial pool is the system node pool that runs TrueFoundry platform components (ArgoCD, Istio, tfy-agent, etc.) and must always be on-demand with at least 2 nodes. You can add, remove, or resize the other CPU/GPU pools to match your workload needs. GPU pools can be removed entirely if you don’t plan to run GPU workloads. Make sure you have sufficient Azure compute quotas for the instance types you select.

Network Configuration - Choose between New Vnet or Existing Vnet depending on your use case.
DNS Configuration - Configure the DNS zone and domains that will point to the cluster’s load balancer. This also provisions a TLS certificate for those domains. Select New DNS Zone or Existing DNS Zone if you want TrueFoundry to manage DNS in Azure. If you use an external DNS provider (e.g., Route53, Cloudflare), you can skip this section.
Storage account (container) for OpenTofu/Terraform State - OpenTofu/Terraform state will be stored in this container. It can be a preexisting storage account or a new storage account name. The new storage account will automatically be created by our script.
Platform Features - This is to decide which features like Blob Storage, Cluster Integration (using Azure AD), and Container Registry will be enabled for your cluster. To read more on how these integrations are used in the platform, please refer to the platform features page.

The key fields to fill up here are:

Region - The region and availability zones where you want to create the cluster.
Resource Group - The resource group where the cluster is already created.
Cluster Name - Your cluster name.
Network Configuration - Existing Vnet and subnet details.
DNS Configuration - Configure the DNS zone and domains that will point to the cluster’s load balancer. This also provisions a TLS certificate for those domains. Select New DNS Zone or Existing DNS Zone if you want TrueFoundry to manage DNS in Azure. If you use an external DNS provider (e.g., Route53, Cloudflare), you can skip this section.
Cluster Addons - TrueFoundry needs to install addons like ArgoCD, Argo Workflows, Keda, Istio, etc. Please disable the addons that are already installed on your cluster so that the TrueFoundry installation does not override the existing configuration and affect your existing workloads.
Storage account (container) for OpenTofu/Terraform State - OpenTofu/Terraform state will be stored in this container. It can be a preexisting storage account or a new storage account name. The new storage account will automatically be created by our script.
Platform Features - This is to decide which features like Blob Storage, Cluster Integration (using Azure AD), and Container Registry will be enabled for your cluster. To read more on how these integrations are used in the platform, please refer to the platform features page.

Copy the curl command and execute it on your local machine

You will be presented with a curl command to download and execute the script. The script will take care of installing the prerequisites, downloading OpenTofu/Terraform code, and running it on your local machine to create the cluster. This will take around 40-50 minutes to complete.

Verify the cluster is showing as connected in the platform

Once the script is executed, the cluster will be shown as connected in the platform.

Create DNS Record

You can get the load balancer’s IP address by going to the platform section in the bottom left panel under the Clusters section. Under the preferred cluster, you’ll see the load balancer IP address under the Base Domain URL section.

Create a DNS record in your Azure DNS Zone or your DNS provider with the following details:

Record Type	Record Name	Record value
A	*.tfy.example.com	LOADBALANCER_IP_ADDRESS

Start deploying workloads to your cluster

You can start by deploying your first service.

FAQ

Can I use my own certificate and key files to add TLS to the load balancer?

If you have your own certificate files (for example, from another certificate provider or self-signed), you can use them directly with TrueFoundry.

Create a Kubernetes secret with your certificate and key, or create a self-signed certificate:

# Generate a self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout tls.key -out tls.crt \
  -subj "/CN=*.example.com" \
  -addext "subjectAltName = DNS:example.com,DNS:*.example.com"

# Create secret from local certificate files
kubectl create secret tls example-com-tls \
  --cert=path/to/cert/file \
  --key=path/to/key/file \
  -n istio-system

Once the secret is created, head over to the cluster page and navigate to the tfy-istio-ingress add-on. Add the secret name in the tfyGateway.spec.servers[1].tls.credentialName section and ensure that tfyGateway.spec.servers[1].port.protocol is set to HTTPS. Here we are using example-com-tls as the secret name, which contains the certificate and key.

    servers:
      - <REDACTED>
      - hosts:
          - "*"
        port:
          name: https-tfy-wildcard
          number: 443
          protocol: HTTPS
        tls:
          mode: SIMPLE
          credentialName: example-com-tls

Self-signed certificates will cause browser warnings. They should only be used for testing or internal systems. To connect to services with self-signed certificates, you have to pass the CA certificate to verify the SSL certificate.

How do I add node pools after cluster creation?

If you need to add or modify node pools after the cluster is created, you can do so using the Azure CLI. Set the following variables before running the commands:

export RESOURCE_GROUP="<your-resource-group>"
export CLUSTER_NAME="<your-cluster-name>"
export INSTANCE_SIZE="<instance-type>"   # e.g. Standard_D4ds_v5

You can browse available instance types and pricing at azureprice.net.

On-Demand CPU
Spot CPU
On-Demand GPU
Spot GPU

az aks nodepool add \
    --resource-group $RESOURCE_GROUP \
    --cluster-name $CLUSTER_NAME \
    --name <NODEPOOL_NAME> \
    --enable-cluster-autoscaler \
    --enable-encryption-at-host \
    --node-vm-size $INSTANCE_SIZE \
    --min-count 0 \
    --node-count 0 \
    --node-osdisk-size 100 \
    --max-count 10 \
    --no-wait

az aks nodepool add \
    --resource-group $RESOURCE_GROUP \
    --cluster-name $CLUSTER_NAME \
    --name <NODEPOOL_NAME> \
    --priority Spot \
    --eviction-policy Delete \
    --spot-max-price -1 \
    --enable-cluster-autoscaler \
    --enable-encryption-at-host \
    --node-vm-size $INSTANCE_SIZE \
    --min-count 0 \
    --node-count 0 \
    --max-count 10 \
    --no-wait

Spot pools automatically get the taint kubernetes.azure.com/scalesetpriority:spot. Pods must tolerate this taint to be scheduled on spot nodes. TrueFoundry handles this automatically when you select a spot node pool during deployment.

az aks nodepool add \
    --resource-group $RESOURCE_GROUP \
    --cluster-name $CLUSTER_NAME \
    --name <NODEPOOL_NAME> \
    --enable-cluster-autoscaler \
    --enable-encryption-at-host \
    --node-vm-size $INSTANCE_SIZE \
    --node-taints nvidia.com/gpu=Present:NoSchedule \
    --min-count 0 \
    --node-count 0 \
    --node-osdisk-size 100 \
    --max-count 2 \
    --mode user \
    --no-wait

The nvidia.com/gpu=Present:NoSchedule taint prevents non-GPU workloads from being scheduled on expensive GPU nodes.

az aks nodepool add \
    --resource-group $RESOURCE_GROUP \
    --cluster-name $CLUSTER_NAME \
    --name <NODEPOOL_NAME> \
    --priority Spot \
    --eviction-policy Delete \
    --spot-max-price -1 \
    --enable-cluster-autoscaler \
    --enable-encryption-at-host \
    --node-vm-size $INSTANCE_SIZE \
    --node-taints nvidia.com/gpu=Present:NoSchedule \
    --min-count 0 \
    --node-count 0 \
    --max-count 2 \
    --mode user \
    --no-wait

New node pools are automatically synced in TrueFoundry if the Azure AD application has Reader access on the AKS cluster.

When should I use spot vs on-demand node pools?

	On-Demand	Spot
Availability	Guaranteed — no interruptions	Can be reclaimed by Azure at any time
Cost	Standard pricing	Up to 60-90% cheaper
Best for	Production services, databases, platform components	Dev/test workloads, batch jobs, interruptible tasks

The system node pool (initial) must always be on-demand. For user workloads, spot pools can bring significant cost savings if the application can tolerate interruptions. Make sure you have sufficient spot quotas in your region.

​Architecture

​Access Policies

​Requirements

​Permissions required to create the infrastructure

​Microsoft Graph permission

​Setting up compute plane

​FAQ

Architecture

Access Policies

Requirements

Permissions required to create the infrastructure

Microsoft Graph permission

Setting up compute plane

FAQ