Deploy Compute Plane - TrueFoundry Docs

TrueFoundry provides an easy way to attach a new compute plane to the control plane. It provides the OpenTofu/Terraform code using which you can bring up all the necessary components of the compute plane in your cloud account. The key components of the compute plane are:

Kubernetes Cluster - This is the primary compute on which the applications deployed by the control plane run. This can be EKS, GKE, AKS, OpenShift, Oracle Kubernetes Engine, or any other standard Kubernetes cluster. The control plane should have read access to the cluster configuration. The following addons need to be present on the compute-plane cluster depending on your use case. You can bring your own addons and have full flexibility on the configuration of the addons.

ArgoCD (Essential)

TrueFoundry relies on ArgoCD to deploy applications to the compute-plane cluster. The infra applications are deployed in the default project in ArgoCD while the user-deployed applications are deployed in the tfy-apps project.If you are using your own ArgoCD, please make sure of the following requirements:

Ensure ArgoCD has access to create Argo applications in all namespaces. For this, the following must be set:

server.extraArgs[0]="--insecure"
server.extraArgs[1]="--application-namespaces=*"
controller.extraArgs[0]="--application-namespaces=*"

Create a tfy-apps project with the following spec.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: tfy-apps
  namespace: argocd
spec:
  clusterResourceWhitelist:
  - group: '*'
    kind: '*'
  destinations:
  - namespace: '*'
    server: '*'
  sourceNamespaces:
  - '*'
  sourceRepos:
  - '*'

Add the following configuration to the configmap of ArgoCD to force ArgoCD to use labels instead of annotations for the applications.

configs:
  cm:
    application.instanceLabelKey: argocd.argoproj.io/instance
    application.resourceTrackingMethod: label
    controller.resource.health.persist: 'true'

View the default ArgoCD configuration file that TrueFoundry installs.

Istio (Essential)

Istio is a powerful service mesh and ingress controller. TrueFoundry uses Istio as the primary ingress controller in the compute-plane cluster. TrueFoundry does not inject the sidecar by default — it is only injected where needed, for the following use cases:

Request Count Based autoscaling
OAuth-based authentication and authorization for Jupyter Notebooks.
Intercepts feature to redirect / mirror traffic to other applications.
Authentication for services deployed on the cluster.

If you are already using Istio in your cluster, TrueFoundry should be able to work with it without any additional configuration. The TrueFoundry agent automatically discovers the Istio gateways and exposes the domains to the control plane.

Please ensure that if you have multiple Istio gateways, they do not have the same domains configured. If that is the case, you need to specify which gateway to use for the TrueFoundry components as a variable in the tfy-agent helm chart.

There are three Istio components that TrueFoundry installs:

istio-base - The set of CRDs that are required for Istio to work. View the istio-base ArgoCD configuration.
istio-discovery - The pilot service that is responsible for the discovery of the services in the cluster. View the istio-discovery ArgoCD configuration.
tfy-istio-ingress - The ingress gateway that is responsible for the ingress of the services to the cluster. View the tfy-istio-ingress ArgoCD configuration.

Argo Rollouts (Essential)

Argo Rollouts is used to power the canary and blue-green rollout strategies in TrueFoundry.If you are already using Argo Rollouts in your cluster, TrueFoundry should be able to work with it without any additional configuration.View the Argo Rollouts ArgoCD configuration.

Prometheus (Essential)

Prometheus is used to power the metrics feature on the platform. It also powers the autoscaling, autoshutdown, and autopilot features of the platform. TrueFoundry uses the open-source kube-prometheus-stack for running Prometheus in the cluster.If you are already using kube-prometheus-stack in your cluster, TrueFoundry should be able to work with it with the following configuration changes:

  kube-state-metrics:
    metricsLabelsAllowlist:
    - pods=[truefoundry.com/application,truefoundry.com/component-type,truefoundry.com/component,truefoundry.com/application-id]
    - nodes=[karpenter.sh/capacity-type,eks.amazonaws.com/capacityType,kubernetes.azure.com/scalesetpriority,kubernetes.azure.com/accelerator,cloud.google.com/gke-provisioning,node.kubernetes.io/instance-type]

and

  alertmanager:
    alertmanagerSpec:
      alertmanagerConfigMatcherStrategy:
        type: None

View the Prometheus ArgoCD configuration.

Argo Workflows (Optional)

TrueFoundry uses Argo Workflows to power the Jobs feature on the platform.If you are already using Argo Workflows in your cluster, TrueFoundry should be able to work with the following configuration:

  controller:
    workflowDefaults:
      spec:
        activeDeadlineSeconds: 432000
        ttlStrategy:
          secondsAfterCompletion: 3600
    namespaceParallelism: 1000
    parallelism: 1000

View the Argo Workflows ArgoCD configuration.

Keda (Optional)

Keda is used to power the autoscaling feature on the platform. TrueFoundry uses the open-source keda for event-driven autoscaling in the cluster.If you are already using Keda in your cluster, TrueFoundry should be able to work without any additional configuration.View the Keda ArgoCD configuration.

VictoriaLogs (Optional)

VictoriaLogs and Vector are used to power the logs feature on the platform. This is optional and you can choose to provide your own logging solution.If you are already using VictoriaLogs in your cluster, TrueFoundry should be able to work without any additional configuration. If you are already using Vector to ingest logs, TrueFoundry should be able to work with the following configuration:

  customConfig:
    sinks:
      vlogs:
        type: elasticsearch
        query:
          _time_field: timestamp
          _stream_fields: namespace,pod,container,stream,truefoundry_com_application,truefoundry_com_deployment_version,truefoundry_com_component_type,truefoundry_com_retry_number,job_name,sparkoperator_k8s_io_app_name,truefoundry_com_buildName
        inputs:
          - parser
    transforms:
      parser:
        type: remap
        inputs:
          - k8s
        source: >
          if .message == "" {
            .message = " "
          }

          # Extract basic pod information

          .service = .kubernetes.container_name

          .container = .kubernetes.container_name

          .app = .kubernetes.container_name

          .pod = .kubernetes.pod_name

          .node = .kubernetes.pod_node_name

          .namespace = .kubernetes.pod_namespace

          .job_name = .kubernetes.job_name

          # Extract ALL pod labels dynamically using for_each

          pod_labels = object(.kubernetes.pod_labels) ?? {}

          # Iterate through all pod labels and add them with a prefix

          for_each(pod_labels) -> |key, value| {
            label_key = replace(replace(replace(key, ".", "_"), "/", "_"), "-", "_")
            . = set!(., [label_key], string(value) ?? "")
          }

          # Clean up kubernetes metadata

          del(.kubernetes)

          del(.file)
    sources:
      k8s:
        type: kubernetes_logs
        glob_minimum_cooldown_ms: 1000

View the VictoriaLogs ArgoCD configuration.

GPU Operator (Optional)

GPU Operator is used to deploy workloads on the GPU nodes. It’s a TrueFoundry-provided helm chart that’s based on NVIDIA’s GPU Operator.If you are already using NVIDIA’s GPU Operator in your cluster, TrueFoundry should be able to work without any additional configuration.The ArgoCD configuration is available for the following cloud providers:

Grafana (Optional)

Grafana is a monitoring tool that can be installed to view metrics and logs and create dashboards on the cluster. TrueFoundry doesn’t directly use Grafana to power the monitoring dashboard on the platform, but it is available to view additional cluster-level metrics as a separate addon.If you are using Grafana in your cluster, you can use it for monitoring the cluster. But if you want to use the TrueFoundry-provided Grafana, you can install the TrueFoundry Grafana helm chart that comes with a lot of inbuilt dashboards for cluster monitoring.View the Grafana ArgoCD configuration.

[AWS Only] Karpenter (Essential)

Karpenter is required for supporting dynamic node provisioning on AWS EKS.If you are already using Karpenter in your cluster, TrueFoundry should be able to work with the following additional configuration:

Install the eks-node-monitoring-agent helm chart.
Configure Karpenter to use the eks-node-monitoring-agent.

settings:
  featureGates:
    nodeRepair: true

View the Karpenter ArgoCD configuration.TrueFoundry also installs tfy-karpenter-config, another helm chart that installs the nodepools and nodeclasses. If you are already using Karpenter in your cluster, TrueFoundry requires the following nodepool types to be present:

Nodepool Type	Configuration	Purpose
Critical	amd64 linux on-demand nodepool with taint `class.truefoundry.com/component=critical:NoSchedule` and label `class.truefoundry.com/component=critical`	For running TrueFoundry critical workloads like Prometheus, VictoriaLogs, and tfy-agent.
GPU nodepool	amd64 linux on-demand/spot (both) with taint `nvidia.com/gpu=true:NoSchedule` and label `nvidia.com/gpu.deploy.operands=true`	For running user deployed GPU applications.
Default nodepool	amd64 linux on-demand/spot (both) without any taints	For running user deployed CPU applications.

View the tfy-karpenter-config ArgoCD configuration.

[AWS Only] Metrics-Server (Essential)

Metrics-Server is required on the AWS EKS cluster for autoscaling.If you are already using Metrics-Server in your cluster, TrueFoundry should be able to work without any additional configuration.View the Metrics-Server ArgoCD configuration.

[AWS Only] AWS EBS CSI Driver (Essential)

AWS EBS CSI Driver is required for supporting EBS volumes on the EKS cluster.If you are already using AWS EBS CSI Driver in your cluster, TrueFoundry should be able to work without any additional configuration. A default storage class is expected to be present in the cluster, preferably gp3 backed by encrypted volumes.View the AWS EBS CSI Driver ArgoCD configuration.

[AWS Only] AWS EFS CSI Driver (Optional)

AWS EFS CSI Driver is required for supporting EFS volumes on the EKS cluster.If you are already using AWS EFS CSI Driver in your cluster, TrueFoundry should be able to work without any additional configuration. A storage class is expected to be present in the cluster which can be used for mounting EFS volumes.View the AWS EFS CSI Driver ArgoCD configuration.

[AWS Only] AWS Load Balancer Controller (Essential)

AWS Load Balancer Controller is required for supporting the load balancer on EKS.If you are already using AWS Load Balancer Controller in your cluster, TrueFoundry should be able to work without any additional configuration.View the AWS Load Balancer Controller ArgoCD configuration.

[AWS Only] TFY Inferentia Operator (Optional)

TFY Inferentia Operator is required for supporting Inferentia machines on EKS.If you are already using Inferentia Operator in your cluster, TrueFoundry should be able to work without any additional configuration.View the TFY Inferentia Operator ArgoCD configuration.

Cert-Manager (Optional)

Cert-Manager is required for provisioning certificates for exposing services. In AWS you can use the AWS Certificate Manager to provision the certificates. For more details on how to set up the certificates, please refer to the TrueFoundry documentation.

To simplify installation, TrueFoundry packages these addons together in cloud-specific inframold charts, so you do not need to install them one by one:

There are different inframold helm charts per cloud provider since the addons required in the compute-plane differ depending on the cloud provider. For example, Karpenter and Inferentia Operator are required in AWS only. The tolerations and affinity settings also differ for each cloud provider.

Docker Registry - This is the registry where the Docker images built by the control plane are pushed. This can be ECR, GCR, ACR, Quay, Docker Hub, JFrog, Harbor, or any other standard Docker registry. The control plane should have access to push to this registry.
Blob Storage (Optional) - This will be used to store the ML models and artifacts. This is optional and only needed if you intend to use the model registry feature. The control plane should have access to read and write to this blob storage. Possible integrations are AWS S3, Azure Blob Storage, GCP Cloud Storage, MinIO, or any other S3-compatible storage.
Secret Store (Optional) - This will be used to store the secrets for the applications. This is optional and only needed if you intend to use the secret store feature. The control plane should have access to read and write to this secret store. Possible integrations are AWS Parameter Store, AWS Secrets Manager, Azure Key Vault, and GCP Secret Manager.
DNS and Certificates - You need to set up a domain name and point it to the load balancer in the compute-plane cluster. This lets you provide domain names to the workloads deployed on the cluster. The domain name can be a single domain or a wildcard domain. For example, if you point a domain like *.example.com (wildcard domain) to the load balancer, the services will be exposed like service1.example.com, service2.example.com, etc. However, if you point a domain like tfy.example.com (non-wildcard domain) to the load balancer, the services will be exposed like tfy.example.com/service1, tfy.example.com/service2, etc. Many frontend applications do not work with path-based routing (the latter case), so we recommend using wildcard domains. You also need to provision certificates to terminate the TLS traffic on the load balancer. For this you can use the AWS Certificate Manager or cert-manager with GCP Cloud DNS or Azure DNS. You can also bring your pre-created certificates.

FAQ

How to install in an air-gapped / restricted network environment?

In an air-gapped environment, the compute plane cluster cannot pull images or helm charts from the internet. You must make all artifacts available in a private registry and deploy a policy engine to rewrite image/helm references at runtime.

Upload Artifacts to Your Private Registry

First, upload all images shipped as part of your inframold installation to your private registry. You can either:

configure a mirror/pull-through cache for tfy.jfrog.io, or
download and upload artifacts using the script below.

All required images are listed in an artifacts-manifest.json in the infra-charts repository for the inframold chart you are installing.Use the manifest URL that matches your inframold chart:

Inframold Chart	`artifacts-manifest.json` URL
`tfy-k8s-aws-eks-inframold`	https://github.com/truefoundry/infra-charts/blob/main/charts/tfy-k8s-aws-eks-inframold/artifacts-manifest.json
`tfy-k8s-azure-aks-inframold`	https://github.com/truefoundry/infra-charts/blob/main/charts/tfy-k8s-azure-aks-inframold/artifacts-manifest.json
`tfy-k8s-gcp-gke-standard-inframold`	https://github.com/truefoundry/infra-charts/blob/main/charts/tfy-k8s-gcp-gke-standard-inframold/artifacts-manifest.json
`tfy-k8s-generic-inframold`	https://github.com/truefoundry/infra-charts/blob/main/charts/tfy-k8s-generic-inframold/artifacts-manifest.json

Download the artifacts manifest

wget <link_to_artifacts_manifest_for_your_inframold_chart> -O artifacts-manifest.json

Authenticate with your target registry

docker login --username <username> --password <password> <target_registry>

Set up the upload script

Clone the repo and install dependencies:

git clone https://github.com/truefoundry/infra-charts.git
python3 -m venv venv && source venv/bin/activate
pip install -r infra-charts/scripts/upload-artifacts/requirements.txt

Run the upload script for images and helm charts

Run once for images, once for helm charts:

# Push container images
python infra-charts/scripts/upload-artifacts/upload_artifacts.py \
  image artifacts-manifest.json <registry_url>

# Push helm charts
python infra-charts/scripts/upload-artifacts/upload_artifacts.py \
  helm artifacts-manifest.json <registry_url>

The script accepts three arguments:

artifact_type — either image or helm
file_path — path to the artifacts-manifest.json file
destination_registry — your private registry URL

If you set up a mirror or pull-through cache of TrueFoundry’s registry (tfy.jfrog.io) in your own private registry, you can skip the artifact upload step and go directly to the Kyverno setup below.

Configure Kyverno Policy Engine

Kyverno is a CNCF policy engine that intercepts admission requests and rewrites image/helm repository references to point to your private registry. This is required because TrueFoundry does not currently provide a direct way to override all image URIs in the inframold chart values.

Install Kyverno

Create kyverno-values.yaml:

admissionController:
  rbac:
    clusterRole:
      extraResources:
        - apiGroups:
            - 'argoproj.io'
          resources:
            - applications
          verbs:
            - get
            - list
            - watch
            - patch
            - update

backgroundController:
  rbac:
    clusterRole:
      extraResources:
        - apiGroups:
            - 'argoproj.io'
          resources:
            - applications
          verbs:
            - get
            - list
            - watch

reportsController:
  rbac:
    clusterRole:
      extraResources:
        - apiGroups:
            - 'argoproj.io'
          resources:
            - applications
          verbs:
            - get
            - list
            - watch

Install the chart:

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update kyverno
helm install kyverno kyverno/kyverno \
  --namespace kyverno \
  --create-namespace \
  -f kyverno-values.yaml

Install ArgoCD CRDs

These CRDs must be installed before the tfy-kyverno-config chart because the Kyverno policies reference ArgoCD Application resources for Helm repository mutation.

kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/master/manifests/crds/application-crd.yaml

Install the tfy-kyverno-config chart

Create kyverno-config-values.yaml:

replaceImageRegistry:
  enabled: true
  excludeNamespaces:
    - kube-system
    - kube-public
    - kube-node-lease
  includeNamespaces: []
  registryReplacementMap:
    "*": <registry_url>

replaceArgoHelmRepo:
  enabled: true
  excludeNamespaces: []
  includeNamespaces: []
  registryReplacementMap:
    "*": <registry_url>

Install the chart:

helm install tfy-kyverno-config <registry_url>/infra-charts/tfy-kyverno-config \
  -n kyverno -f kyverno-config-values.yaml

Then proceed with the standard compute plane installation. Image references will be transparently redirected to your private registry by the policy engine.

​FAQ

​Upload Artifacts to Your Private Registry

​Configure Kyverno Policy Engine

FAQ

Upload Artifacts to Your Private Registry

Configure Kyverno Policy Engine