Skip to main content
AI Gateway is OpenTelemetry (OTEL) compliant, making it easy to integrate with modern observability tools and platforms. Tracing and metrics can each be exported from the TrueFoundry dashboard to external OTEL backends. Self-hosted gateways also run a disk-backed OTEL buffer in the gateway plane by default. For Prometheus scraping of /metrics, labels, and the full metric catalog on self-hosted gateways, see Prometheus & Grafana integration.
We have built integrations with many observability partners — see the full list here.

​
Tracing

The OTEL Traces Exporter sends only trace data to your OTEL platform, not metrics. For OpenTelemetry metrics export, see Metrics below.
OpenTelemetry tracing allows you to capture detailed traces of requests as they flow through the AI Gateway. This enables debugging, performance analysis, and end-to-end visibility. Traces storage is managed by TrueFoundry. Optionally, traces can also be configured to be exported to any existing OTEL compatible observability platform. This will not affect the storage of traces managed by TrueFoundry.
OpenTelemetry traces storage and export flow in TrueFoundry

AI Gateway - OpenTelemetry Tracing Flow

​
Export Traces to OTEL Compatible Observability Platforms

To export spans to your existing OTEL Platform:
OpenTelemetry Config form for exporting spans to existing OTEL platform request with spans

AI Gateway - OpenTelemetry Tracing Export Config Form

  1. Go to AI Gateway → Controls → Settings in the TrueFoundry dashboard
  2. Scroll down to the OTEL Config section and click the edit button
  3. Enable the OTEL Traces Exporter Configuration toggle
  4. Select either HTTP Configuration or gRPC Configuration based on your OTEL platform’s endpoint
  5. Enter the complete traces endpoint for eg., the traces endpoint for Grafana Cloud is https://otlp-gateway-prod-ap-south-1.grafana.net/otlp/v1/traces
  6. Select the encoding (only for HTTP Configuration) as either proto or json
  7. Optionally, enter the headers required to export spans to the OTEL provider for eg., authorization, region, etc..
For gRPC endpoints, always include the scheme. Use http:// for plaintext (e.g., http://otel-collector:4317). If omitted, TLS is used by default.

​
Buffer traces for hybrid gateways

The OTEL buffer runs next to the AI Gateway in your gateway plane. It stores gateway traces on disk and forwards them to the TrueFoundry control plane, so traces are not lost when the control plane is briefly unreachable. It can also send the same traces to your own OpenTelemetry collectors. The buffer is on by default for self-hosted gateways (on-prem, SaaS-operated, and hybrid). Set tfy-llm-gateway.enableOtelBuffer: false only if you want the gateway to export traces to the control plane directly.
Use additional sinks when you want to fan out the same traces to multiple collectors—not only TrueFoundry’s OTEL collector. To export from the control plane dashboard without the gateway-plane buffer, use the OTEL Config above.

​
Send traces to other collectors

Add destinations under tfy-otel-buffer.additionalSinks. They receive the same traces as the control plane.
1

Store the collector token in a Secret

2

Add the sink to your values file

truefoundry-values.yaml
The traces now appear both in TrueFoundry and in your collector.

​
FAQ

Any collector that accepts OTLP over HTTP. Use protocol.type: http and the collector’s /v1/traces endpoint. OTLP over gRPC is not supported in the current TrueFoundry release.
Add one key per credential to the tfy-otel-buffer-sink-credentials Secret and reference it in the sink as SECRET[additional_sink_credentials.<KEY>], for example in a header.After you change a key, restart the buffer so it picks up the new value:
Each destination buffers and retries on its own, so an unreachable collector does not delay traces to the control plane or to your other collectors.
The disk buffers of all additional sinks together can use up to 5 GiB per replica. This limit can be increased by increasing the buffer volume size.

​
Metrics

The OTEL Metrics Exporter sends only metrics to your OTEL platform, not traces. For OpenTelemetry traces export, see Tracing above.
OpenTelemetry metrics export lets you send gateway metrics to an OTLP-compatible observability platform. Configure this in the same OTEL Config editor as traces; it does not replace metrics stored and shown inside TrueFoundry.
OpenTelemetry metrics storage and export flow in TrueFoundry

AI Gateway - OpenTelemetry Metrics Flow

​
Export Metrics to OTEL Compatible Observability Platforms

To export metrics to your existing OTEL platform:
OpenTelemetry Config form for exporting metrics to an OTEL platform

AI Gateway - OpenTelemetry Metrics Export Config Form

  1. Go to AI Gateway → Controls → Settings in the TrueFoundry dashboard
  2. Scroll down to the OTEL Config section and click the edit button
  3. Enable the OTEL Metrics Exporter Configuration toggle
  4. Select either HTTP Configuration or gRPC Configuration based on your OTEL platform’s metrics endpoint
  5. Enter the complete metrics endpoint—for example, Datadog uses https://otlp.datadoghq.com/v1/metrics
  6. Select the encoding (only for HTTP Configuration) as either proto or json
  7. Optionally, enter the headers required to export metrics to the OTEL provider—for example, API keys or vendor-specific headers
  8. Optionally, enable Export Delta Metrics to send counters and histograms with delta temporality instead of cumulative. Enable this for backends such as Datadog that expect delta-encoded metrics.
  9. Click Save
For gRPC endpoints, always include the scheme. Use http:// for plaintext (e.g., http://otel-collector:4317). If omitted, TLS is used by default.

​
Example 1: Exporting Metrics to Datadog

Datadog supports OTLP metrics ingest natively. Use the following configuration: Step 1 — Set the environment variable on your tfy-llm-gateway deployment Datadog’s OTLP ingest endpoint only accepts delta-encoded metrics. Set this environment variable in your tfy-llm-gateway deployment before configuring the exporter:
Step 2 — Configure the OTEL Metrics Exporter in TrueFoundry In the OTEL Config editor, enable OTEL Metrics Exporter Configuration and set: Add the following headers: The dd-otel-metric-config header enables the histogram translator, which converts OTLP histogram metrics into Datadog distribution metrics. This is required to query percentile aggregations such as p50:ai_gateway_request_processing_ms{*}, p95:ai_gateway_request_processing_ms{*}, and similar metrics in the Datadog Metrics Explorer.
For sites other than US1, use the region-specific endpoint—for example, https://otlp.datadoghq.eu/v1/metrics for the EU site. See the Datadog OTLP metrics ingest documentation for the full list of endpoints and additional dd-otel-metric-config options.

​
Example 2: Exporting Metrics to AWS Cloudwatch

AWS Cloudwatch supports OTLP metrics ingest. Use the following configuration: Step 1 - Set the environment variable on your tfy-llm-gateway deployment Cloudwatch’s OTLP ingest endpoint only accepts delta-encoded metrics. Set this environment variable in your tfy-llm-gateway deployment before configuring the exporter:
OTEL Delta metric temporality does not export Gauges and UpDownCounters. Any metrics of these types might not be sent with Delta temporality.
For scraping /metrics with Prometheus, OTLP endpoint patterns, and a full list of metric names and labels, see Prometheus & Grafana integration.