Add and List Models - TrueFoundry Docs

TrueFoundry allows you to connect and use models from providers like OpenAI, Anthropic, AWS Bedrock, Google Vertex AI, and many other providers from a single place. Once added, these models can be used in TrueFoundry AI Gateway and your applications. The models are saved as provider integrations in TrueFoundry. The group of models are saved as a provider account. You can add multiple models to a single provider account; the models in the same provider account are called integrations.

Replace YOUR_CONTROL_PLANE_URL with your actual TrueFoundry control plane URL and YOUR_API_KEY with your API key throughout this guide.

Add Models

API Overview

Here is the curl command example to add a model as a provider account:

curl -X PUT "YOUR_CONTROL_PLANE_URL/api/svc/v1/apply" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "manifest": {
      "type": "provider-account/openai",
      "name": "my-openai-account",
      "auth_data": {
        "type": "openai-api-key",
        "api_key": "sk-..."
      },
      "integrations": [
        {
          "type": "openai-model",
          "name": "gpt-4",
          "model_name": "gpt-4",
          "model_type": "chat"
        }
      ]
    },
    "dryRun": false
  }'

Request Schema

The Apply API accepts a TrueFoundryApplyRequest object with the following structure:

ModelProviderAccount | ModelManifest | ArtifactManifest | Service | ...

required

The manifest object defining the resource to be created or updated. For model provider accounts, this should be a ModelProviderAccount object.

boolean

default:"false"

When set to true, performs a dry run without actually creating or updating the resource. Useful for validation.

ModelProviderAccount Schema

For creating model provider accounts, the manifest should be a ModelProviderAccount object. This is a union type that can be one of any of the following provider account types: Supported Provider Account Types:

🔗 AwsBedrockProviderAccount - AWS Bedrock models
🔗 GoogleVertexProviderAccount - Google Vertex AI models
🔗 GoogleGeminiProviderAccount - Google Gemini models
🔗 AzureOpenAIProviderAccount - Azure OpenAI models
🔗 AnthropicProviderAccount - Anthropic Claude models
🔗 OpenaiProviderAccount - OpenAI models
🔗 CohereProviderAccount - Cohere models
🔗 GroqProviderAccount - Groq models
🔗 MistralAIProviderAccount - Mistral AI models
🔗 TogetherAIProviderAccount - Together AI models
🔗 OllamaProviderAccount - Ollama models
🔗 SelfHostedModelProviderAccount - Self-hosted models
For more details, see the ModelProviderAccount type.

Common Fields for All Provider Accounts:

string

required

The provider account type (e.g., “provider-account/openai”, “provider-account/anthropic”, etc.)

string

required

Name of the provider account. Must be 3-32 characters, lowercase alphanumeric with hyphens, cannot start with a number.

object

required

Authentication data specific to the provider (API keys, credentials, etc.)

array

required

List of model integrations available through this provider account

array

Optional list of users who have access to this provider account

Response Schema

{
  "id": "provider-account-123",
  "name": "my-openai-account",
  "fqn": "tenant-name/openai/my-openai-account",
  "provider": "openai",
  "manifest": {
    "type": "provider-account/openai",
    "name": "my-openai-account",
    "auth_data": {
      "type": "openai-api-key",
      "api_key": "sk-..."
    }
  },
  "integrations": [
    {
      "id": "integration-456",
      "name": "gpt-4",
      "fqn": "tenant-name/openai/my-openai-account/model/gpt-4",
      "type": "model",
      "manifest": {
        "type": "openai-model",
        "name": "gpt-4",
        "model_name": "gpt-4",
        "model_type": "chat"
      },
      "providerAccountFqn": "tenant-name/openai/my-openai-account",
      "createdBySubject": {
        "subjectSlug": "user@example.com"
      },
      "createdAt": "2024-01-15T10:30:00Z",
      "updatedAt": "2024-01-15T10:30:00Z"
    }
  ],
  "createdBySubject": {
    "subjectSlug": "user@example.com"
  },
  "createdAt": "2024-01-15T10:30:00Z",
  "updatedAt": "2024-01-15T10:30:00Z",
  "action": "CREATE"
}

Response Fields

string

Unique identifier for the provider account.

string

Name of the provider account.

string

Fully qualified name of the provider account (tenant/provider/name).

string

The provider type (e.g., “openai”, “anthropic”, “aws-bedrock”).

object

The provider account manifest that was applied, excluding integrations.

array

List of provider integrations (models) associated with this account.

object

Information about the user who created the provider account.

string

Timestamp when the provider account was created.

string

Timestamp when the provider account was last updated.

string

The action performed: “CREATE” for new resources, “UPDATE” for existing ones.

Examples

Show OpenAI Provider Account

For connecting to OpenAI’s API:

{
  "type": "provider-account/openai",
  "name": "my-openai-account",
  "auth_data": {
    "type": "openai-api-key",
    "api_key": "sk-your-openai-api-key"
  },
  "integrations": [
    {
      "type": "openai-model",
      "name": "gpt-4",
      "model_name": "gpt-4",
      "model_type": "chat"
    }
  ]
}

Show Anthropic Provider Account

For connecting to Anthropic’s Claude API:

{
  "type": "provider-account/anthropic",
  "name": "my-anthropic-account",
  "auth_data": {
    "type": "anthropic-api-key",
    "api_key": "sk-ant-your-anthropic-api-key"
  },
  "integrations": [
    {
      "type": "anthropic-model",
      "name": "claude-3-sonnet",
      "model_name": "claude-3-sonnet-20240229",
      "model_type": "chat"
    }
  ]
}

Show AWS Bedrock Provider Account

For connecting to AWS Bedrock:

{
  "type": "provider-account/aws-bedrock",
  "name": "my-aws-bedrock-account",
  "region": "us-east-1",
  "auth_data": {
    "type": "aws-access-key",
    "access_key_id": "your-access-key",
    "secret_access_key": "your-secret-key"
  },
  "integrations": [
    {
      "type": "bedrock-model",
      "name": "claude-3-sonnet",
      "model_name": "anthropic.claude-3-sonnet-20240229-v1:0",
      "model_type": "chat"
    }
  ]
}

Error Handling

The Apply API may return various error responses. Common scenarios include:

400 Bad Request: Invalid manifest structure or missing required fields
401 Unauthorized: Invalid or missing API key
403 Forbidden: Insufficient permissions for the operation
404 Not Found: Provider account not found
409 Conflict: Provider account with the same name already exists
422 Unprocessable Entity: Invalid provider credentials or configuration

List Models

You can list and retrieve model integrations using the List Provider Integrations API. This allows you to query existing model integrations with filtering and pagination support.

API Overview

Here is the curl command example to list model integrations:

curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model" \
  -H "accept: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Query Parameters

string

default:"model"

Filter integrations by type. For models, use model.

string

Filter by fully qualified name (FQN) of a specific integration. Cannot be used with id.

string

Filter by integration ID. Cannot be used with fqn.

number

default:"0"

Pagination offset - number of items to skip.

number

Maximum number of items to return. If not specified, returns all items.

Response Schema

{
  "data": [
    {
      "id": "cmfpjat8x04yg01oe1jng7l5l",
      "name": "gpt-4",
      "fqn": "truefoundry:openai:my-openai-account:model:gpt-4",
      "type": "model",
      "manifest": {
        "name": "gpt-4",
        "type": "integration/model/openai",
        "model_id": "gpt-4",
        "model_types": ["chat"],
        "cost": {
          "metric": "public_cost"
        }
      },
      "metadata": {},
      "providerAccountFqn": "truefoundry:openai:my-openai-account",
      "providerAccount": {
        "id": "cmc93zjju01h001pfdr3l437d",
        "name": "my-openai-account",
        "fqn": "truefoundry:openai:my-openai-account",
        "provider": "openai",
        "manifest": {
          "name": "my-openai-account",
          "type": "provider-account/openai",
          "auth_data": {
            "type": "api-key",
            "api_key": "dummy-key"
          },
          "collaborators": [
            {
              "role_id": "provider-account-manager",
              "subject": "user:user@example.com"
            },
            {
              "role_id": "provider-account-access",
              "subject": "team:everyone"
            }
          ]
        },
        "createdBySubject": {
          "subjectId": "cm7t30kps08zz01s78jsgdtru",
          "subjectSlug": "user@example.com",
          "subjectType": "user",
          "subjectPatName": "user-pat-1",
          "subjectDisplayName": "John Doe"
        },
        "createdAt": "2024-01-15T10:30:00.000Z",
        "updatedAt": "2024-01-15T10:30:00.000Z",
        "createdBy": "user@example.com"
      },
      "createdBySubject": {
        "subjectId": "cm7t30kps08zz01s78jsgdtru",
        "subjectSlug": "user@example.com",
        "subjectType": "user",
        "subjectPatName": "user-pat-1",
        "subjectDisplayName": "John Doe"
      },
      "createdAt": "2024-01-15T10:30:00.000Z",
      "updatedAt": "2024-01-15T10:30:00.000Z",
      "createdBy": "user@example.com"
    }
  ],
  "pagination": {
    "total": 1,
    "offset": 0,
    "limit": 10
  }
}

Response Fields

array

Array of model integration objects.

Show Integration Object Properties

string

Unique identifier for the model integration.

string

Name of the model integration.

string

Fully qualified name of the integration in the format: tenant:provider:account:model:name (colon-separated).

string

Type of the integration (e.g., “model”).

object

The model integration manifest (auth_data is excluded for security).

object

Additional metadata associated with the integration.

string

FQN of the provider account this integration belongs to.

object

Full provider account object that contains this integration.

Show Provider Account Properties

string

Unique identifier for the provider account.

string

Name of the provider account.

string

Fully qualified name of the provider account.

string

Provider type (e.g., “openai”, “anthropic”, “aws-bedrock”).

object

Provider account manifest including auth_data and collaborators.

object

Information about the user who created the provider account.

string

Timestamp when the provider account was created.

string

Timestamp when the provider account was last updated.

string

ID of the user who created the provider account.

object

Information about the user who created the integration.

Show Subject Properties

string

Unique identifier for the subject.

string

Type of subject (e.g., “user”).

string

Slug identifier for the subject (e.g., email address).

string | null

Personal Access Token name if the action was performed using a PAT, or null.

string | null

Display name of the subject, or null if not available.

string

Timestamp when the integration was created.

string

Timestamp when the integration was last updated.

string

ID of the user who created the integration.

object

Pagination metadata.

Show Pagination Properties

number

Total number of integrations matching the query.

number

Current pagination offset.

number

Maximum number of items returned per page.

Examples

Show List All Models

To list all model integrations in your tenant:

curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model" \
  -H "accept: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Show List Models with Pagination

To retrieve models with pagination (10 items starting from offset 20):

curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&limit=10&offset=20" \
  -H "accept: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Show Get Specific Model by FQN

To retrieve a specific model by its fully qualified name:

curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&fqn=truefoundry:openai:my-openai-account:model:gpt-4" \
  -H "accept: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Show Get Specific Model by ID

To retrieve a specific model by its integration ID:

curl -X GET "YOUR_CONTROL_PLANE_URL/api/svc/v1/provider-integrations?type=model&id=integration-456" \
  -H "accept: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY"

Error Handling

The List API may return various error responses. Common scenarios include:

400 Bad Request: Invalid query parameters (e.g., both fqn and id provided)
401 Unauthorized: Invalid or missing API key
404 Not Found: Integration with specified id or fqn not found

​Add Models

​API Overview

​Request Schema

​ModelProviderAccount Schema

​Response Schema

​Response Fields

​Examples

​Error Handling

​List Models

​API Overview

​Query Parameters

​Response Schema

​Response Fields

​Examples

​Error Handling

Add Models

API Overview

Request Schema

ModelProviderAccount Schema

Response Schema

Response Fields

Examples

Error Handling

List Models

API Overview

Query Parameters

Response Schema

Response Fields

Examples

Error Handling