Skip to main content
API Reference: POST /moderations

​
Provider capabilities

The table below summarizes gateway support for this endpoint by provider.
Legend:
  • ✅ Supported by Provider and Truefoundry
  • Provided by provider, but not by Truefoundry
  • Provider does not support this feature
For every gateway endpoint and provider, see Supported APIs.

​
Supported Providers

  • OpenAI
Use the moderations endpoint to identify content that may violate usage policies, such as text or images involving violence, hate speech, harassment, self-harm, or sexual content. You can use two models for this endpoint:
  • omni-moderation-latest: This model and all snapshots support more categorization options and multi-modal inputs.
  • text-moderation-latest (Legacy): Older model that supports only text inputs and fewer input categorizations. The newer omni-moderation models will be the best choice for new applications.
You can add these models in your OpenAI provider account. Code Snippets
Here’s a full example output, where the input is an image from a single frame of a war movie. The model correctly predicts indicators of violence in the image, with a violence category score of greater than 0.8.

​
Key Fields

  • flagged: true if any content is considered harmful.
  • categories: Dictionary of category flags, true means the input falls under that harmful category.
  • category_scores: Probability scores (0 to 1) for each category.
  • category_applied_input_types : Indicates whether the category was triggered by the text or image.