Submitting content to Moderation API

How to submit content for moderation and which content types you can analyze with Moderation API.

To analyze content, send a POST request to the /moderate endpoint. The API accepts different content types (text, image, object, video, audio) and returns results immediately.

---

Node.js example

import ModerationAPI from "@moderation-api/sdk";

// Configure with environment variable MODAPI_SECRET_KEY
const moderationApi = new ModerationAPI();

// Text moderation
const textResult = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Hello world!",
  },
  // Optional content data
  contentId: "text-1",
  authorId: "user-123",
  conversationId: "room-456",
  metadata: {
    customField: "value",
  },
});

// Use the API's recommendation
if (textResult.recommendation.action === "reject") {
  // Block the content
} else if (textResult.recommendation.action === "review") {
  // Send to moderation queue
} else {
  // Content approved - add to database
}

// Image moderation
const imageResult = await moderationApi.content.submit({
  content: {
    type: "image",
    url: "https://example.com/image.jpg",
  },
  // Optional content data
  contentId: "image-1",
  authorId: "user-123",
  metadata: {
    customField: "value",
  },
});

// Simple flagged check
if (imageResult.evaluation.flagged) {
  // Return error to user etc.
} else {
  // Add to database etc.
}

---

Content metadata

You can add metadata to the content you send for moderation. Some fields are used by the moderation pipeline to improve accuracy, while others enhance the dashboard experience.

contentId

Specify a contentId to associate the request with specific content. This is typically the content’s unique identifier from your database.

If you don’t specify a contentId, the API generates a random ID for the content.

When you include a contentId, submitting the same ID again updates the existing content. This is useful when using review queues - you can update content in the queue without creating duplicate items.

The contentId can also be used to execute actions in the review queue programmatically. For example, you can allow users to report content on your platform and then add it to a review queue.

conversationId

Use conversationId to group related content together, such as messages in a chatroom or comments on a post.

If you’re using review queues, the conversationId can filter the queue to show content from a specific conversation.

Enable context awareness to improve moderation accuracy using the conversationId.

authorId

Use authorId to identify the user who created the content.

This enables user-level moderation in review queues and allows filtering by specific users.

Enable context awareness to improve moderation accuracy using the authorId.

metaType

Use metaType to specify what kind of content you’re moderating. This helps the API apply appropriate analysis:

Value	Use case
message	Chat messages, direct messages
post	Forum posts, social media posts
comment	Comments on posts or articles
review	Product or service reviews
profile	User profile information
product	Product listings
event	Event descriptions
other	Any other content type

const result = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Great product, highly recommend!",
  },
  metaType: "review",
  authorId: "user-123",
});

channel

Use channel to route content to a specific channel configuration. If not provided, the project’s default channel is used.

const result = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Hello world!",
  },
  channel: "high-risk-content",
});

metadata

Use metadata to attach any additional information to the request. This object can contain custom key-value pairs.

Metadata is displayed in review queues and included in webhooks.

If you add a link in metadata, it will be clickable from the review queue. This is useful for linking back to the original content in your application.

clientAction

Use clientAction to submit your own recommendation alongside the content. This is for cases where you already run your own flagging — a blocklist, a third-party tool, or your own logic — and want that decision reflected in the moderation result and your review queues.

Without it, your own flags live outside Moderation API, so content you’ve flagged but our analysis considered clean never shows up where your moderators work. clientAction brings the two together.

const result = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Check out my site: example.com",
  },
  contentId: "post-789",
  authorId: "user-123",
  clientAction: {
    action: "review",
    behavior: "escalate",
    source: "ip-blocklist",
    reason: "Author IP is on our internal blocklist",
  },
});

// recommendation.action is now at least "review", and
// recommendation.reason_codes includes "client_override"

The clientAction object accepts:

Field	Required	Description
action	Yes	Your recommendation: allow, review, or reject
behavior	No	How your recommendation combines with ours (see below). Defaults to escalate
source	No	Where your recommendation came from, e.g. "ip-blocklist". Shown in the review queue
reason	No	A human-readable explanation, shown in the review queue

Behavior controls how your recommendation combines with the API’s own recommendation:

Behavior	What happens
escalate (default)	Your recommendation is applied only when it’s stricter than ours — reject is stricter than review, which is stricter than allow. This lets you push content into review or reject, but never loosens the API’s own decision.
override	Your recommendation replaces the API’s recommendation entirely, even if that means loosening it.

When your recommendation changes the outcome, the response includes the client_override reason code.

clientAction sets recommendation.action and adds the client_override reason code, but it does not change evaluation.flagged — that always reflects the API’s own analysis.

If you rely on review queues, make sure the queue is configured to include these items — for example by showing all items rather than only flagged ones — so content you’ve escalated still appears. See queue configuration.

---

Context awareness

Enable Context awareness in your channel settings, then include authorId and/or conversationId in API requests. This allows models to analyze previous messages for improved accuracy.

When context awareness is enabled, models analyze the current message alongside recent conversation history. The API retrieves previous messages with the same conversationId or authorId and provides them to the model sequentially, allowing it to understand the full context before making a decision.

LLM-based policies can use the conversationId to see previous messages in the same conversation, and authorId to see previous messages from the same author.

This can prevent unwanted content spread across multiple messages:

msg 1 -> f
msg 2 -> u
msg 3 -> c
msg 4 -> k [FLAGGED with context awareness]

It also helps understand messages in the context of a conversation:

user 1 -> What's the worst thing you know?
user 2 -> European people [FLAGGED with context awareness]

---

Content types

The /moderate endpoint accepts different content types through the content object:

Text

Analyze text content

Image

Analyze image content

Object

Analyze mixed content types

Video (enterprise)

Analyze video content

Audio (enterprise)

Analyze audio content

Text

Text moderation is the most common type. Use it for:

• Chat messages
• Forum posts
• Comments
• Reviews
• Product descriptions
• Profile bios

const result = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Hello world!",
  },
});

If you’re analyzing chat messages or thread-based content, enable context awareness for better accuracy.

Image

Image moderation analyzes visual content to detect inappropriate or harmful images, including nudity, violence, or other objectionable content.

const result = await moderationApi.content.submit({
  content: {
    type: "image",
    url: "https://example.com/image.jpg",
  },
});

Object

Object moderation analyzes multiple fields at once, useful for moderating entire entities like user profiles or product listings.

const result = await moderationApi.content.submit({
  content: {
    type: "object",
    data: {
      title: { type: "text", text: "Product name" },
      description: { type: "text", text: "Product description" },
      image: { type: "image", url: "https://example.com/product.jpg" },
    },
  },
  metaType: "product",
});

The response includes flagged_fields in each policy result, showing which specific fields triggered the flag.

Audio (enterprise)

Audio files are automatically transcribed to text, then analyzed by all enabled text-based policies. Useful for podcasts, voice messages, or other audio content. See Analyze audio for supported formats, limits, and transcription quality settings.

const result = await moderationApi.content.submit({
  content: {
    type: "audio",
    url: "https://example.com/audio.mp3",
  },
});

Video (enterprise)

Video moderation analyzes video content to detect inappropriate scenes or actions, including violence, nudity, or other objectionable content.

const result = await moderationApi.content.submit({
  content: {
    type: "video",
    url: "https://example.com/video.mp4",
  },
});

---

Opt out of content store

Set doNotStore to true to prevent the content from being stored. The content will still be analyzed but won’t appear in the dashboard or review queues.

const result = await moderationApi.content.submit({
  content: {
    type: "text",
    text: "Hello world!",
  },
  doNotStore: true,
});

Setting doNotStore to true will make parts of the moderation dashboard less useful, as content won’t be available for review or analysis.

Do not disable content storage if you want to train or optimize models based on your data.