> ## Documentation Index
> Fetch the complete documentation index at: https://heygen-1fa696a7.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# HeyGen Decision

> Classify text, images, video, and audio, evaluate yes/no questions, and score content against your own criteria with HeyGen Decision 1.

![](https://mintcdn.com/heygen-1fa696a7/hfMXXwJzjE7vBSYZ/images/theme/research-3.webp?fit=max\&auto=format\&n=hfMXXwJzjE7vBSYZ\&q=85\&s=c8191ba4743fd85050f1af1a93ed4969)

**HeyGen Decision 1 turns text, images, video, and audio into decisions your application can use.** Supply text or JSON in `state`, add optional media in `assets`, and ask named questions about them. The API returns an answer for each question, with probabilities you can use to route a request, rank a result, or decide when a person should review it.

Choose the question type that matches the answer you need. You can combine all three in one request.

| Question type | Use it for | Answer |
| - | - | - |
| `noul` | A yes/no question or a statement to evaluate | An estimated probability of true, from 0 to 1 |
| `choice` | Picking one category from a set you define | The selected category, a probability for every category, and confidence |
| `score` | Rating content against an ordered scale | A score from 0 to the last level's index, probabilities, a level legend, and confidence |

Every request starts from text. Add media in `assets` when the decision depends on what something looks like or sounds like:

> [card] [Text](https://heygen-1fa696a7.mintlify.site/docs/models/decision/text.md)
>
> Messages, transcripts, records, and JSON.

> [card] [Image](https://heygen-1fa696a7.mintlify.site/docs/models/decision/image.md)
>
> Photos, screenshots, thumbnails, and documents.

> [card] [Video](https://heygen-1fa696a7.mintlify.site/docs/models/decision/video.md)
>
> Clips, ads, and recordings, with their soundtrack.

> [card] [Audio](https://heygen-1fa696a7.mintlify.site/docs/models/decision/audio.md)
>
> Calls, voicemails, voiceovers, and music.

> [!NOTE]
>
> URLs inside `state` are text. The model reads the URL as text. It fetches media only from `assets`.

## The API

| | |
| - | - |
| Models | `heygen-decision-1-turbo` (Turbo, the default), `heygen-decision-1` (Quality) |
| Endpoint | `POST /v3/models/decisions` |
| Completion | Synchronous HTTP `200`: answers are in the response. No polling or webhook. |
| Input | One `state`, optional image, video, and audio `assets`, and 1–400 named questions |
| Auth | `X-Api-Key` from the [API keys page](https://platform.heygen.com/api-keys), with the `assets:write` scope. Trial API keys and OAuth are not supported. |

If you omit `model`, the API uses `heygen-decision-1-turbo`. See [Models](#models) for each model's context window and media limits.

### Make a decision

This example classifies a support request, checks whether work is blocked, and rates its impact in one call.

```bash theme={null}
curl -X POST https://api.heygen.com/v3/models/decisions \
  -H "X-Api-Key: $HEYGEN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "heygen-decision-1-turbo",
    "state": "Video export fails for everyone on our team after today’s update. We have no workaround and cannot deliver our project.",
    "questions": {
      "team": {
        "type": "choice",
        "instructions": "Which support team should handle this request?",
        "criteria": {
          "video": "Video creation, rendering, or export problems",
          "account": "Sign-in, access, or workspace administration",
          "billing": "Payments, invoices, or subscription charges"
        }
      },
      "blocked": {
        "type": "noul",
        "instructions": "Is the customer unable to complete their work without a workaround?"
      },
      "impact": {
        "type": "score",
        "instructions": "How much does this issue affect the customer’s work?",
        "criteria": [
          "Cosmetic issue; work can continue normally",
          "Work is slower, but a usable workaround exists",
          "Work cannot continue and no workaround is available"
        ]
      }
    }
  }'
```

Illustrative response; IDs, probabilities, and token counts vary:

```json Response theme={null}
{
  "data": {
    "id": "decision-0123456789abcdef",
    "request_id": "0123456789abcdef0123456789abcdef",
    "model": "heygen-decision-1-turbo",
    "answers": {
      "team": {
        "type": "choice",
        "choice": "video",
        "probabilities": { "video": 0.8, "account": 0.15, "billing": 0.05 },
        "confidence": 0.7
      },
      "blocked": { "type": "noul", "noul": 0.96 },
      "impact": {
        "type": "score",
        "score": 1.75,
        "probabilities": { "0": 0.05, "1": 0.15, "2": 0.8 },
        "legend": {
          "0": "Cosmetic issue; work can continue normally",
          "1": "Work is slower, but a usable workaround exists",
          "2": "Work cannot continue and no workaround is available"
        },
        "confidence": 0.625
      }
    },
    "usage": { "input_tokens": 1200, "output_tokens": 0 }
  }
}
```

Read `data.answers.team.choice` to select a support queue. Use the other answers in your own application logic. One question's answer does not become input to another question in the same request. When a question depends on an earlier answer, ask it in a later request.

## Parameters

| Field | Required | Description |
| - | - | - |
| `model` | No | `heygen-decision-1-turbo` (default) or `heygen-decision-1`. A `null` or unknown value is rejected. |
| `state` | Yes | The material to evaluate: a string, JSON object, or JSON array. Objects and arrays are serialized as text. A root number, boolean, or `null` is not accepted. |
| `assets` | No | Media shared by all questions: optional `images`, `videos`, and `audios` maps. Omit it or use `null` for text only. See [Media inputs](#media-inputs). |
| `assets.images` | No | An object that maps names to [image](https://heygen-1fa696a7.mintlify.site/docs/models/decision/image.md) inputs. Each input uses `url`, `asset_id`, or `base64`. |
| `assets.videos` | No | An object that maps names to [video](https://heygen-1fa696a7.mintlify.site/docs/models/decision/video.md) inputs, with the same source formats. |
| `assets.audios` | No | An object that maps names to [audio](https://heygen-1fa696a7.mintlify.site/docs/models/decision/audio.md) inputs, with the same source formats. |
| `questions` | Yes | An object with 1–400 entries. Each key names a question and becomes the matching key in `data.answers`. Names must be 1–256 characters, contain no colon or newline, and have no leading or trailing whitespace. A request may have at most 4,096 options in total; see [Request limits](#request-limits). |
| `questions.{name}.type` | Yes | `noul`, `choice`, or `score`. The type sets the shape of `criteria` and the answer. |
| `questions.{name}.instructions` | No | The question to answer or the rule to apply. Accepts a string, JSON object, array, or `null`. If omitted or blank, the question name is used, with underscores replaced by spaces. Explicit instructions are clearer. |
| `questions.{name}.criteria` | Depends on type | Optional true/false descriptions for `noul`; required named options for `choice`; required ordered levels for `score`. See below. |

Unknown request and question fields are rejected. There is no top-level `instructions`, `prompt`, `temperature`, or `max_tokens` parameter.

For how to shape `state`, see [Text](https://heygen-1fa696a7.mintlify.site/docs/models/decision/text.md).

## Media inputs

Add media under `assets.images`, `assets.videos`, or `assets.audios`. Each is an object keyed by names you choose, 1–1,024 characters long. All questions share all supplied media, and any question type works with any input. You can mix images, video, and audio in one request.

Names organize your request. The model does not see them, so use `state` to say what each input is.

Each input page has a worked example: [Image](https://heygen-1fa696a7.mintlify.site/docs/models/decision/image.md), [Video](https://heygen-1fa696a7.mintlify.site/docs/models/decision/video.md), and [Audio](https://heygen-1fa696a7.mintlify.site/docs/models/decision/audio.md).

### Choose an asset source

Each named input accepts one of these source objects. You can mix source types in one request.

```json Uploaded asset theme={null}
{ "type": "asset_id", "asset_id": "ASSET_ID" }
```

```json HTTPS URL theme={null}
{ "type": "url", "url": "https://example.com/product.jpg" }
```

```json Inline base64 theme={null}
{ "type": "base64", "media_type": "image/jpeg", "data": "BASE64_ENCODED_IMAGE" }
```

* **Uploaded asset:** use an `asset_id` from the [asset upload API](https://heygen-1fa696a7.mintlify.site/docs/upload-assets.md) in the calling workspace. The asset must still exist, match the media group, and be readable by the caller. A generated output's `image_id` or `video_id` is not an `asset_id`.
* **URL:** use a public HTTPS URL that serves the media directly, without authentication or redirects, with a matching `image/*`, `video/*`, or `audio/*` `Content-Type`. Local file paths and web page URLs are not media inputs.
* **Base64:** provide only the base64-encoded bytes in `data`, without a `data:` prefix. Set `media_type` to match the file and group, for example `image/png` under `images`, `video/mp4` under `videos`, or `audio/mpeg` under `audios`.

### Media input requirements

| Requirement | Images | Video | Audio |
| - | - | - | - |
| Formats | JPEG, PNG, WebP | MP4, WebM, MOV | WAV, MP3, FLAC, OGG, M4A, AAC, WebM |
| Count per request | 100 | 10 | 10 |
| Length of each item | — | 60 s | 30 minutes (1,800 s) |
| Each URL download | 32 MiB | 50 MiB | 32 MiB |
| Each inline base64 file, decoded | 5 MiB | 16 MiB | 16 MiB |

The model resizes images and video to fit the context window. See [Context window](#context-window).

For larger files, upload with the [asset upload API](https://heygen-1fa696a7.mintlify.site/docs/upload-assets.md) or the [direct upload flow](https://heygen-1fa696a7.mintlify.site/docs/direct-upload.md) and pass the `asset_id`. An item passed as an `asset_id` is read from storage, so the URL download cap does not apply.

The **entire JSON request body must fit within 32 MiB**, including state, questions, and encoded media. Base64 adds about one third to the file size, so use an uploaded asset or a URL for larger files. A larger body returns HTTP `400` with `invalid_parameter`: `Request body is N bytes, which exceeds the 33554432-byte limit for this endpoint.` Media also uses the model's context and adds to `usage.input_tokens`.

The API measures each video and audio clip before the model reads it:

* A video longer than 60 s returns HTTP `400` with `invalid_parameter`, for example `A video can be at most 60 s; assets.videos.clip is 94.2 s.`
* An audio clip longer than 30 minutes returns HTTP `400` with `invalid_parameter`, for example `An audio clip can be at most 1,800 s; assets.audios.call is 1,931.4 s.`
* A video or clip whose length cannot be read returns `400`, for example `Could not read the duration of assets.audios.call; provide a valid audio file.`

These requests are not charged and do not reach the model.

## Question types

### Noul: evaluate a yes/no question

Use `noul` to test a condition. The answer's `noul` value is the model's estimated probability of true: near 1 favors yes, near 0 favors no, and near 0.5 is uncertain. It does not include a separate `confidence` field.

You can optionally define what each outcome means:

```json Noul question theme={null}
{
  "type": "noul",
  "instructions": "Does the request meet the refund policy?",
  "criteria": {
    "true": "All requirements for a refund are met by the supplied evidence.",
    "false": "At least one requirement is not met or lacks supporting evidence."
  }
}
```

`criteria` accepts only the keys `true` and `false`. Either can be omitted or `null` to keep its default meaning. A description can be text, a JSON object, or an array. Choose your decision threshold with examples from your application; 0.5 is not a guarantee that an answer is reliable.

### Choice: pick a category

`criteria` is an object with 1–255 options. Its keys are nonempty option names, and its values describe when to select each option. A description can be text, a JSON object, an array, or `null`. With `null` or an empty string, only the option name is shown.

The answer includes:

| Field | Meaning |
| - | - |
| `choice` | The option with the highest probability |
| `probabilities` | A value from 0 to 1 for every option, keyed by your option names; the values sum to 1 |
| `confidence` | How far the highest probability is above a uniform guess, rescaled to 0–1 |

Make categories distinguishable. If none of your categories may fit, add an explicit option such as `other` and describe when it applies. The model chooses among the options you supply; it does not invent another category.

A choice with one option returns that option with probability `1` and confidence `1`, without running the model for that question. This is a fixed answer, not a prediction.

### Score: rate against ordered levels

`criteria` is an array of 1–10 level descriptions, ordered from low to high. Each level can be a nonempty string, a JSON object, or an array. Describe an observable condition at each level rather than labels such as “bad,” “okay,” and “good” without context.

Levels are numbered from zero. Three levels give a score from `0` to `2`; five levels give a score from `0` to `4`. The answer includes:

| Field | Meaning |
| - | - |
| `score` | The probability-weighted average of the level indices, which may be fractional |
| `probabilities` | A probability for each level, keyed by its index as a string: `"0"`, `"1"`, and so on |
| `legend` | The mapping from each string index back to its level description; structured descriptions are returned as serialized text |
| `confidence` | Concentration around the most likely level, accounting for the distance between levels |

In the response above, `0 × 0.05 + 1 × 0.15 + 2 × 0.8 = 1.75`. This places the result near the highest-impact level. It does not mean that 87.5% of users are affected. Check the distribution as well as the average: different distributions can give the same score.

A score with one level returns `score: 0`, `probabilities: {"0": 1}`, and `confidence: 1`, without running the model for that question. Its legend contains that one level.

## Read probabilities and confidence

Confidence comes from the returned distribution. It is **not a calibrated probability that the answer is correct**, and you cannot compare it between question types.

For a choice with at least two options, let `N` be the option count and `p` the highest probability. Confidence is `(p - 1/N) / (1 - 1/N)`, limited to 0–1. Three equal probabilities give confidence `0`; a highest probability of `0.8` gives confidence `0.7`.

Score confidence also considers how far probability lies from the most likely level. Probability on a neighboring level counts less against confidence than probability far away. Use both the score and its distribution when you decide whether to accept or review a result.

Test thresholds on labeled examples from your own use case. A high value can still be wrong, especially when the state lacks necessary evidence or the criteria overlap.

## Response fields

All success fields are inside `data`.

| Field | Description |
| - | - |
| `id` | Identifier for this decision result |
| `request_id` | Request identifier to keep when you troubleshoot or contact support |
| `model` | The public model name you sent, or `heygen-decision-1-turbo` if you omitted it |
| `answers` | One answer for every question, under the same name. Each answer's `type` matches the question. |
| `usage.input_tokens` | Total input tokens of the model reads for this request. You pay for these tokens. |
| `usage.output_tokens` | Always `0` |

## Models

Each model has one context window. The window is the most tokens one model read can take: the formatted state, media, question, criteria, and one answer token. It is not a limit on the raw `state` alone.

### `heygen-decision-1-turbo` (default)

* **Context window:** 262,144 tokens
* **Input modalities:** text, image, video, audio
* **Output:** answers with probabilities; no generated text
* **Media per request:** 100 images, 10 videos (each at most 60 s), 10 audio clips (each at most 30 minutes)

### `heygen-decision-1`

* **Context window:** 262,144 tokens
* **Input modalities:** text, image, video, audio
* **Output:** answers with probabilities; no generated text
* **Media per request:** 100 images, 10 videos (each at most 60 s), 10 audio clips (each at most 30 minutes)

A request with more media than the model takes returns HTTP `400` with `invalid_parameter` before it is charged, for example `heygen-decision-1 takes at most 100 images, 10 videos and 10 audio clips; this request has 11 videos`.

### Context window

The model resizes images and video to fit the context window. If a request still does not fit, the API returns HTTP `400` with `context_length_exceeded` and this message, where `{model}` is the model you named:

```text theme={null}
This request is too long for the context window of {model}. Shorten the text or audio, or send fewer or shorter images and videos, then try again.
```

The request is not charged.

### Tokens and usage

`usage.input_tokens` is the total input tokens of the model reads for a request. Each question reads the shared input again, so `usage.input_tokens` grows with the number of questions and can be larger than the context window. Read `usage.input_tokens` from real requests to estimate your usage.

Questions with a single choice option or a single score level are answered without a model read. A request that has only those fixed questions reports zero input and output tokens, and the API does not fetch or read its media. API access checks still apply.

## Request limits

| Limit | Both models |
| - | - |
| Questions per request | 1–400 |
| Question name | 1–256 characters |
| Choice options | 1–255 |
| Score levels | 1–10 |
| Options in total per request | 4,096 |
| JSON request body | 32 MiB |
| Time for one request | 60 s |

Options in total counts each choice question's options, each score question's levels, and 2 for each `noul` question (yes and no). A request with more returns HTTP `400` with `invalid_parameter` before it is charged, and the message gives the count, for example `A request may have at most 4,096 options in total; this one has 4,097.`

## Errors and retries

Errors use the standard [HeyGen error format](https://heygen-1fa696a7.mintlify.site/docs/error-codes.md), separate from the success `data` object. Requests that fail are not charged.

| Code | HTTP | When |
| - | - | - |
| `invalid_parameter` | `400` | A field or limit is wrong, or the model cannot read an input. |
| `invalid_parameter` | `400` | The request body is over 32 MiB. |
| `context_length_exceeded` | `400` | The request does not fit the context window: `This request is too long for the context window of {model}. Shorten the text or audio, or send fewer or shorter images and videos, then try again.` |
| `unauthorized` | `401` | The API key is missing or wrong. |
| `insufficient_credit` | `402` | The balance is too low. |
| `forbidden`, `insufficient_api_key_scope`, `api_key_required`, `plan_upgrade_required`, `resource_access_denied` | `403` | The key, scope, or plan does not allow it. Trial API keys and OAuth are not supported. |
| `rate_limit_exceeded` | `429` | Too many requests (100 per second). |
| `internal_error` | `500` | Any other failure. Retry with backoff. |
| `service_unavailable` | `503` | The model is busy. Retry with backoff. |
| `gateway_timeout` | `504` | No answer within 60 s. Retry. |

The default request limit is 100 requests per second, shared by API keys owned by the same user in the same workspace. Keep concurrency bounded and increase it gradually. There is no idempotency key for this endpoint: a retried request runs again and is charged again.

## Build it with an agent

Paste this into your coding agent to build a small integration:

```text Prompt for your agent wrap theme={null}
Help me classify content with HeyGen Decision 1. Read https://developers.heygen.com/docs/models/heygen-decision.md and https://developers.heygen.com/docs/for-ai-agents.md first.

Use HEYGEN_API_KEY from my environment without printing it. If it is missing, point me to https://platform.heygen.com/api-keys; do not ask me to paste it into chat. Ask what text, images, video, or audio I want to evaluate and what decisions I need. Put text in state and media in assets.images, assets.videos, or assets.audios, following the documented source formats and limits. For local files, use the asset upload API and its returned asset_id; never invent an ID or URL. Write explicit instructions and clear criteria for each question, then make one POST /v3/models/decisions request and set model explicitly. Show me the answers, probabilities, and usage.input_tokens. Do not assume the endpoint supports the video APIs' polling, callbacks, or idempotency keys.
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.