Skip to main content
HeyGen Decision 1 turns text, images, video, and audio into decisions your application can use. Supply text or JSON in state, add optional media in assets, and ask named questions about them. The API returns an answer for each question, with probabilities you can use to route a request, rank a result, or decide when a person should review it. Choose the question type that matches the answer you need. You can combine all three in one request. Every request starts from text. Add media in assets when the decision depends on what something looks like or sounds like:

Text

Messages, transcripts, records, and JSON.

Image

Photos, screenshots, thumbnails, and documents.

Video

Clips, ads, and recordings, with their soundtrack.

Audio

Calls, voicemails, voiceovers, and music.
URLs inside state are text. The model reads the URL as text. It fetches media only from assets.

The API

If you omit model, the API uses heygen-decision-1-turbo. See Models for each model’s context window and media limits.

Make a decision

This example classifies a support request, checks whether work is blocked, and rates its impact in one call.
Illustrative response; IDs, probabilities, and token counts vary:
Response
Read data.answers.team.choice to select a support queue. Use the other answers in your own application logic. One question’s answer does not become input to another question in the same request. When a question depends on an earlier answer, ask it in a later request.

Parameters

Unknown request and question fields are rejected. There is no top-level instructions, prompt, temperature, or max_tokens parameter. For how to shape state, see Text.

Media inputs

Add media under assets.images, assets.videos, or assets.audios. Each is an object keyed by names you choose, 1–1,024 characters long. All questions share all supplied media, and any question type works with any input. You can mix images, video, and audio in one request. Names organize your request. The model does not see them, so use state to say what each input is. Each input page has a worked example: Image, Video, and Audio.

Choose an asset source

Each named input accepts one of these source objects. You can mix source types in one request.
  • Uploaded asset: use an asset_id from the asset upload API in the calling workspace. The asset must still exist, match the media group, and be readable by the caller. A generated output’s image_id or video_id is not an asset_id.
  • URL: use a public HTTPS URL that serves the media directly, without authentication or redirects, with a matching image/*, video/*, or audio/* Content-Type. Local file paths and web page URLs are not media inputs.
  • Base64: provide only the base64-encoded bytes in data, without a data: prefix. Set media_type to match the file and group, for example image/png under images, video/mp4 under videos, or audio/mpeg under audios.

Media input requirements

The model resizes images and video to fit the context window. See Context window. For larger files, upload with the asset upload API or the direct upload flow and pass the asset_id. An item passed as an asset_id is read from storage, so the URL download cap does not apply. The entire JSON request body must fit within 32 MiB, including state, questions, and encoded media. Base64 adds about one third to the file size, so use an uploaded asset or a URL for larger files. A larger body returns HTTP 400 with invalid_parameter: Request body is N bytes, which exceeds the 33554432-byte limit for this endpoint. Media also uses the model’s context and adds to usage.input_tokens. The API measures each video and audio clip before the model reads it:
  • A video longer than 60 s returns HTTP 400 with invalid_parameter, for example A video can be at most 60 s; assets.videos.clip is 94.2 s.
  • An audio clip longer than 30 minutes returns HTTP 400 with invalid_parameter, for example An audio clip can be at most 1,800 s; assets.audios.call is 1,931.4 s.
  • A video or clip whose length cannot be read returns 400, for example Could not read the duration of assets.audios.call; provide a valid audio file.
These requests are not charged and do not reach the model.

Question types

Noul: evaluate a yes/no question

Use noul to test a condition. The answer’s noul value is the model’s estimated probability of true: near 1 favors yes, near 0 favors no, and near 0.5 is uncertain. It does not include a separate confidence field. You can optionally define what each outcome means:
Noul question
criteria accepts only the keys true and false. Either can be omitted or null to keep its default meaning. A description can be text, a JSON object, or an array. Choose your decision threshold with examples from your application; 0.5 is not a guarantee that an answer is reliable.

Choice: pick a category

criteria is an object with 1–255 options. Its keys are nonempty option names, and its values describe when to select each option. A description can be text, a JSON object, an array, or null. With null or an empty string, only the option name is shown. The answer includes: Make categories distinguishable. If none of your categories may fit, add an explicit option such as other and describe when it applies. The model chooses among the options you supply; it does not invent another category. A choice with one option returns that option with probability 1 and confidence 1, without running the model for that question. This is a fixed answer, not a prediction.

Score: rate against ordered levels

criteria is an array of 1–10 level descriptions, ordered from low to high. Each level can be a nonempty string, a JSON object, or an array. Describe an observable condition at each level rather than labels such as “bad,” “okay,” and “good” without context. Levels are numbered from zero. Three levels give a score from 0 to 2; five levels give a score from 0 to 4. The answer includes: In the response above, 0 × 0.05 + 1 × 0.15 + 2 × 0.8 = 1.75. This places the result near the highest-impact level. It does not mean that 87.5% of users are affected. Check the distribution as well as the average: different distributions can give the same score. A score with one level returns score: 0, probabilities: {"0": 1}, and confidence: 1, without running the model for that question. Its legend contains that one level.

Read probabilities and confidence

Confidence comes from the returned distribution. It is not a calibrated probability that the answer is correct, and you cannot compare it between question types. For a choice with at least two options, let N be the option count and p the highest probability. Confidence is (p - 1/N) / (1 - 1/N), limited to 0–1. Three equal probabilities give confidence 0; a highest probability of 0.8 gives confidence 0.7. Score confidence also considers how far probability lies from the most likely level. Probability on a neighboring level counts less against confidence than probability far away. Use both the score and its distribution when you decide whether to accept or review a result. Test thresholds on labeled examples from your own use case. A high value can still be wrong, especially when the state lacks necessary evidence or the criteria overlap.

Response fields

All success fields are inside data.

Models

Each model has one context window. The window is the most tokens one model read can take: the formatted state, media, question, criteria, and one answer token. It is not a limit on the raw state alone.

heygen-decision-1-turbo (default)

  • Context window: 262,144 tokens
  • Input modalities: text, image, video, audio
  • Output: answers with probabilities; no generated text
  • Media per request: 100 images, 10 videos (each at most 60 s), 10 audio clips (each at most 30 minutes)

heygen-decision-1

  • Context window: 262,144 tokens
  • Input modalities: text, image, video, audio
  • Output: answers with probabilities; no generated text
  • Media per request: 100 images, 10 videos (each at most 60 s), 10 audio clips (each at most 30 minutes)
A request with more media than the model takes returns HTTP 400 with invalid_parameter before it is charged, for example heygen-decision-1 takes at most 100 images, 10 videos and 10 audio clips; this request has 11 videos.

Context window

The model resizes images and video to fit the context window. If a request still does not fit, the API returns HTTP 400 with context_length_exceeded and this message, where {model} is the model you named:
The request is not charged.

Tokens and usage

usage.input_tokens is the total input tokens of the model reads for a request. Each question reads the shared input again, so usage.input_tokens grows with the number of questions and can be larger than the context window. Read usage.input_tokens from real requests to estimate your usage. Questions with a single choice option or a single score level are answered without a model read. A request that has only those fixed questions reports zero input and output tokens, and the API does not fetch or read its media. API access checks still apply.

Request limits

Options in total counts each choice question’s options, each score question’s levels, and 2 for each noul question (yes and no). A request with more returns HTTP 400 with invalid_parameter before it is charged, and the message gives the count, for example A request may have at most 4,096 options in total; this one has 4,097.

Errors and retries

Errors use the standard HeyGen error format, separate from the success data object. Requests that fail are not charged. The default request limit is 100 requests per second, shared by API keys owned by the same user in the same workspace. Keep concurrency bounded and increase it gradually. There is no idempotency key for this endpoint: a retried request runs again and is charged again.

Build it with an agent

Paste this into your coding agent to build a small integration:
Prompt for your agent