Name: cjwbw AniPortrait Audio-Driven Portrait Animation API
Brand: magicapi
SKU: api-magicapi-audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid
Availability: InStock
Author: API Market

Shashank Agarwal

# 🗣️ Audio-Driven Portrait Animation API (cjwbw AniPortrait)

## Overview
AniPortrait (audio-to-video) animates a single portrait photo so that it talks along with an audio clip. Its developer describes it as audio-driven synthesis of photorealistic portrait animations. This version is packaged by cjwbw.

Send a portrait `image` and an `audio` clip. The model returns two video URLs: `video`, the animated portrait, and `pose`, a video of the facial landmark (pose) sequence the model derived from the audio.

## Features
- **One photo plus audio**: no video or training per person; the face in `image` is animated to match `audio`.
- **Two outputs**: the final animated `video` and the intermediate `pose` video.
- **Output size**: `width` and `height` (default 512 x 512).
- **Frame rate**: `fps` (default 30).
- **Quality controls**: `steps` (default 25) inference steps and `guidance_scale` (default 3.5).
- **Seed**: set `seed` to reproduce a result; leave it out for a random seed.

## Use Cases
1. **Talking avatars**: animate a presenter or mascot portrait to read a voice-over.
2. **Narrated explainers**: turn a still portrait and a recorded script into a short talking-head clip.
3. **Prototyping dubbing and voice content**: preview how a voice track looks on a face.
4. **Animation research**: use the `pose` output to inspect the landmark motion derived from speech.

## Visual Examples
### Example Input
![AniPortrait example input: an AI-generated face portrait](https://blog.api.market/wp-content/uploads/2026/10/ip-adapter-faceid-input-api-market.png)

Audio: a short spoken English clip (WAV), sent as `audio`.

Settings: `width` `512`, `height` `512`, `fps` `30`, `steps` `25`, `guidance_scale` `3.5`.

### Example Output
`video` (animated portrait): [Watch the example output](https://blog.api.market/wp-content/uploads/2026/10/aniportrait-audio2vid-output-2-6wfh8ggv-api-market.mp4)

`pose` (landmark sequence): [Watch the pose video](https://blog.api.market/wp-content/uploads/2026/10/aniportrait-audio2vid-output-1-6wfh8ggv-api-market.mp4)

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `image` | string (URL) | Yes |  |  | Input portrait image (jpeg, png, gif or webp) |
| `audio` | string (URL) | Yes |  |  | Input audio that drives the animation |
| `width` | integer | No | `512` |  | Width of output video |
| `height` | integer | No | `512` |  | Height of output video |
| `guidance_scale` | number | No | `3.5` |  | Scale for classifier-free guidance |
| `steps` | integer | No | `25` |  | Inference steps |
| `fps` | integer | No | `30` |  | Frames per second in the output video |
| `seed` | integer | No | random |  | Random seed. Leave blank to randomize the seed |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.325 per API unit.
- **1 API unit = 1 successful request with up to 300 seconds of processing.** A longer job bills one more unit for each started block of 300 seconds (for example, 420 seconds of processing is 2 units, $0.65).
- Processing time grows with the length of the audio, `width`, `height`, `fps` and `steps`.
- **Example cost**: the example request (512 x 512, 30 fps, 25 steps) took 255.5 seconds of processing, which is 1 unit, so $0.325.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

Video jobs take minutes. A polling interval of 5 seconds and an overall timeout of 1800 seconds (30 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "3f976d8f2308f5c676a484e873f7d1ac09763f789fa211894df1ed96d3d17cb2",
    "input": {
      "fps": 30,
      "audio": "https://blog.api.market/wp-content/uploads/2026/10/gpt-4o-mini-transcribe-input-v2-api-market.wav",
      "image": "https://blog.api.market/wp-content/uploads/2026/10/ip-adapter-faceid-input-api-market.png",
      "steps": 25,
      "width": 512,
      "height": 512,
      "guidance_scale": 3.5
    }
  }'
```

Response (201 Created):
```json
{
  "id": "6wfh8ggvw5rmy0d1f1a9pndhxc",
  "version": "3f976d8f2308f5c676a484e873f7d1ac09763f789fa211894df1ed96d3d17cb2",
  "status": "starting",
  "created_at": "2026-10-26T13:09:00.705Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "6wfh8ggvw5rmy0d1f1a9pndhxc",
  "status": "succeeded",
  "output": {
    "pose": "https://blog.api.market/wp-content/uploads/2026/10/aniportrait-audio2vid-output-1-6wfh8ggv-api-market.mp4",
    "video": "https://blog.api.market/wp-content/uploads/2026/10/aniportrait-audio2vid-output-2-6wfh8ggv-api-market.mp4"
  },
  "error": null,
  "created_at": "2026-10-26T13:09:00.705Z",
  "started_at": "2026-10-26T13:10:30.760Z",
  "completed_at": "2026-10-26T13:14:46.276Z"
}
```

`output` is an object with two MP4 URLs: `video`, the animated portrait, and `pose`, the landmark sequence video.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `image` or `audio`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
Requests are rate limited to keep usage fair. If you receive a 429 response, wait and retry with exponential backoff.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "3f976d8f2308f5c676a484e873f7d1ac09763f789fa211894df1ed96d3d17cb2", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 1800000;
  const pollInterval = 5000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 30 minutes");
}

createAndWaitForPrediction({
  "fps": 30,
  "audio": "https://blog.api.market/wp-content/uploads/2026/10/gpt-4o-mini-transcribe-input-v2-api-market.wav",
  "image": "https://blog.api.market/wp-content/uploads/2026/10/ip-adapter-faceid-input-api-market.png",
  "steps": 25,
  "width": 512,
  "height": 512,
  "guidance_scale": 3.5
}).then((output) => console.log(output.video, output.pose));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/audio-driven-portrait-animation-api-cjwbw-aniportrait-audio2vid"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "3f976d8f2308f5c676a484e873f7d1ac09763f789fa211894df1ed96d3d17cb2", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 1800
    poll_interval = 5
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 30 minutes")


output = create_and_wait_for_prediction({
    "fps": 30,
    "audio": "https://blog.api.market/wp-content/uploads/2026/10/gpt-4o-mini-transcribe-input-v2-api-market.wav",
    "image": "https://blog.api.market/wp-content/uploads/2026/10/ip-adapter-faceid-input-api-market.png",
    "steps": 25,
    "width": 512,
    "height": 512,
    "guidance_scale": 3.5
})
print(output["video"])
print(output["pose"])
```

## FAQ
- **Which output is the talking portrait?**
  - `video`. The `pose` output is the landmark (pose) sequence the model built from the audio, useful for debugging or further animation work.
- **What kind of portrait works best?**
  - A clear, front-facing photo with one face. The example uses a head-and-shoulders portrait at 512 x 512.
- **How long does a request take and what does it cost?**
  - Processing grows with the audio length, output size, `fps` and `steps`. Up to 300 seconds of processing is one unit ($0.325); each further started 300 seconds adds a unit.
- **Can I animate photos of real people?**
  - Only use photos and voices of people who have agreed to it, and follow the laws on likeness and privacy where you operate.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.


Turn a portrait photo and a voice clip into a talking-head video with AniPortrait, plus its facial pose video. $0.325 per request, pay as you go.