Name: AnimateDiff API: Text to Video with Camera Motion
Brand: magicapi
SKU: api-magicapi-animate-diffusion-zsxkib-animate-diff
Availability: InStock
Author: API Market

Shashank Agarwal

# 🎬 Text-to-Video Generator API (zsxkib AnimateDiff)

## Overview
AnimateDiff animates text-to-image diffusion models: it turns a text `prompt` into a short video clip. This version includes motion LoRAs for camera moves such as panning, zooming and rolling.

Pick one of five base image models (`base_model`), set the clip length in `frames` (1 to 32, played back at 8 fps, so up to 4 seconds) and optionally add camera motion. The API returns a list with one video URL, as `mp4` (default) or `gif`.

## Features
- **Short clips**: `frames` from 1 to 32 (default 16) at 8 fps playback; 16 frames is 2 seconds.
- **Five base models**: `realisticVisionV20_v20` (default), `lyriel_v16`, `majicmixRealistic_v5Preview`, `rcnzCartoon3d_v10`, `toonyou_beta3`.
- **Camera motion LoRAs**: zoom in, zoom out, pan left, pan right, pan up, pan down, rolling clockwise and rolling anticlockwise, each with a strength from 0 (off) to 1.
- **Custom size**: `width` and `height` in pixels (default 512 x 512).
- **Negative prompt**: list what to avoid; a long default negative prompt is applied when you leave it out.
- **Output formats**: `mp4` (default) or `gif`.

## Use Cases
1. **Animated backgrounds**: short looping scenes for websites, slides and streams.
2. **Social media clips**: 2 to 4 second animations from a text description.
3. **Camera move previews**: test pans and zooms over a generated scene.
4. **Stylized animation**: cartoon-style clips with the `rcnzCartoon3d_v10` or `toonyou_beta3` base models.

## Visual Examples
### Example Input
Prompt: "a medium shot of a vibrant coral reef with a variety of marine life, rainbow, visual effects, prores, cineon, royal, monumental, hyperrealistic, exceptional, visually stunning"

Settings: `base_model` `realisticVisionV20_v20`, `frames` 16, `width` 768, `height` 512, `steps` 25, `guidance_scale` 7.5, `seed` -1, `negative_prompt` empty, `pan_right_motion_strength` 0.75, all other motion strengths 0.

### Example Output
[Watch the example output](https://blog.api.market/wp-content/uploads/2026/10/animate-diff-output-drggrtfc-api-market.mp4): a 2-second, 768 x 512 mp4 clip of the coral reef with a pan to the right.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | `photo of vocano, rocks, storm weather, wind, lava waves, lightning, 8k uhd, dslr, soft lighting, high quality, film grain, Fujifilm XT3` |  | Text prompt describing the video |
| `negative_prompt` | string | No | `blur, haze, deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers, deformed, distorted, disfigured, poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation` |  | What to avoid in the video |
| `base_model` | string | No | `realisticVisionV20_v20` | `realisticVisionV20_v20`, `lyriel_v16`, `majicmixRealistic_v5Preview`, `rcnzCartoon3d_v10`, `toonyou_beta3` | Base model (DreamBooth checkpoint) |
| `steps` | integer | No | `25` | 1 to 100 | Number of inference steps |
| `guidance_scale` | number | No | `7.5` | 0 to 20 | How closely the video adheres to the prompt and its contents |
| `frames` | integer | No | `16` | 1 to 32 | Length of the video in frames. Playback is at 8 fps, so 16 frames is 2 seconds |
| `width` | integer | No | `512` |  | Width in pixels |
| `height` | integer | No | `512` |  | Height in pixels |
| `seed` | integer | No | `-1` |  | Seed for different results and reproducibility. Use -1 for a random seed |
| `zoom_in_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Zoom In motion LoRA. 0 disables the LoRA |
| `zoom_out_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Zoom Out motion LoRA. 0 disables the LoRA |
| `pan_left_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Pan Left motion LoRA. 0 disables the LoRA |
| `pan_right_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Pan Right motion LoRA. 0 disables the LoRA |
| `pan_up_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Pan Up motion LoRA. 0 disables the LoRA |
| `pan_down_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Pan Down motion LoRA. 0 disables the LoRA |
| `rolling_clockwise_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Rolling Clockwise motion LoRA. 0 disables the LoRA |
| `rolling_anticlockwise_motion_strength` | number | No | `0` | 0 to 1 | Strength of the Rolling Anticlockwise motion LoRA. 0 disables the LoRA |
| `output_format` | string | No | `mp4` | `mp4`, `gif` | Output format of the video |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.099 per API unit.
- **1 API unit = 1 successful request with up to 91 seconds of processing.** A request that runs longer is billed one more unit for each started 91-second block (for example, 120 seconds of processing is 2 units, $0.198). More `frames`, more `steps` and larger `width` and `height` take longer.
- The example request above (16 frames at 768 x 512, 25 steps) took 62.4 seconds of processing, which is 1 unit, so it cost $0.099.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/animate-diffusion-zsxkib-animate-diff`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/animate-diffusion-zsxkib-animate-diff/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "269a616c8b0c2bbc12fc15fd51bb202b11e94ff0f7786c026aa905305c4ed9fb",
    "input": {
      "seed": -1,
      "steps": 25,
      "width": 768,
      "frames": 16,
      "height": 512,
      "prompt": "a medium shot of a vibrant coral reef with a variety of marine life, rainbow, visual effects, prores, cineon, royal, monumental, hyperrealistic, exceptional, visually stunning",
      "base_model": "realisticVisionV20_v20",
      "guidance_scale": 7.5,
      "negative_prompt": "",
      "pan_up_motion_strength": 0,
      "zoom_in_motion_strength": 0,
      "pan_down_motion_strength": 0,
      "pan_left_motion_strength": 0,
      "zoom_out_motion_strength": 0,
      "pan_right_motion_strength": 0.75,
      "rolling_clockwise_motion_strength": 0,
      "rolling_anticlockwise_motion_strength": 0
    }
  }'
```

Response (201 Created):
```json
{
  "id": "drggrtfce5rmw0d1d8xvyjnpj0",
  "version": "269a616c8b0c2bbc12fc15fd51bb202b11e94ff0f7786c026aa905305c4ed9fb",
  "status": "starting",
  "created_at": "2026-10-23T19:27:54.737Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/animate-diffusion-zsxkib-animate-diff/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "drggrtfce5rmw0d1d8xvyjnpj0",
  "status": "succeeded",
  "output": [
    "https://blog.api.market/wp-content/uploads/2026/10/animate-diff-output-drggrtfc-api-market.mp4"
  ],
  "error": null,
  "created_at": "2026-10-23T19:27:54.737Z",
  "started_at": "2026-10-23T19:27:55.737Z",
  "completed_at": "2026-10-23T19:28:40.737Z"
}
```

`output` is an array of URLs; for the example it holds one mp4 video.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example `frames` above 32 or a `base_model` that is not in the list.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/animate-diffusion-zsxkib-animate-diff";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "269a616c8b0c2bbc12fc15fd51bb202b11e94ff0f7786c026aa905305c4ed9fb", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "seed": -1,
  "steps": 25,
  "width": 768,
  "frames": 16,
  "height": 512,
  "prompt": "a medium shot of a vibrant coral reef with a variety of marine life, rainbow, visual effects, prores, cineon, royal, monumental, hyperrealistic, exceptional, visually stunning",
  "base_model": "realisticVisionV20_v20",
  "guidance_scale": 7.5,
  "negative_prompt": "",
  "pan_up_motion_strength": 0,
  "zoom_in_motion_strength": 0,
  "pan_down_motion_strength": 0,
  "pan_left_motion_strength": 0,
  "zoom_out_motion_strength": 0,
  "pan_right_motion_strength": 0.75,
  "rolling_clockwise_motion_strength": 0,
  "rolling_anticlockwise_motion_strength": 0
}).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/animate-diffusion-zsxkib-animate-diff"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "269a616c8b0c2bbc12fc15fd51bb202b11e94ff0f7786c026aa905305c4ed9fb", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "seed": -1,
    "steps": 25,
    "width": 768,
    "frames": 16,
    "height": 512,
    "prompt": "a medium shot of a vibrant coral reef with a variety of marine life, rainbow, visual effects, prores, cineon, royal, monumental, hyperrealistic, exceptional, visually stunning",
    "base_model": "realisticVisionV20_v20",
    "guidance_scale": 7.5,
    "negative_prompt": "",
    "pan_up_motion_strength": 0,
    "zoom_in_motion_strength": 0,
    "pan_down_motion_strength": 0,
    "pan_left_motion_strength": 0,
    "zoom_out_motion_strength": 0,
    "pan_right_motion_strength": 0.75,
    "rolling_clockwise_motion_strength": 0,
    "rolling_anticlockwise_motion_strength": 0
}))
```

## FAQ
- **How long can a clip be?**
  - Up to 32 frames. Playback is at 8 fps, so 32 frames is 4 seconds and the default 16 frames is 2 seconds.
- **How do I add camera motion?**
  - Set one of the `*_motion_strength` fields between 0 and 1, for example `pan_right_motion_strength` 0.75 as in the example. 0 turns a motion off.
- **Can I start from my own image?**
  - No. This model generates video from a text prompt only; there is no image input.
- **How is a request billed?**
  - By processing time: $0.099 covers up to 91 seconds, and each further started 91-second block adds one unit.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Turn a text prompt into a clip of up to 32 frames with AnimateDiff, five base models and camera motion LoRAs, as mp4 or gif. Pay as you go, no monthly fee.