Name: Realistic Lipsync Animation API
Brand: magicapi
SKU: api-magicapi-realistic-lipsync-api-sync-lipsync-2
Availability: InStock
Author: API Market

Shashank Agarwal

# 👄 Realistic Lipsync Animation API (Sync Labs 2.0)

## Overview
The Realistic Lipsync Animation API by Sync Labs takes an existing video with a visible face and a new audio track, and returns the video with the lip movements re-synced to that audio. Both a video and an audio file are required; it does not animate a still image.

## Features
- 🚀 **Ultra-Realistic Lipsync**: Generate highly accurate lipsync animations that closely mirror actual human lip movements.
- 🔄 **Synchronization Modes**: `sync_mode` sets how different audio and video lengths are handled: `loop` (default), `bounce`, `cut_off`, `silence` or `remap`.
- 🎭 **Expressiveness Control**: Adjustable temperature setting to control the expressiveness of the lipsync.
- 🗣️ **Active Speaker Detection**: Optional `active_speaker` flag (off by default) detects whoever is speaking in the clip and applies the lipsync to them.
- ⏱️ **Processing Time**: Typically a few minutes, depending on video length.

## Use Cases
1. **Filmmaking and Animation**
   Enhance dialogue scenes in films and animations by syncing pre-recorded audio with character mouth movements.

2. **E-Learning Platforms**
   Produce engaging educational videos by ensuring the instructor's lip movements align perfectly with the spoken content.

3. **Marketing and Advertising**
   Create compelling promotional videos where brand ambassadors' speech is accurately lip-synced for a more authentic viewer experience.

4. **Gaming**
   Integrate seamless lipsync animations for in-game characters to enhance storytelling and player immersion.

5. **Virtual Assistants and Chatbots**
   Improve the realism of virtual assistants and chatbots by syncing their audio responses with lifelike lip movements.

## Visual Examples

### Example Input

```json
{
  "input": {
    "video": "https://blog.api.market/wp-content/uploads/2026/09/lipsync-2-input-video-api-market.mp4",
    "audio": "https://blog.api.market/wp-content/uploads/2026/09/grok-text-to-speech-output-api-market.mp3",
    "sync_mode": "loop"
  }
}
```

![Input video, generated with Grok Imagine Video on api.market](https://blog.api.market/wp-content/uploads/2026/09/lipsync-2-input-video-api-market.mp4)

![Input audio, generated with Grok TTS on api.market](https://blog.api.market/wp-content/uploads/2026/09/grok-text-to-speech-output-api-market.mp3)

### Example Output

![Example output from Realistic Lipsync Animation API (Sync Labs 2.0)](https://blog.api.market/wp-content/uploads/2026/09/lipsync-2-output-api-market.mp4)

This exact request was run through api.market and billed 11 API units ($0.693).
## Pricing
- Pay As You Go (PAYG) pricing - you only pay for what you use.
- Price: $0.063 per API unit.
- 1 API unit = 1 second of output video, rounded up to whole seconds. There is no duration input: the output length comes from your input video, input audio and `sync_mode`. With the default `loop` mode, a 13.7-second video and a 19.2-second audio file returned a 19.25-second video, billed as 20 units.
- Failed or cancelled predictions are not billed.
- GET requests to check prediction status (GET /api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions/{id}) are free and don’t count toward API unit usage.
- No monthly subscription fee.

## API Documentation

### Base URL
All API requests should be made to: `https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2`

### Authentication
Include your API key in the request header:
```plaintext
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API uses asynchronous processing for predictions. The typical flow is:

1. Submit a prediction request to the POST endpoint.
2. Receive a prediction ID in the response.
3. Poll the GET endpoint with the prediction ID to check status.
4. Continue polling until the prediction is complete.

The processing time varies depending on the complexity of the request. Implement:
- A polling interval of 1 second between status checks.
- A maximum timeout of 600 seconds (10 minutes).
- Status checking logic that continues while status is "starting" or "processing".
- Result handling when status becomes "succeeded".
- Error handling when status becomes "failed" or timeout is reached.

### Endpoints

#### Create Prediction
Create a new lipsync prediction.

`POST /api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions`

**Input parameters** (sent inside `input`):

| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
| `video` | string (URI) | Yes | none | Input video file (.mp4) with the face to lip-sync. |
| `audio` | string (URI) | Yes | none | Input audio file. The schema lists .wav; the .mp3 file in the example also works. |
| `sync_mode` | string | No | `loop` | How to handle audio and video of different lengths: `loop`, `bounce`, `cut_off`, `silence` or `remap`. |
| `temperature` | number | No | `0.5` | How expressive the lipsync can be, from 0 to 1. |
| `active_speaker` | boolean | No | `false` | Detects the active speaker, so whoever is speaking in the clip is used for the lipsync. |
```bash
# Example for creating a prediction
curl -X POST "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "4f8dc3cfda4ff844a6158ac347d21fcd025210f6dad4b16265fc53074ee4f77f",
    "input": {
      "audio": "https://blog.api.market/wp-content/uploads/2026/09/grok-text-to-speech-output-api-market.mp3",
      "video": "https://blog.api.market/wp-content/uploads/2026/09/lipsync-2-input-video-api-market.mp4",
      "sync_mode": "loop",
      "temperature": 0.5,
      "active_speaker": true
    }
  }'
```

#### Check Prediction Status
Check the status of a lipsync prediction.
```bash
# Example for checking prediction status
curl -X GET "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

### Status Codes
- **200 OK**: Request successful.
- **201 Created**: Resource successfully created.
- **400 Bad Request**: Invalid input parameters or malformed request.
- **401 Unauthorized**: Missing or invalid API key.
- **403 Forbidden**: Valid API key but insufficient permissions.
- **404 Not Found**: Resource not found.
- **429 Too Many Requests**: Rate limit exceeded.
- **500 Internal Server Error**: Server-side error.
- **503 Service Unavailable**: Service temporarily unavailable.

### Rate Limits
No fixed rate limit is documented. If you receive a 429 response, retry with exponential backoff.

### Error Handling
A 400 response means an input is missing or invalid (`video` and `audio` are both required). Retry 429 and 5xx responses with exponential backoff. A prediction that ends with status `failed` carries an `error` field and is not billed.

#### Example Error Response
```json
{
  "error": {
    "code": "invalid_parameter",
    "message": "The parameter 'audio' is required",
    "status": 400
  }
}
```

## Implementation Guide

### Example Code

#### JavaScript
```javascript
// JavaScript example with proper polling and timeout
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2";

async function createAndWaitForPrediction(inputData) {
  try {
    // Step 1: Create the prediction
    const response = await fetch(`${baseUrl}/predictions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-api-market-key": apiKey
      },
      body: JSON.stringify({ version: "4f8dc3cfda4ff844a6158ac347d21fcd025210f6dad4b16265fc53074ee4f77f", input: inputData })
    });
    
    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${response.statusText}`);
    }
    
    const prediction = await response.json();
    const predictionId = prediction.id;
    
    // Step 2: Poll for results with timeout
    return await pollForResults(predictionId);
  } catch (error) {
    console.error("Error:", error);
    throw error;
  }
}

function delay(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function pollForResults(predictionId) {
  const MAX_TIMEOUT = 600000; // 600 seconds (10 minutes)
  const POLL_INTERVAL = 1000; // 1 second between checks
  
  const startTime = Date.now();
  
  while (Date.now() - startTime < MAX_TIMEOUT) {
    // Check prediction status
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: {
        "x-api-market-key": apiKey
      }
    });
    
    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${response.statusText}`);
    }
    
    const prediction = await response.json();
    
    // Check status and return if completed
    if (prediction.status === "succeeded") {
      return prediction.output;
    }
    
    // Handle failure
    if (prediction.status === "failed") {
      throw new Error(`Prediction failed: ${prediction.error || "Unknown error"}`);
    }
    
    // Wait before next poll
    await delay(POLL_INTERVAL);
  }
  
  throw new Error("Prediction timed out after 10 minutes");
}
```

#### Python
```python
# Python example with proper polling and timeout
import requests
import time

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2"

def create_and_wait_for_prediction(input_data):
    # Step 1: Create the prediction
    headers = {
        "Content-Type": "application/json",
        "x-api-market-key": API_KEY
    }
    
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "4f8dc3cfda4ff844a6158ac347d21fcd025210f6dad4b16265fc53074ee4f77f", "input": input_data}
    )
    
    if response.status_code not in (200, 201):
        raise Exception(f"API error: {response.status_code} {response.text}")
    
    prediction = response.json()
    prediction_id = prediction["id"]
    
    # Step 2: Poll for results with timeout
    return poll_for_results(prediction_id)

def poll_for_results(prediction_id):
    MAX_TIMEOUT = 600  # 600 seconds (10 minutes)
    POLL_INTERVAL = 1  # 1 second between checks
    
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()
    
    while time.time() - start_time < MAX_TIMEOUT:
        # Check prediction status
        response = requests.get(
            f"{BASE_URL}/predictions/{prediction_id}",
            headers=headers
        )
        
        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")
        
        prediction = response.json()
        
        # Check status and return if completed
        if prediction["status"] == "succeeded":
            return prediction["output"]
        
        # Handle failure
        if prediction["status"] == "failed":
            error_msg = prediction.get("error", "Unknown error")
            raise Exception(f"Prediction failed: {error_msg}")
        
        # Wait before next poll
        time.sleep(POLL_INTERVAL)
    
    raise Exception("Prediction timed out after 10 minutes")
```

#### cURL
```bash
# Step 1: Create a prediction
curl -X POST "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "4f8dc3cfda4ff844a6158ac347d21fcd025210f6dad4b16265fc53074ee4f77f",
    "input": {
      "audio": "https://blog.api.market/wp-content/uploads/2026/09/grok-text-to-speech-output-api-market.mp3",
      "video": "https://blog.api.market/wp-content/uploads/2026/09/lipsync-2-input-video-api-market.mp4",
      "sync_mode": "loop",
      "temperature": 0.5,
      "active_speaker": true
    }
  }'

# Response will contain a prediction ID like:
# {"id": "abcd1234", "status": "starting"}

# Step 2: Check prediction status (replace abcd1234 with the actual ID)
curl -X GET "https://prod.api.market/api/v1/magicapi/realistic-lipsync-api-sync-lipsync-2/predictions/abcd1234" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"

# Note: You need to repeatedly poll this endpoint manually until status is "succeeded"
```

### Performance Optimization
Tips for optimizing API usage:
- Use appropriate sync_mode settings to match the natural flow of your videos.
- Adjust the temperature setting to balance expressiveness and realism.
- Set `active_speaker` to `true` for clips with more than one person so the lipsync is applied to whoever is speaking.

## FAQ
**Q:** How do I get started with the API?
- **A:** Sign up on API.market, obtain your API key, and follow the documentation to make your first prediction request.

**Q:** What file formats are supported?
- **A:** The model's schema lists .mp4 for the input video and .wav for the input audio. The .mp3 audio file used in the examples also works.

**Q:** How do I handle errors?
- **A:** Refer to the Error Handling section for details on interpreting error responses and retry strategies.

**Q:** How long do predictions take?
- **A:** Prediction times vary depending on request complexity, typically completed within a few minutes.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.

Lip-sync an existing video to a new audio track with Sync Labs' lipsync-2 model. Five sync modes for mismatched lengths and optional active speaker detection.