Name: Qwen2.5-Omni 7B API: Multimodal Chat with Speech Replies
Brand: magicapi
SKU: api-magicapi-multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b
Availability: InStock
Author: API Market

Shashank Agarwal

# 🤖 Multimodal Chat API with Text and Speech Replies (Alibaba Qwen2.5-Omni 7B)

## Overview
Qwen2.5-Omni 7B is an end-to-end multimodal model from Alibaba's Qwen team, packaged by lucataco. Its developer describes it as designed to perceive text, images, audio and video while generating both text and natural speech responses.

Send a text `prompt`, an `image`, an `audio` clip, a `video`, or a mix of them. The API returns a text reply and, when `generate_audio` is `true` (default), a spoken version of the reply as a WAV file in one of two voices.

## Features
- **Four input types**: `prompt` (text), `image`, `audio` and `video`, all optional, in any combination.
- **Spoken replies**: `generate_audio` (default `true`) returns the reply as speech in the `voice` field.
- **Two voices**: `voice_type` `Chelsie` (default) or `Ethan`.
- **Audio inside videos**: `use_audio_in_video` (default `true`) lets the model hear the soundtrack of a video as well as see it.
- **System prompt**: set the model's role with `system_prompt`. The default describes Qwen as a virtual human that can perceive audio and visual inputs and generate text and speech.

## Use Cases
1. **Voice assistants**: send a spoken question as `audio` and play back the spoken reply.
2. **Video conversations**: send a video of someone talking and get a reply to what they said and showed.
3. **Image questions with spoken answers**: ask about a photo and get both text and speech.
4. **Accessible interfaces**: give users spoken answers alongside text.

## Visual Examples
### Example Input
Video: [Watch the example input video](https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-input-api-market.mp4)

Settings: no `prompt` (the model answers the video, including its audio), `use_audio_in_video` `true`, `generate_audio` `true`, `voice_type` `Chelsie`, `system_prompt` set to the default.

### Example Output
Text reply:
> Oh, that's a really cool drawing! It looks like a guitar. You've got the body and the neck drawn in a simple yet effective way. The lines are clean and the shape is well-defined. What made you choose to draw a guitar?

Spoken reply: [Listen to the example output](https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-output-9528p42p-api-market.wav)

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | none |  | Text prompt for the model |
| `image` | string (URL) | No | none |  | Optional image input |
| `audio` | string (URL) | No | none |  | Optional audio input |
| `video` | string (URL) | No | none |  | Optional video input |
| `system_prompt` | string | No | `You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.` |  | System prompt for the model |
| `use_audio_in_video` | boolean | No | `true` |  | Whether to use the audio track of the video |
| `voice_type` | string | No | `Chelsie` | `Chelsie`, `Ethan` | Voice for the audio output |
| `generate_audio` | boolean | No | `true` |  | Whether to generate audio output |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0035 per API unit.
- **1 API unit = 1 request with up to 3 seconds of processing.** A request that takes longer is billed one more unit for each started 3-second block (for example, 20 seconds of processing is 7 units, $0.0245).
- Video and audio inputs, longer replies and `generate_audio` add processing time and units. Setting `generate_audio` to `false` shortens processing when you only need text.
- The example request (video in, text and speech out) took 36.6 seconds of processing, so it was billed 13 units ($0.0455).
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "0ca8160f7aaf85703a6aac282d6c79aa64d3541b239fa4c5c1688b10cb1faef1",
    "input": {
      "video": "https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-input-api-market.mp4",
      "voice_type": "Chelsie",
      "system_prompt": "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.",
      "generate_audio": true,
      "use_audio_in_video": true
    }
  }'
```

Response (201 Created):
```json
{
  "id": "9528p42p9xrmw0d1d8msa0281m",
  "version": "0ca8160f7aaf85703a6aac282d6c79aa64d3541b239fa4c5c1688b10cb1faef1",
  "status": "starting",
  "created_at": "2026-10-23T19:07:36.568Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "9528p42p9xrmw0d1d8msa0281m",
  "status": "succeeded",
  "output": {
    "text": "Oh, that's a really cool drawing! It looks like a guitar. You've got the body and the neck drawn in a simple yet effective way. The lines are clean and the shape is well-defined. What made you choose to draw a guitar?",
    "voice": "https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-output-9528p42p-api-market.wav"
  },
  "error": null,
  "created_at": "2026-10-23T19:07:36.568Z",
  "started_at": "2026-10-23T19:07:37.568Z",
  "completed_at": "2026-10-23T19:07:58.568Z"
}
```

`output` is an object. `text` is the reply as text (always present). `voice` is the URL of the spoken reply as a WAV file, returned when `generate_audio` is `true`.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `voice_type` other than `Chelsie` or `Ethan`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why (for example, a media URL that cannot be downloaded). Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "0ca8160f7aaf85703a6aac282d6c79aa64d3541b239fa4c5c1688b10cb1faef1", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  video: "https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-input-api-market.mp4",
  voice_type: "Chelsie",
  system_prompt: "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.",
  generate_audio: true,
  use_audio_in_video: true
}).then((output) => {
  console.log(output.text);
  console.log("Voice:", output.voice);
});
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/multimodal-ai-perception-and-response-lucataco-qwen2-5-omni-7b"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "0ca8160f7aaf85703a6aac282d6c79aa64d3541b239fa4c5c1688b10cb1faef1", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "video": "https://blog.api.market/wp-content/uploads/2026/10/qwen2-5-omni-7b-input-api-market.mp4",
    "voice_type": "Chelsie",
    "system_prompt": "You are Qwen, a virtual human developed by the Qwen Team, Alibaba Group, capable of perceiving auditory and visual inputs, as well as generating text and speech.",
    "generate_audio": True,
    "use_audio_in_video": True,
})
print(output["text"])
print("Voice:", output.get("voice"))
```

## FAQ
- **Do I need to send a prompt?**
  - No. All inputs are optional. In the example only a video was sent, and the model replied to what was said and shown in it. Add a `prompt` to ask a specific question.
- **How do I get only text?**
  - Set `generate_audio` to `false`. The response then has `text` only, and processing is shorter.
- **Which voices are available?**
  - `Chelsie` (default) and `Ethan`, set with `voice_type`.
- **Can it hear the sound in a video?**
  - Yes, when `use_audio_in_video` is `true` (default). Set it to `false` to use only the frames.
- **Should I change the system prompt?**
  - You can. The default describes the model as a virtual human that generates text and speech; keep it when you want spoken replies.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Chat with Alibaba's Qwen2.5-Omni 7B using text, images, audio and video, and get replies as text plus a spoken WAV in two voices. Pay as you go, no monthly fee.