Name: MiniCPM-V 4.5 API: Image and Video Understanding
Brand: magicapi
SKU: api-magicapi-multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9
Availability: InStock
Author: API Market

Shashank Agarwal

# 📷 Image and Video Understanding API (OpenBMB MiniCPM-V 4.5)

## Overview
This API runs MiniCPM-V 4.5, a multimodal language model from OpenBMB, packaged by sai88uk. Its developer describes it as a GPT-4o level multimodal model for single-image, multi-image and high-FPS video understanding.

Send a `question` together with an `image` or a `video`, and the model answers in text. `thinking_mode` (`fast`, `deep` or `ultra`) controls how much the model reasons before answering.

## Features
- **Video questions**: send a `video` and ask about what happens or what is shown in it. `video_fps` (0.5 to 10, default 5) sets how many frames per second are sampled; it is adjusted automatically for long videos.
- **Image questions**: send an `image` to describe it, read text in it or answer questions about it.
- **Three thinking modes**: `fast` (default), `deep` and `ultra`. With `ultra`, the example answer starts with the model's reasoning inside `<think>...</think>`.
- **Plain text answer**: the output is a single string.

## Use Cases
1. **Reading text in videos**: list signs, captions and on-screen text, as in the example.
2. **Video summaries and Q&A**: describe what happens in a clip or answer a specific question about it.
3. **Image description**: caption photos or explain screenshots and charts.
4. **Content review**: check what a video or image shows before publishing it.

## Visual Examples
### Example Input
Video: [Watch the example input video](https://blog.api.market/wp-content/uploads/2026/10/minicpm-v-45-v9-input-api-market.mp4) (a street scene with billboards and shop signs)

Question: "Look through this video and read all the signs and text exist in this video and write them down for me"

Settings: `thinking_mode` `ultra`, `video_fps` `10`.

### Example Output
The output is one string. With `ultra` it begins with the model's reasoning, then the answer. Shortened:
```text
<think>
I need to analyze the video and extract all visible text from the signs and billboards. First, I need to identify each sign and transcribe the text accurately.
...
</think>

Here are all the readable signs and texts in the video:

### Billboards and Large Signs:
- CINEMA (on a red billboard, appears later)
- STOP (on a traffic sign pole)
...
```
The full answer has 18 entries under billboards and large signs, 15 under text on buildings and vehicles (many repeats), then a short notes section.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `question` | string | Yes |  |  | Question about the image or video |
| `image` | string (URL) | No | none |  | Input image (for image chat) |
| `video` | string (URL) | No | none |  | Input video (for video analysis) |
| `thinking_mode` | string | No | `fast` | `fast`, `deep`, `ultra` | Thinking mode |
| `video_fps` | number | No | `5` | 0.5 to 10 | Video FPS (adjusted automatically for long videos) |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.011 per API unit.
- **1 API unit = 1 request with up to 7 seconds of processing.** A request that takes longer is billed one more unit for each started 7-second block (for example, 30 seconds of processing is 5 units, $0.055).
- Videos, a higher `video_fps`, `deep` or `ultra` thinking and long answers take longer and add units. An image question in `fast` mode is the quickest.
- The example request (a video at 10 fps with `ultra` thinking) took 52.8 seconds of processing, so it was billed 8 units ($0.088).
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "5f9e86550c3540aab9292e0cae22f71bb75724be3c9bb72ebf0798d028f0f27b",
    "input": {
      "video": "https://blog.api.market/wp-content/uploads/2026/10/minicpm-v-45-v9-input-api-market.mp4",
      "question": "Look through this video and read all the signs and text exist in this video and write them down for me",
      "video_fps": 10,
      "thinking_mode": "ultra"
    }
  }'
```

Response (201 Created):
```json
{
  "id": "jahathygbxrna0d1d8t9adg120",
  "version": "5f9e86550c3540aab9292e0cae22f71bb75724be3c9bb72ebf0798d028f0f27b",
  "status": "starting",
  "created_at": "2026-10-23T19:20:08.736Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK, `output` shortened):
```json
{
  "id": "jahathygbxrna0d1d8t9adg120",
  "status": "succeeded",
  "output": "<think>\nI need to analyze the video and extract all visible text from the signs and billboards. ...\n</think>\n\nHere are all the readable signs and texts in the video:\n\n### Billboards and Large Signs:\n- CINEMA (on a red billboard, appears later)\n- STOP (on a traffic sign pole)\n...",
  "error": null,
  "created_at": "2026-10-23T19:20:08.736Z",
  "started_at": "2026-10-23T19:20:09.736Z",
  "completed_at": "2026-10-23T19:20:50.736Z"
}
```

`output` is a single string with the answer. In `ultra` mode (and possibly `deep`) it can start with the model's reasoning inside `<think>...</think>`; strip that block if you only want the final answer.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `question` or a `video_fps` above 10.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why (for example, a video URL that cannot be downloaded). Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "5f9e86550c3540aab9292e0cae22f71bb75724be3c9bb72ebf0798d028f0f27b", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  video: "https://blog.api.market/wp-content/uploads/2026/10/minicpm-v-45-v9-input-api-market.mp4",
  question: "Look through this video and read all the signs and text exist in this video and write them down for me",
  video_fps: 10,
  thinking_mode: "ultra"
}).then((output) => {
  const answer = output.replace(/<think>[\s\S]*?<\/think>/, "").trim();
  console.log(answer);
});
```

#### Python
```python
import re
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/multi-image-video-understanding-api-sai88uk-minicpm-v-45-v9"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "5f9e86550c3540aab9292e0cae22f71bb75724be3c9bb72ebf0798d028f0f27b", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "video": "https://blog.api.market/wp-content/uploads/2026/10/minicpm-v-45-v9-input-api-market.mp4",
    "question": "Look through this video and read all the signs and text exist in this video and write them down for me",
    "video_fps": 10,
    "thinking_mode": "ultra",
})
print(re.sub(r"<think>.*?</think>", "", output, flags=re.S).strip())
```

## FAQ
- **Should I send an image or a video?**
  - Either. Use `image` for a single picture and `video` for a clip. `question` is always required.
- **What is the difference between the thinking modes?**
  - `fast` (default) answers directly. `deep` and `ultra` let the model reason more before answering, which takes longer and can use more units. In `ultra` mode the output can include the reasoning inside `<think>...</think>`.
- **How do I control how much of the video the model sees?**
  - Set `video_fps` between 0.5 and 10 (default 5). For long videos the frame rate is adjusted automatically.
- **Is everything in the answer correct?**
  - Not always. In the example the model read most signs but also repeated some and misspelled others, so check results before relying on them.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Ask OpenBMB's MiniCPM-V 4.5 about an image or a video: read on-screen text, describe scenes and answer questions, with three thinking modes.