Name: CogVLM API: Image Description and Visual Q&A
Brand: magicapi
SKU: api-magicapi-visual-language-model-cjwbw-cogvlm
Availability: InStock
Author: API Market

Shashank Agarwal

# 💬 Image Description and Visual Question Answering API (THUDM CogVLM)

## Overview
CogVLM is an open-source visual language model; its license is published by THUDM, and its developer describes it as a powerful open-source visual language model. It reads one image and answers a text query about it.

Send an image URL as `image` and your question or instruction as `query`. The default query is "Describe this image.", so a request with only an image returns a description. Set `vqa` to `true` to switch the model to its VQA (visual question answering) mode. The answer comes back as plain text.

## Features
- **Image descriptions**: with the default `query` ("Describe this image.") the model writes a description of the image.
- **Free-form questions**: put any question or instruction about the image in `query`.
- **VQA mode**: `vqa` (default `false`) enables the model's visual question answering mode.
- **Plain-text output**: `output` is a single string, ready to store or display.

## Use Cases
1. **Alt text and captions**: generate descriptions of images for accessibility and image galleries.
2. **Catalog and listing text**: describe product or property photos as a starting point for written listings.
3. **Questions about an image**: ask what is shown, which colors or objects appear, or what text is visible.
4. **Search indexing**: turn image content into text that a search engine or database can index.

## Visual Examples
### Example Input
Input image:

![Example input image for the CogVLM API: a basketball game](https://blog.api.market/wp-content/uploads/2026/10/cogvlm-input-api-market-1.png)

Query: "Describe this image." with `vqa` set to `false`:

```json
{
  "vqa": false,
  "image": "https://blog.api.market/wp-content/uploads/2026/10/cogvlm-input-api-market-1.png",
  "query": "Describe this image."
}
```

### Example Output
The model returned this text:

> This image captures a moment from a basketball game. Two players are prominently featured: one wearing a yellow jersey with the number 24 and the word 'Lakers' printed on it, and the other wearing a navy blue jersey with the word 'Washington' and the number 34. The player in yellow is holding a basketball and appears to be dribbling it, while the player in navy blue is reaching out with his arm, possibly trying to block or defend. The background shows a filled stadium with spectators, indicating that this is a professional game.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `query` | string | No | `Describe this image.` |  | Question or instruction about the image |
| `image` | string (URL) | Yes | none |  | Image to describe or ask about (URL to a jpeg, png, gif or webp image) |
| `vqa` | boolean | No | `false` |  | Enable the model's VQA (visual question answering) mode |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.063 per API unit.
- **1 API unit = 1 successful request that finishes within 58 seconds of processing.** A request that takes longer is billed 1 unit for each started 58 seconds of processing time (for example, 59 seconds of processing = 2 units = $0.126).
- The example request below took 9.3 seconds of processing, so it cost $0.063 (1 unit).
- Long answers take longer to generate, so a request that needs more than 58 seconds of processing uses more than 1 unit.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/visual-language-model-cjwbw-cogvlm`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well. The example request took about 9 seconds of processing.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/visual-language-model-cjwbw-cogvlm/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "a5092d718ea77a073e6d8f6969d5c0fb87d0ac7e4cdb7175427331e1798a34ed",
    "input": {
      "vqa": false,
      "image": "https://blog.api.market/wp-content/uploads/2026/10/cogvlm-input-api-market-1.png",
      "query": "Describe this image."
    }
  }'
```

Response (201 Created):
```json
{
  "id": "9jg3pfnp4drmr0d1ccm85t3s0c",
  "version": "a5092d718ea77a073e6d8f6969d5c0fb87d0ac7e4cdb7175427331e1798a34ed",
  "status": "starting",
  "created_at": "2026-10-22T10:29:35.414Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/visual-language-model-cjwbw-cogvlm/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "9jg3pfnp4drmr0d1ccm85t3s0c",
  "status": "succeeded",
  "output": "This image captures a moment from a basketball game. Two players are prominently featured: one wearing a yellow jersey with the number 24 and the word 'Lakers' printed on it, and the other wearing a navy blue jersey with the word 'Washington' and the number 34. The player in yellow is holding a basketball and appears to be dribbling it, while the player in navy blue is reaching out with his arm, possibly trying to block or defend. The background shows a filled stadium with spectators, indicating that this is a professional game.",
  "error": null,
  "created_at": "2026-10-22T10:29:35.414Z",
  "started_at": "2026-10-22T10:33:44.722Z",
  "completed_at": "2026-10-22T10:33:53.975Z"
}
```

`output` is a single string containing the model's answer.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `image`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/visual-language-model-cjwbw-cogvlm";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "a5092d718ea77a073e6d8f6969d5c0fb87d0ac7e4cdb7175427331e1798a34ed", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "vqa": false,
    "image": "https://blog.api.market/wp-content/uploads/2026/10/cogvlm-input-api-market-1.png",
    "query": "Describe this image."
  }).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/visual-language-model-cjwbw-cogvlm"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "a5092d718ea77a073e6d8f6969d5c0fb87d0ac7e4cdb7175427331e1798a34ed", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "vqa": False,
    "image": "https://blog.api.market/wp-content/uploads/2026/10/cogvlm-input-api-market-1.png",
    "query": "Describe this image."
}))
```

## FAQ
- **Do I have to write a query?**
  - No. `query` defaults to "Describe this image.", so sending only `image` returns a description.
- **What does `vqa` change?**
  - It switches the model to its visual question answering mode. It is `false` by default, and the example request leaves it off.
- **Can I send more than one image?**
  - No. Each request takes one `image`. Send one request per image.
- **What format is the answer in?**
  - Plain text. `output` is a single string.
- **How much does a request cost?**
  - $0.063 per API unit, and one unit covers up to 58 seconds of processing. The example took about 9.3 seconds and cost $0.063.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Describe images and answer questions about them in plain text with CogVLM, an open-source visual language model. $0.063 per API unit, pay as you go.