Name: Molmo-7B-D API: Image Q&A and Captioning by Ai2
Brand: magicapi
SKU: api-magicapi-image-question-answering-captioning-api-zsxkib-molmo-7b
Availability: InStock
Author: API Market

Shashank Agarwal

# 💬 Image Question Answering and Captioning API (Ai2 Molmo-7B-D)

## Overview
This API runs allenai/Molmo-7B-D-0924, a vision-language model from Ai2 (the Allen Institute for AI), packaged by zsxkib. It answers questions about images and writes captions.

Send an `image` and a `text` prompt, either a question or an instruction such as "Describe this image". Each request returns the answer as one string.

## Features
- **Questions and captions**: ask about objects, people, colours or the setting, or ask for a detailed description.
- **Answer length**: `max_new_tokens` caps the answer at 1 to 1000 tokens (default 200). `length_penalty` below 1.0 encourages shorter answers and above 1.0 longer ones.
- **Sampling**: `temperature` (0.1 to 2, default 1), `top_k` (1 to 100, default 50) and `top_p` (0 to 1, default 1).
- **Text output**: the answer comes back as a single string.

## Use Cases
1. **Alt text**: draft image descriptions for websites and apps.
2. **Catalogue enrichment**: describe product or listing photos for search and filters.
3. **Image tagging**: ask what an image shows to label and sort it.
4. **Content review**: ask direct questions about an image, such as whether it shows a person or an animal.

## Visual Examples
### Example Input
Question: "What do you see? Give me a detailed answer"

![Example input image: a white dog sitting on a wooden bench in a field](https://blog.api.market/wp-content/uploads/2026/10/molmo-7b-input-api-market.png)

```json
{
  "text": "What do you see? Give me a detailed answer",
  "image": "https://blog.api.market/wp-content/uploads/2026/10/molmo-7b-input-api-market.png",
  "top_k": 50,
  "top_p": 1,
  "temperature": 1,
  "length_penalty": 1,
  "max_new_tokens": 200
}
```

### Example Output
Output:

> I see a charming scene featuring a large, fluffy white dog sitting on a wooden bench in the middle of a field. The dog appears to be a poodle mix, with curly fur covering its entire body. It's sitting upright with its front paws hanging over the edge of the bench, looking directly at the camera with a happy expression. The dog's tongue is hanging out, and its eyes are dark and expressive.
>
> The bench is made of dark brown wood with a lattice design on the backrest. It's positioned on a patch of brown, dead grass, which suggests it might be late fall or early winter. In the background, there's a vast field filled with tall, brown grass, and further back, I can see a line of bare trees against a light blue sky.
>
> The overall atmosphere is peaceful and serene. The dog seems to be enjoying its time outdoors, perhaps waiting for its owner or just taking in the scenery. The contrast between the white, fluffy dog and the brown

The answer stops mid-sentence because it reached the `max_new_tokens` limit of 200.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `image` | string (URL) | Yes |  |  | URL of the input image |
| `text` | string | Yes |  |  | Text prompt or question about the image |
| `top_k` | integer | No | `50` | 1 to 100 | Number of highest-probability tokens kept for top-k filtering |
| `top_p` | number | No | `1` | 0 to 1 | Cumulative probability for top-p filtering |
| `max_new_tokens` | integer | No | `200` | 1 to 1000 | Maximum number of new tokens to generate |
| `temperature` | number | No | `1` | 0.1 to 2 | Randomness in token selection (higher values increase randomness) |
| `length_penalty` | number | No | `1` | 0.1 to 2 | Exponential penalty on length (values below 1.0 encourage shorter outputs, above 1.0 longer outputs) |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.056 per API unit.
- **1 API unit = 1 request with up to 51 seconds of processing.** A longer request uses one more unit per started 51-second block (for example, 52 seconds of processing uses 2 units, $0.112).
- Longer answers take longer to generate, so a high `max_new_tokens` can push a request past 51 seconds.
- **Example cost**: the example took 10.91 seconds of processing, which is 1 unit, $0.056.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/image-question-answering-captioning-api-zsxkib-molmo-7b`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/image-question-answering-captioning-api-zsxkib-molmo-7b/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "76ebd700864218a4ca97ac1ccff068be7222272859f9ea2ae1dd4ac073fa8de8",
    "input": {
      "text": "What do you see? Give me a detailed answer",
      "image": "https://blog.api.market/wp-content/uploads/2026/10/molmo-7b-input-api-market.png",
      "top_k": 50,
      "top_p": 1,
      "temperature": 1,
      "length_penalty": 1,
      "max_new_tokens": 200
    }
  }'
```

Response (201 Created):
```json
{
  "id": "wbqshjekmhrmw0d1cczvzgzd0m",
  "version": "76ebd700864218a4ca97ac1ccff068be7222272859f9ea2ae1dd4ac073fa8de8",
  "status": "starting",
  "created_at": "2026-10-22T10:54:49.903Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/image-question-answering-captioning-api-zsxkib-molmo-7b/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "wbqshjekmhrmw0d1cczvzgzd0m",
  "status": "succeeded",
  "output": " I see a charming scene featuring a large, fluffy white dog sitting on a wooden bench in the middle of a field. The dog appears to be a poodle mix, with curly fur covering its entire body. ...",
  "error": null,
  "created_at": "2026-10-22T10:54:49.903Z",
  "started_at": "2026-10-22T10:55:31.494Z",
  "completed_at": "2026-10-22T10:55:43.402Z"
}
```

`output` is a single string with the model's answer (shortened above). It can start with a space, so trim it before display.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `image` or `text`, or a `max_new_tokens` above 1000.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/image-question-answering-captioning-api-zsxkib-molmo-7b";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "76ebd700864218a4ca97ac1ccff068be7222272859f9ea2ae1dd4ac073fa8de8", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "text": "What do you see? Give me a detailed answer",
  "image": "https://blog.api.market/wp-content/uploads/2026/10/molmo-7b-input-api-market.png",
  "top_k": 50,
  "top_p": 1,
  "temperature": 1,
  "length_penalty": 1,
  "max_new_tokens": 200
}).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/image-question-answering-captioning-api-zsxkib-molmo-7b"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "76ebd700864218a4ca97ac1ccff068be7222272859f9ea2ae1dd4ac073fa8de8", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "text": "What do you see? Give me a detailed answer",
    "image": "https://blog.api.market/wp-content/uploads/2026/10/molmo-7b-input-api-market.png",
    "top_k": 50,
    "top_p": 1,
    "temperature": 1,
    "length_penalty": 1,
    "max_new_tokens": 200
})
print(output)
```

## FAQ
- **Why does the answer stop mid-sentence?**
  - It reached `max_new_tokens`. The default is 200; raise it (up to 1000) for longer answers.
- **How do I get a shorter answer?**
  - Ask for one in `text` (for example "Answer in one sentence"), lower `max_new_tokens`, or set `length_penalty` below 1.0.
- **Can I send more than one image?**
  - No. Each request takes one `image`.
- **What does a request cost?**
  - $0.056 for up to 51 seconds of processing. The example, with a 200-token answer, took 10.91 seconds.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Ask questions about images and get detailed descriptions with Ai2's Molmo-7B-D vision-language model. Pay as you go, no monthly fee.