Name: RAM Grounded SAM API: Automatic Image Tagging
Brand: magicapi
SKU: api-magicapi-strong-image-tagging-segment-anything-idea-research-ram-grounded-sam
Availability: InStock
Author: API Market

Shashank Agarwal

# 🏷️ Image Tagging and Segmentation API (IDEA Research RAM Grounded SAM)

## Overview
RAM Grounded SAM combines the Recognize Anything Model (RAM), which tags what an image contains, with Grounded Segment Anything, which locates and segments each tag. Its developer describes RAM as a strong image tagging model.

Send an image URL. The model returns a comma-separated list of tags and a JSON list of detected objects, each with a label, a confidence score and a bounding box. Set `show_visualisation` to `true` to also get images with the boxes and masks drawn on them.

## Features
- **Automatic tags**: `tags` lists the objects, scenes and attributes the model recognizes, with no class list needed.
- **Bounding boxes**: `json_data.mask` gives one entry per detected object with its `label`, `logit` (confidence) and `box` (pixel coordinates of the box corners).
- **Optional visualisations**: with `show_visualisation` set to `true`, the response also includes `masked_img` and `rounding_box_img` image URLs.
- **SAM-HQ option**: set `use_sam_hq` to `true` to segment with SAM-HQ instead of SAM.

## Use Cases
1. **Photo library tagging**: add searchable keywords to every image in a collection.
2. **Product and listing images**: detect which objects appear in a photo and where.
3. **Dataset bootstrapping**: create first-pass labels and boxes for detection datasets before human review.
4. **Content analysis**: count and locate objects in scenes such as rooms, streets or shelves.

## Visual Examples
### Example Input
Image:

![Living room photo sent to the RAM Grounded SAM example request](https://blog.api.market/wp-content/uploads/2026/10/ram-grounded-sam-input-api-market-1.jpg)

Settings: `show_visualisation` `false`.

### Example Output
Tags returned for the example image:

```
armchair, blanket, lamp, carpet, couch, dog, gray, green, home, living room, picture frame, pillow, plant, room, sit, wood floor
```

`json_data.mask` listed 15 entries: the background (value 0) and 14 detected objects, including a dog (confidence 0.63), an armchair (0.57), a wood floor (0.54), a carpet (0.54) and a lamp (0.51). The first three entries:

```json
[
  { "label": "background", "value": 0 },
  { "box": [848.2978515625, 613.2449340820312, 1099.6187744140625, 814.8465576171875], "label": "dog", "logit": 0.63, "value": 1 },
  { "box": [389.3468017578125, 602.1126098632812, 1270.0858154296875, 990.3211059570312], "label": "armchair", "logit": 0.57, "value": 2 }
]
```

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `input_image` | string (URL) | Yes |  |  | Input image |
| `use_sam_hq` | boolean | No | `false` |  | Use SAM-HQ instead of SAM for prediction |
| `show_visualisation` | boolean | No | `false` |  | Output the bounding boxes and masks drawn on the image |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0019 per API unit.
- **1 API unit = 1 successful request with up to 1 second of processing time.** A longer request counts one unit per started second of processing.
- The example request above took 1.18 seconds of processing, so it was billed as 2 units and costs $0.0038.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/strong-image-tagging-segment-anything-idea-research-ram-grounded-sam`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/strong-image-tagging-segment-anything-idea-research-ram-grounded-sam/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "80a2aede4cf8e3c9f26e96c308d45b23c350dd36f1c381de790715007f1ac0ad",
    "input": {
      "input_image": "https://blog.api.market/wp-content/uploads/2026/10/ram-grounded-sam-input-api-market-1.jpg",
      "show_visualisation": false
    }
  }'
```

Response (201 Created):
```json
{
  "id": "7dgmvessx9rne0d1ccr87q5y40",
  "version": "80a2aede4cf8e3c9f26e96c308d45b23c350dd36f1c381de790715007f1ac0ad",
  "status": "starting",
  "created_at": "2026-10-22T10:37:47.534Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/strong-image-tagging-segment-anything-idea-research-ram-grounded-sam/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK), with `json_data.mask` shortened to its first three of 15 entries:
```json
{
  "id": "7dgmvessx9rne0d1ccr87q5y40",
  "status": "succeeded",
  "output": {
    "json_data": {
      "mask": [
        { "label": "background", "value": 0 },
        { "box": [848.2978515625, 613.2449340820312, 1099.6187744140625, 814.8465576171875], "label": "dog", "logit": 0.63, "value": 1 },
        { "box": [389.3468017578125, 602.1126098632812, 1270.0858154296875, 990.3211059570312], "label": "armchair", "logit": 0.57, "value": 2 }
      ],
      "tags": "armchair, blanket, lamp, carpet, couch, dog, gray, green, home, living room, picture frame, pillow, plant, room, sit, wood floor"
    },
    "masked_img": null,
    "rounding_box_img": null,
    "tags": "armchair, blanket, lamp, carpet, couch, dog, gray, green, home, living room, picture frame, pillow, plant, room, sit, wood floor"
  },
  "error": null,
  "created_at": "2026-10-22T10:37:47.534Z",
  "started_at": "2026-10-22T10:37:48.414Z",
  "completed_at": "2026-10-22T10:37:49.592Z"
}
```

`output` is an object. `tags` is a comma-separated string of recognized tags. `json_data.mask` lists the background (value 0) and one entry per detected object with `label`, `logit` (confidence), `box` (`[x1, y1, x2, y2]` in pixels) and `value` (the object's index). `masked_img` and `rounding_box_img` are image URLs when `show_visualisation` is `true` and `null` otherwise.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `input_image`.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/strong-image-tagging-segment-anything-idea-research-ram-grounded-sam";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "80a2aede4cf8e3c9f26e96c308d45b23c350dd36f1c381de790715007f1ac0ad", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  input_image: "https://blog.api.market/wp-content/uploads/2026/10/ram-grounded-sam-input-api-market-1.jpg",
  show_visualisation: false
}).then((output) => {
  console.log("Tags:", output.tags);
  for (const item of output.json_data.mask) {
    if (item.box) console.log(item.label, item.logit, item.box);
  }
});
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/strong-image-tagging-segment-anything-idea-research-ram-grounded-sam"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "80a2aede4cf8e3c9f26e96c308d45b23c350dd36f1c381de790715007f1ac0ad", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "input_image": "https://blog.api.market/wp-content/uploads/2026/10/ram-grounded-sam-input-api-market-1.jpg",
    "show_visualisation": False,
})
print("Tags:", output["tags"])
for item in output["json_data"]["mask"]:
    if "box" in item:
        print(item["label"], item["logit"], item["box"])
```

## FAQ
- **Do I have to tell the model what to look for?**
  - No. The input has no class list. The model tags the image on its own and then locates each tag.
- **What are the numbers in `box`?**
  - The corners of the bounding box in pixels of the input image: `[x1, y1, x2, y2]`. `logit` is the detection confidence.
- **How do I get the mask and box images?**
  - Set `show_visualisation` to `true`. The response then includes `masked_img` and `rounding_box_img` URLs; otherwise both are `null`.
- **How much does one request cost?**
  - $0.0019 per started second of processing. The example took 1.18 seconds and cost $0.0038 (2 units).
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Tag images automatically and locate every tag with bounding boxes using RAM Grounded SAM. No class list needed. $0.0019 per started second of processing.