Name: Meta SAM 3 API: Text-Prompted Image Segmentation
Brand: magicapi
SKU: api-magicapi-promptable-image-segmentation-vufinder-sam3
Availability: InStock
Author: API Market

Shashank Agarwal

# ✂️ Promptable Image Segmentation API (Meta SAM 3)

## Overview
SAM 3 (Segment Anything Model 3) is Meta's segmentation model. Its developer describes it as a unified foundation model for promptable segmentation in images. This listing is packaged by vufinder.

Send an `image` and a list of `prompts`. Each prompt is a JSON string that names a concept in `text` (for example "pillow"), and/or marks the object with positive or negative boxes or points. The API returns one JSON results file per prompt and, when `visualize` is `true`, one PNG per prompt showing the result on the image.

## Features
- **Text prompts**: describe a concept such as `{"text": "pillow"}` to segment it in the image.
- **Box and point prompts**: `positive_boxes`, `negative_boxes`, `positive_points` and `negative_points` with coordinates normalized to 0 to 1. Boxes are `center_x, center_y, width, height`.
- **Several prompts per request**: `prompts` is a list; each prompt gets its own results file (`split_output`, default `true`).
- **Visualizations**: `visualize` returns one PNG per prompt with the result drawn on the image.
- **Confidence threshold**: `confidence_threshold` from 0.01 to 0.99 (default 0.5).
- **Cut-out masks**: `offset_masks` returns masks as cut-outs with the (x, y) offset of their upper-left corner in the original image.
- **Extras**: `concat_input` copies the prompts into the output JSON under `input`, and `backbone_output` returns the backbone output as a PT file.

## Use Cases
1. **Image editing**: select objects by name or by a click, then mask, cut out or replace them.
2. **Dataset labelling**: create segmentation masks for training data from short text concepts.
3. **Product and catalog images**: isolate items such as furniture, clothing or packaging.
4. **Scene analysis**: find and outline all instances of a concept in a photo.

## Visual Examples
### Example Input
One photo with three prompts: the text concept "pillow", one positive box and one positive point. `visualize` was on and `confidence_threshold` was 0.5.

![SAM 3 example input photo](https://blog.api.market/wp-content/uploads/2026/10/sam3-input-api-market.jpg)

```json
{
  "image": "https://blog.api.market/wp-content/uploads/2026/10/sam3-input-api-market.jpg",
  "prompts": [
    "{\"text\": \"pillow\"}",
    "{\"positive_boxes\": [[0.19, 0.73, 0.3, 0.3]]}",
    "{\"positive_points\": [[0.5, 0.87]]}"
  ],
  "visualize": true,
  "backbone_output": false,
  "confidence_threshold": 0.5
}
```

### Example Output
One visualization per prompt, in the order of `prompts`:

![SAM 3 result for the text prompt pillow](https://blog.api.market/wp-content/uploads/2026/10/sam3-output-1-3p61mg2e-api-market.jpg)

![SAM 3 result for the positive box prompt](https://blog.api.market/wp-content/uploads/2026/10/sam3-output-2-3p61mg2e-api-market.png)

![SAM 3 result for the positive point prompt](https://blog.api.market/wp-content/uploads/2026/10/sam3-output-3-3p61mg2e-api-market.png)

The response also returned three JSON files in `results`, one per prompt, with the predicted masks. `backbone_outputs` was `null` because `backbone_output` was off.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `image` | string (URL) | Yes | none |  | Input image URL (JPG, JPEG, PNG or WEBP) |
| `prompts` | array of strings | Yes | none |  | List of prompts, one JSON string per concept. Each prompt can contain `text` (string), `positive_boxes` and `negative_boxes` (lists of four floats), `positive_points` and `negative_points` (lists of two floats). At least one of `text`, `positive_boxes` or `positive_points` is required. Coordinates are normalized to 0 to 1; boxes are `center_x, center_y, width, height`. |
| `confidence_threshold` | number | No | `0.5` | `0.01` to `0.99` | Confidence threshold for the model |
| `visualize` | boolean | No | `false` |  | Whether to return the results as one PNG per prompt |
| `backbone_output` | boolean | No | `false` |  | Whether to return the backbone output as a PT file |
| `offset_masks` | boolean | No | `false` |  | Whether to return the masks as cut-outs, with the (x, y) offset of their upper-left corner in the original image |
| `concat_input` | boolean | No | `false` |  | Whether to copy the input prompts into the output JSON files under the `input` key |
| `split_output` | boolean | No | `true` |  | Whether to split the output JSON into separate files per prompt |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0044 per API unit.
- **1 API unit = 1 successful request with up to 4 seconds of processing.** A request that takes longer is billed one more unit for each started block of 4 seconds (for example, 9 seconds of processing is 3 units, $0.0132).
- More prompts and larger images take longer to process and can use more units.
- **Example cost**: the example request above took 5.5 seconds of processing, so it used 2 API units and cost $0.0088.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/promptable-image-segmentation-vufinder-sam3`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/promptable-image-segmentation-vufinder-sam3/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "1bf97763d5dfd3a1584adca913a8ef4b43c684fca97e04e39e4c50a3a5e09650",
    "input": {
      "image": "https://blog.api.market/wp-content/uploads/2026/10/sam3-input-api-market.jpg",
      "prompts": [
        "{\"text\": \"pillow\"}",
        "{\"positive_boxes\": [[0.19, 0.73, 0.3, 0.3]]}",
        "{\"positive_points\": [[0.5, 0.87]]}"
      ],
      "visualize": true,
      "backbone_output": false,
      "confidence_threshold": 0.5
    }
  }'
```

Response (201 Created):
```json
{
  "id": "3p61mg2ej5rmt0d1d1c8eakjdw",
  "version": "1bf97763d5dfd3a1584adca913a8ef4b43c684fca97e04e39e4c50a3a5e09650",
  "status": "starting",
  "created_at": "2026-10-23T10:39:40.360Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/promptable-image-segmentation-vufinder-sam3/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "3p61mg2ej5rmt0d1d1c8eakjdw",
  "status": "succeeded",
  "output": {
    "backbone_outputs": null,
    "results": [
      "URL of the JSON results for prompt 1",
      "URL of the JSON results for prompt 2",
      "URL of the JSON results for prompt 3"
    ],
    "visualizations": [
      "https://blog.api.market/wp-content/uploads/2026/10/sam3-output-1-3p61mg2e-api-market.jpg",
      "https://blog.api.market/wp-content/uploads/2026/10/sam3-output-2-3p61mg2e-api-market.png",
      "https://blog.api.market/wp-content/uploads/2026/10/sam3-output-3-3p61mg2e-api-market.png"
    ]
  },
  "error": null,
  "created_at": "2026-10-23T10:39:40.360Z",
  "started_at": "2026-10-23T10:39:40.371Z",
  "completed_at": "2026-10-23T10:39:45.900Z"
}
```

`output` is an object. `results` is a list of JSON file URLs with the predicted masks (one per prompt when `split_output` is `true`). `visualizations` is a list of PNG URLs, one per prompt, when `visualize` is `true`. `backbone_outputs` is a PT file URL when `backbone_output` is `true`. Fields that were not requested are `null`.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `image` or `prompts`, a prompt that is not valid JSON, or a `confidence_threshold` outside 0.01 to 0.99.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/promptable-image-segmentation-vufinder-sam3";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "1bf97763d5dfd3a1584adca913a8ef4b43c684fca97e04e39e4c50a3a5e09650", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "image": "https://blog.api.market/wp-content/uploads/2026/10/sam3-input-api-market.jpg",
  "prompts": [
    "{\"text\": \"pillow\"}",
    "{\"positive_boxes\": [[0.19, 0.73, 0.3, 0.3]]}",
    "{\"positive_points\": [[0.5, 0.87]]}"
  ],
  "visualize": true,
  "backbone_output": false,
  "confidence_threshold": 0.5
}).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/promptable-image-segmentation-vufinder-sam3"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "1bf97763d5dfd3a1584adca913a8ef4b43c684fca97e04e39e4c50a3a5e09650", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "image": "https://blog.api.market/wp-content/uploads/2026/10/sam3-input-api-market.jpg",
    "prompts": [
        "{\"text\": \"pillow\"}",
        "{\"positive_boxes\": [[0.19, 0.73, 0.3, 0.3]]}",
        "{\"positive_points\": [[0.5, 0.87]]}"
    ],
    "visualize": True,
    "backbone_output": False,
    "confidence_threshold": 0.5
}))
```

## FAQ
- **How do I write a prompt?**
  - Each item in `prompts` is a JSON string, for example `{"text": "pillow"}`, `{"positive_boxes": [[0.19, 0.73, 0.3, 0.3]]}` or `{"positive_points": [[0.5, 0.87]]}`. You can combine keys in one prompt and add negative boxes or points to exclude areas.
- **What coordinate format do boxes and points use?**
  - All coordinates are normalized to the range 0 to 1. Boxes are `center_x, center_y, width, height`; points are `x, y`.
- **Can I segment several concepts at once?**
  - Yes. Add one prompt per concept to `prompts`. With `split_output` on (the default) you get one results file per prompt.
- **How do I get fewer or more detections?**
  - Raise `confidence_threshold` (up to 0.99) to keep only confident results, or lower it (down to 0.01) to keep more.
- **How is a request billed?**
  - $0.0044 per request for up to 4 seconds of processing, plus one unit for each further started block of 4 seconds. The example with three prompts took 5.5 seconds, so it used 2 units and cost $0.0088.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Segment objects in images with Meta SAM 3 using text, box or point prompts; get masks as JSON and optional per-prompt visualizations.