Name: LocateAnything-3B API: Visual Grounding and Detection
Brand: magicapi
SKU: api-magicapi-visual-grounding-api-csviverdeia-locateanything-3b-h100
Availability: InStock
Author: API Market

Shashank Agarwal

# 🎯 Visual Grounding and Object Detection API (csviverdeia LocateAnything-3B)

## Overview
LocateAnything-3B is a 3-billion-parameter visual grounding model. Tell it what to find in plain language, for example "detect all cattle", and it returns a bounding box for every match, the number of detections and an annotated copy of your image or video.

Send either an `image` or a `video`, not both. For images, `tiles` splits the picture into a grid to catch small or distant objects, and `verify` re-checks each box with CLIP to drop false positives. For videos, the model samples frames (`detect_fps`, `max_detect_frames`) and can assign track IDs with `tracker`.

## Features
- **Open-vocabulary grounding**: describe the target in `prompt` instead of picking from a fixed class list.
- **Image or video input**: send `image` for a single picture or `video` for frame-by-frame detection.
- **Tiling for dense scenes**: `tiles` (1 to 5) runs detection on an N x N grid at full resolution; useful for counting many small objects.
- **False-positive check**: set `verify` to `true` and name the object in `verify_object` to re-check every box with CLIP.
- **Video tracking**: `tracker` set to `sort` or `reid` gives detections IDs that follow objects across frames.
- **Annotated output**: the response includes an annotated image or video plus the boxes as JSON in pixel coordinates.

## Use Cases
1. **Counting**: livestock in a field, cars in a parking lot or items on a shelf.
2. **Dataset labelling**: draft bounding boxes for a custom class described in words.
3. **Video analysis**: locate and track people or vehicles across the frames of a clip.
4. **Visual search**: check whether an image contains a described object and where it is.

## Visual Examples
### Example Input
Prompt: "detect all cattle"

![Input photo: cattle in a field](https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-input-api-market.jpeg)

Other fields: `generation_mode` `hybrid`, `tracker` `none`, `detect_fps` `3`, `max_detect_frames` `24`, `max_new_tokens` `2048`, `temperature` `0.2`, `top_p` `0.9`, `repetition_penalty` `1.1`.

### Example Output
The model found 35 cattle (`num_detections` is `35`) and returned this annotated image:

![LocateAnything-3B output: the input photo with a box drawn around each of the 35 cattle](https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-output-kh82wwwq-api-market.png)

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `image` | string (URL) | No | none |  | Input RGB image. Use this for grounding on an image |
| `video` | string (URL) | No | none |  | Input video. Use this for grounding on a video (sampled into frames). Send `image` or `video`, not both |
| `prompt` | string | No | `Detect all the main objects and output their bounding boxes.` |  | Natural-language instruction of what to locate, for example 'Detect the red car' |
| `generation_mode` | string | No | `hybrid` | `hybrid`, `ar`, `mtp` | Decoding mode. `hybrid` balances speed and accuracy |
| `tiles` | integer | No | `1` | 1 to 5 | [Image] Splits the image into an N x N grid and detects in each tile at full resolution, which recovers small or distant objects. 1 = off; 3 = 3 x 3 = 9 passes (slower). Useful for dense counting |
| `verify` | boolean | No | `false` |  | [Image] Checks every box with CLIP and drops false positives (for example a rock or a bush marked as the object). Requires `verify_object` |
| `verify_object` | string | No | `""` |  | [Image] What every box must contain, in English and singular (for example 'cow', 'egg', 'box', 'person'). Used by the CLIP check |
| `verify_threshold` | number | No | `0.15` | 0 to 0.9 | [Image] Strictness of the CLIP check. Low (0.1 to 0.2) only removes obvious false positives (recommended); high (0.4 and above) is strict and can remove real small or blurry objects |
| `detect_fps` | number | No | `3` | 0.5 to 10 | [Video] How many times per second to run detection. Higher keeps boxes closer to moving objects but is slower |
| `max_detect_frames` | integer | No | `24` | 1 to 120 | [Video] Maximum number of frames to run detection on, spread across the whole video, to limit processing time |
| `tracker` | string | No | `none` | `none`, `sort`, `reid` | [Video] `none` = boxes per frame without IDs; `sort` = IDs from motion (Kalman filter and Hungarian matching); `reid` = SORT plus appearance features (ResNet18), IDs that hold through occlusion but slower |
| `max_new_tokens` | integer | No | `2048` | 64 to 8192 | Maximum number of tokens the model generates |
| `temperature` | number | No | `0.2` | 0 to 2 | Sampling temperature |
| `top_p` | number | No | `0.9` | 0 to 1 | Nucleus sampling probability |
| `repetition_penalty` | number | No | `1.1` | 1 to 2 | Penalty for repeated tokens; 1 means no penalty |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0088 per API unit.
- **1 API unit = 1 request with up to 5 seconds of processing.** A request that takes longer uses one unit per started block of 5 seconds (for example, 12 seconds of processing uses 3 units, $0.0264).
- Videos, more `tiles`, more `max_detect_frames` and the `reid` tracker take longer and can use more units.
- **Example cost**: the example request took 2.3 seconds of processing, which is 1 unit, $0.0088.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/visual-grounding-api-csviverdeia-locateanything-3b-h100`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/visual-grounding-api-csviverdeia-locateanything-3b-h100/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "5b815a521fbfc4b330c62725cd465eaf60452eb0e2dd34ed35649cc9d38e4510",
    "input": {
      "image": "https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-input-api-market.jpeg",
      "top_p": 0.9,
      "prompt": "detect all cattle",
      "tracker": "none",
      "detect_fps": 3,
      "temperature": 0.2,
      "max_new_tokens": 2048,
      "generation_mode": "hybrid",
      "max_detect_frames": 24,
      "repetition_penalty": 1.1
    }
  }'
```

Response (201 Created):
```json
{
  "id": "kh82wwwq5nrnr0d1d0vsbaw690",
  "version": "5b815a521fbfc4b330c62725cd465eaf60452eb0e2dd34ed35649cc9d38e4510",
  "status": "starting",
  "created_at": "2026-10-23T10:03:56.377Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/visual-grounding-api-csviverdeia-locateanything-3b-h100/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "kh82wwwq5nrnr0d1d0vsbaw690",
  "status": "succeeded",
  "output": {
    "detections": "{\"modality\": \"image\", \"answer\": \"<ref>cattle</ref><box><0><438><78><569></box><box><3><378><97><447></box>\", \"coords\": \"pixels\", \"tiles\": 1, \"verified\": null, \"image_width\": 626, \"image_height\": 352, \"boxes\": [{\"x1\": 0.0, \"y1\": 154.18, \"x2\": 48.83, \"y2\": 200.29}, {\"x1\": 1.88, \"y1\": 133.06, \"x2\": 60.72, \"y2\": 157.34}], \"points\": []}",
    "image": "https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-output-kh82wwwq-api-market.png",
    "num_detections": 35,
    "video": null
  },
  "error": null,
  "created_at": "2026-10-23T10:03:56.377Z",
  "started_at": "2026-10-23T10:05:24.428Z",
  "completed_at": "2026-10-23T10:05:26.765Z"
}
```

`output` is an object. `num_detections` is the number of boxes found. `image` (for image input) or `video` (for video input) is the URL of the annotated file; the other one is `null`. `detections` is a JSON string with `modality`, the raw model `answer`, `coords` (`pixels`), `tiles`, `verified`, `image_width`, `image_height`, `boxes` (each with `x1`, `y1`, `x2`, `y2` in pixels) and `points`. The example above is shortened to 2 of its 35 boxes.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `tiles` value above 5 or a `tracker` that is not in the list.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/visual-grounding-api-csviverdeia-locateanything-3b-h100";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "5b815a521fbfc4b330c62725cd465eaf60452eb0e2dd34ed35649cc9d38e4510", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "image": "https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-input-api-market.jpeg",
  "top_p": 0.9,
  "prompt": "detect all cattle",
  "tracker": "none",
  "detect_fps": 3,
  "temperature": 0.2,
  "max_new_tokens": 2048,
  "generation_mode": "hybrid",
  "max_detect_frames": 24,
  "repetition_penalty": 1.1
}).then((output) => console.log(output.num_detections, JSON.parse(output.detections).boxes));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/visual-grounding-api-csviverdeia-locateanything-3b-h100"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "5b815a521fbfc4b330c62725cd465eaf60452eb0e2dd34ed35649cc9d38e4510", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "image": "https://blog.api.market/wp-content/uploads/2026/10/locateanything-3b-h100-input-api-market.jpeg",
    "top_p": 0.9,
    "prompt": "detect all cattle",
    "tracker": "none",
    "detect_fps": 3,
    "temperature": 0.2,
    "max_new_tokens": 2048,
    "generation_mode": "hybrid",
    "max_detect_frames": 24,
    "repetition_penalty": 1.1
})
print(output["num_detections"], output["detections"])
```

## FAQ
- **Can I send an image and a video in the same request?**
  - No. Send either `image` or `video`.
- **How do I count many small objects?**
  - Raise `tiles` (for example to `3`, which runs 9 passes on a 3 x 3 grid). It is slower, so it can use more units.
- **How do I remove wrong boxes?**
  - Set `verify` to `true` and `verify_object` to the singular English name of the object, for example `cow`. Keep `verify_threshold` low (0.1 to 0.2) unless you want a strict filter.
- **What coordinates do the boxes use?**
  - Pixels of the input image, given as `x1`, `y1` (top left) and `x2`, `y2` (bottom right). `image_width` and `image_height` are included in `detections`.
- **Do detections in a video keep the same ID across frames?**
  - Only with `tracker` set to `sort` or `reid`. With `none`, each frame has its own boxes without IDs.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Describe what to find and LocateAnything-3B returns bounding boxes and an annotated image or video, with tiling for dense counts and optional tracking in video.