Name: Embedding Gemma 2 API: Multimodal Embeddings by Google
Brand: magicapi
SKU: api-magicapi-multimodal-embedding-api-yanexr-embedding-gemma-2
Availability: InStock
Author: API Market

Shashank Agarwal

# 🔗 Multimodal Embedding API (Google Embedding Gemma 2)

## Overview
Embedding Gemma 2 is a multimodal embedding model. Its packager, yanexr, describes it as Google DeepMind's 740M multimodal embedding model that embeds text, images, video and audio into one shared 768-dimensional space (or 512, 256 or 128), with an 8K token input window.

Send any mix of `text`, `images`, `videos` and `audio`. Each input gets its own embedding, or set `combine` to `true` to get one embedding for everything together. Set `task` to match how you will use the vectors, and `dimensions` to choose their length. The output is an object with an `embeddings` list.

## Features
- **Text, images, video and audio**: all inputs map into one shared space, so a text query can be compared with images, video or audio.
- **Batch inputs**: `text` takes one string or a list of strings; `images`, `videos` and `audio` take lists of URLs. Each item gets its own embedding.
- **Combined embeddings**: `combine` makes one embedding for all inputs together, for example a product photo with its description. Write `<|image|>`, `<|video|>` or `<|audio|>` in the text to place files at that spot.
- **Task-specific embeddings**: `task` (default `document`) supports `search_query`, `question_answering`, `fact_checking`, `code_retrieval`, `classification`, `clustering`, `similarity` and `none`.
- **Vector length**: `dimensions` of 768 (default), 512, 256 or 128. Shorter vectors need less storage at a small cost in quality.
- **Video and image detail**: `image_tokens`, `video_frame_tokens`, `video_fps`, `max_video_frames` and `include_video_audio` control how much of each image and video is used.

## Use Cases
1. **Semantic search**: embed documents and products with `task` `document` and queries with `search_query`, then rank by similarity.
2. **Cross-modal search**: find images, videos or audio clips that match a text query.
3. **Product catalogs**: combine a product photo and its description into one embedding with `combine`.
4. **Clustering and classification**: group or label items using the `clustering` or `classification` task for all inputs.

## Visual Examples
### Example Input
Two product descriptions in `text` (a JSON array of strings) and two product images in `images`, with `task` `document` and 768 dimensions:

![Embedding Gemma 2 example input image 1](https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-api-market.jpg)

![Embedding Gemma 2 example input image 2](https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-2-api-market.jpg)

```json
{
  "task": "document",
  "text": [
    "Lightweight waterproof hiking jacket with taped seams and a packable hood.",
    "Insulated stainless steel water bottle that keeps drinks cold for 24 hours."
  ],
  "audio": [],
  "images": [
    "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-api-market.jpg",
    "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-2-api-market.jpg"
  ],
  "videos": [],
  "combine": false,
  "video_fps": 1,
  "dimensions": 768,
  "image_tokens": 280,
  "max_video_frames": 32,
  "video_frame_tokens": 140,
  "include_video_audio": false
}
```

### Example Output
Four embeddings of 768 numbers each, in input order: the two texts first, then the two images. The first four values of each were:

```json
{
  "embeddings": [
    [-0.004626674111932516, -0.02641284093260765, 0.0501469150185585, -0.02006773091852665],
    [-0.022537080571055412, -0.0250280499458313, -0.003178280545398593, -0.025611087679862976],
    [0.013732842169702051, -0.01661551557481289, 0.036469973623752594, 0.004452608060091734],
    [-0.03592371195554733, -0.044946156442165375, 0.02650635689496994, -0.03171700984239578]
  ]
}
```

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `text` | string or array of strings | No | `""` |  | Text to embed, or a list of texts. Each text gets its own embedding, unless `combine` is on. |
| `images` | array of URLs | No | `[]` |  | Image URLs to embed. Each image gets its own embedding, unless `combine` is on. |
| `videos` | array of URLs | No | `[]` |  | Video URLs to embed. Each video gets its own embedding, unless `combine` is on. Only the picture is used; see `include_video_audio` for the sound. |
| `audio` | array of URLs | No | `[]` |  | Audio file URLs to embed. Each file gets its own embedding, unless `combine` is on. Video files work too; then only their sound is used. |
| `task` | string | No | `document` | `document`, `search_query`, `question_answering`, `fact_checking`, `code_retrieval`, `classification`, `clustering`, `similarity`, `none` | What the embeddings are for. Embed stored items with `document` and queries with `search_query`. For classification, clustering or similarity, use that task for all inputs. |
| `combine` | boolean | No | `false` |  | Create one embedding for all inputs together instead of one per input, for things that belong together such as a product photo and its description. Write <\|image\|>, <\|video\|> or <\|audio\|> in the text to place files at that spot. |
| `dimensions` | integer | No | `768` | `768`, `512`, `256`, `128` | Length of the embeddings. Shorter ones need less storage at a small cost in quality. Use the same value for queries and stored items. |
| `include_video_audio` | boolean | No | `false` |  | Advanced. Add the sound of each video to that video's embedding. |
| `image_tokens` | integer | No | `280` | `70`, `140`, `280`, `560`, `1120` | Advanced. Detail per image. Higher captures more detail but is slower. |
| `video_frame_tokens` | integer | No | `140` | `70`, `140`, `280`, `560`, `1120` | Advanced. Detail per video frame. Higher captures more detail but is slower and fits fewer frames. |
| `video_fps` | number | No | `1` | `0.1` to `10` | Advanced. Frames taken per second of video. |
| `max_video_frames` | integer | No | `32` | `1` to `113` | Advanced. Most frames taken from one video. Longer videos are sampled evenly over their full length. |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0022 per API unit.
- **1 API unit = 1 successful request with up to 2 seconds of processing.** A request that takes longer is billed one more unit for each started block of 2 seconds (for example, 5 seconds of processing is 3 units, $0.0066).
- Short texts and a few images usually finish within one unit; many inputs, long videos or long audio take longer and can use more units.
- **Example cost**: the example request above took 0.2 seconds of processing, so it used 1 API unit and cost $0.0022.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/multimodal-embedding-api-yanexr-embedding-gemma-2`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/multimodal-embedding-api-yanexr-embedding-gemma-2/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "b3b0100d9dec44054cd54bdcabff0d242dae77d6cf9497ce7c4cd1fe324f7ec2",
    "input": {
      "task": "document",
      "text": [
        "Lightweight waterproof hiking jacket with taped seams and a packable hood.",
        "Insulated stainless steel water bottle that keeps drinks cold for 24 hours."
      ],
      "audio": [],
      "images": [
        "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-api-market.jpg",
        "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-2-api-market.jpg"
      ],
      "videos": [],
      "combine": false,
      "video_fps": 1,
      "dimensions": 768,
      "image_tokens": 280,
      "max_video_frames": 32,
      "video_frame_tokens": 140,
      "include_video_audio": false
    }
  }'
```

Response (201 Created):
```json
{
  "id": "z93kzggfrnrmy0d1d21sk3wt8w",
  "version": "b3b0100d9dec44054cd54bdcabff0d242dae77d6cf9497ce7c4cd1fe324f7ec2",
  "status": "starting",
  "created_at": "2026-10-23T11:26:22.313Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/multimodal-embedding-api-yanexr-embedding-gemma-2/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK, each vector shortened to its first 4 of 768 numbers):
```json
{
  "id": "z93kzggfrnrmy0d1d21sk3wt8w",
  "status": "succeeded",
  "output": {
    "embeddings": [
      [
        -0.004626674111932516,
        -0.02641284093260765,
        0.0501469150185585,
        -0.02006773091852665
      ],
      [
        -0.022537080571055412,
        -0.0250280499458313,
        -0.003178280545398593,
        -0.025611087679862976
      ],
      [
        0.013732842169702051,
        -0.01661551557481289,
        0.036469973623752594,
        0.004452608060091734
      ],
      [
        -0.03592371195554733,
        -0.044946156442165375,
        0.02650635689496994,
        -0.03171700984239578
      ]
    ]
  },
  "error": null,
  "created_at": "2026-10-23T11:26:22.313Z",
  "started_at": "2026-10-23T11:26:42.616Z",
  "completed_at": "2026-10-23T11:26:42.828Z"
}
```

`output` is an object with an `embeddings` list. Each item is one vector of `dimensions` numbers, one per input in order (texts, then images in the example), or a single vector when `combine` is `true`.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `dimensions` value other than 768, 512, 256 or 128, a `task` that is not in the list or a `video_fps` outside 0.1 to 10.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/multimodal-embedding-api-yanexr-embedding-gemma-2";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "b3b0100d9dec44054cd54bdcabff0d242dae77d6cf9497ce7c4cd1fe324f7ec2", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "task": "document",
  "text": [
    "Lightweight waterproof hiking jacket with taped seams and a packable hood.",
    "Insulated stainless steel water bottle that keeps drinks cold for 24 hours."
  ],
  "audio": [],
  "images": [
    "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-api-market.jpg",
    "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-2-api-market.jpg"
  ],
  "videos": [],
  "combine": false,
  "video_fps": 1,
  "dimensions": 768,
  "image_tokens": 280,
  "max_video_frames": 32,
  "video_frame_tokens": 140,
  "include_video_audio": false
}).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/multimodal-embedding-api-yanexr-embedding-gemma-2"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "b3b0100d9dec44054cd54bdcabff0d242dae77d6cf9497ce7c4cd1fe324f7ec2", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "task": "document",
    "text": [
        "Lightweight waterproof hiking jacket with taped seams and a packable hood.",
        "Insulated stainless steel water bottle that keeps drinks cold for 24 hours."
    ],
    "audio": [],
    "images": [
        "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-api-market.jpg",
        "https://blog.api.market/wp-content/uploads/2026/10/embedding-gemma-2-input-2-api-market.jpg"
    ],
    "videos": [],
    "combine": False,
    "video_fps": 1,
    "dimensions": 768,
    "image_tokens": 280,
    "max_video_frames": 32,
    "video_frame_tokens": 140,
    "include_video_audio": False
}))
```

## FAQ
- **Which `task` should I use?**
  - Embed the items you store with `document` and the user queries with `search_query`. For classification, clustering or similarity, use that task for every input so the vectors are comparable.
- **How do I embed several texts in one request?**
  - Send `text` as a JSON array of strings, for example `["first text", "second text"]`. Each string gets its own embedding, as in the example.
- **Can I compare text with images or video?**
  - Yes. All inputs are embedded into one shared space, so you can compare a text vector with image, video or audio vectors. Use the same `dimensions` for everything you compare.
- **When should I use `combine`?**
  - When several inputs describe one thing, such as a product photo and its description. You then get one embedding instead of one per input.
- **How is a request billed?**
  - $0.0022 per request for up to 2 seconds of processing, plus one unit for each further started block of 2 seconds. The example with two texts and two images took 0.2 seconds and cost $0.0022.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Embed text, images, video and audio into one shared space with Embedding Gemma 2: 768, 512, 256 or 128 dimensions, task-specific embeddings.