Name: IBM Granite 3.3 8B Instruct API: Open Source LLM
Brand: magicapi
SKU: api-magicapi-improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct
Availability: InStock
Author: API Market

Shashank Agarwal

# 🧠 Instruction-Following Language Model API (IBM Granite 3.3 8B Instruct)

## Overview
Granite-3.3-8B-Instruct is an 8-billion-parameter language model from IBM with a 128K context length. Its developer describes it as fine-tuned for improved reasoning and instruction following. The model weights are released under the Apache 2.0 license.

Send a single `prompt` (with an optional `system_prompt`) or a list of chat `messages`. You can also pass `documents`, `tools` and a `response_format` to the model's chat template. The reply comes back as a list of text chunks; join them in order to get the full text.

## Features
- **Prompt or chat input**: send one `prompt` or a multi-turn `messages` list.
- **Chat template inputs**: `documents`, `tools`, `tool_choice`, `chat_template` and `chat_template_kwargs` are passed to the chat template.
- **Output format**: `response_format` sets the format the model must output.
- **Length control**: `max_completion_tokens` (or the deprecated `max_tokens`) caps the output; `min_tokens` sets a minimum; `stop` ends generation at given sequences.
- **Sampling controls**: `temperature` (default 0.6), `top_p` (default 0.9), `top_k` (default 50), `presence_penalty`, `frequency_penalty` and `seed`.
- **Low token price**: $0.0384 per million input tokens and $0.32 per million output tokens.

## Use Cases
1. **Question answering**: explanations and how-to answers for technical and general topics.
2. **Chat assistants**: multi-turn conversations with `messages` and a `system_prompt`.
3. **Answers grounded in your text**: pass reference material in `documents` and ask about it.
4. **High-volume text tasks**: rewriting, summarizing and classification where a small, inexpensive model is enough.

## Visual Examples
### Example Input
Prompt: "How is perplexity measured for LLMs and why is it useful?"

Sent with `max_tokens` `512`, `temperature` `0.6`, `top_p` `0.9`, `top_k` `50`, `min_tokens` `0` and both penalties at `0`.

### Example Output
The output chunks joined in order (73 input tokens, 488 output tokens):

```text
Perplexity is a common metric used to evaluate the performance of language models, including large language models (LLMs). It measures how well a model predicts a sample. A lower perplexity score indicates better performance, as it suggests the model is more confident in its predictions. 

To calculate perplexity, we first generate probabilities for each word in a test set given the context of the previous words, according to the model's probability distribution. Then, we take the geometric mean of these probabilities, effectively averaging them. The perplexity score is the inverse of this geometric mean.

Mathematically, perplexity (P) is given by:

P = 2^(-1/Σ(log2(p_i)))

where p_i is the probability assigned by the model to the i-th word in the test set.

Perplexity is useful for LLMs for several reasons:

1. **Model comparison**: Perplexity allows for easy comparison of different language models. Lower perplexity implies a better model, as it demonstrates greater confidence in predicting the test data.

2. **Training optimization**: During the training process, perplexity serves as a proxy for the likelihood of the training data. By monitoring perplexity, researchers can gauge the effectiveness of various hyperparameters, architectures, or training techniques. As the model improves, perplexity should decrease.

3. **Generalization**: Perplexity on unseen data reflects how well the model generalizes to new, unseen examples. A good language model should have low perplexity on diverse and varied test sets, indicating its ability to capture and understand linguistic patterns and structures.

4. **Evaluation of language understanding**: Perplexity is a quantitative measure of a language model's ability to capture the underlying structure and patterns of a language. A lower perplexity score implies that the model has a better grasp of the language, enabling it to generate more coherent and contextually relevant text.

Keep in mind that perplexity has its limitations, as it doesn't directly measure the model's ability to perform specific tasks or understand semantics. Nonetheless, it remains a valuable and widely-used metric for assessing the overall performance of language models.
```

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | none |  | User prompt for a completion request |
| `messages` | array of objects | No | `[]` |  | Chat completion messages, a list of role and content objects |
| `documents` | array of objects | No | `[]` |  | Documents for the request, passed to the chat template |
| `tools` | array of objects | No | `[]` |  | Tool definitions for the request, passed to the chat template |
| `tool_choice` | string | No | none |  | Tool choice for the request. If the choice is a specific function, pass it as a JSON string |
| `response_format` | object | No | none |  | An object specifying the format that the model must output |
| `system_prompt` | string | No | none |  | System prompt. When it is not set, the chat template provides a default |
| `chat_template` | string | No | none |  | A template to format the prompt with. When it is not set, the model's own chat template is used |
| `add_generation_prompt` | boolean | No | `true` |  | Add the generation prompt. Passed to the chat template |
| `chat_template_kwargs` | object | No | `{}` |  | Additional arguments passed to the chat template |
| `min_tokens` | integer | No | `0` |  | The minimum number of tokens the model should generate |
| `max_tokens` | integer | No | none |  | Deprecated in favor of max_completion_tokens. Upper bound on the number of output tokens |
| `max_completion_tokens` | integer | No | none |  | Upper bound on the number of tokens generated for a completion, including visible output tokens and reasoning tokens |
| `temperature` | number | No | `0.6` |  | Sampling temperature. Lower values make the output more deterministic |
| `top_p` | number | No | `0.9` |  | Nucleus sampling: if below 1.0, only the most likely tokens whose cumulative probability reaches top_p are considered |
| `top_k` | integer | No | `50` |  | Number of highest-probability tokens to consider. If above 0, only the top k tokens are kept (top-k filtering) |
| `presence_penalty` | number | No | `0` |  | Presence penalty |
| `frequency_penalty` | number | No | `0` |  | Frequency penalty |
| `stop` | array of strings | No | `[]` |  | Sequences that stop generation. For example, ["<end>", "<stop>"] stops at the first "<end>" or "<stop>" |
| `seed` | integer | No | none |  | Random seed. Leave unset for a random seed |
| `stream` | boolean | No | `false` |  | Request a streaming response. This API returns results through GET /predictions/{id}, so leave it false |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0032 per API unit.
- **1 API unit = 10,000 weighted tokens; an output token counts 1, an input token counts 0.12.**
- That works out to **$0.0384 per million input tokens** and **$0.32 per million output tokens**.
- Each request is rounded up to a whole API unit, so every successful request costs at least $0.0032.
- **Example cost**: the example request used 73 input tokens and 488 output tokens: 73 x 0.12 + 488 = 496.76 weighted tokens, rounded up to 1 API unit = **$0.0032**.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "ibm-granite/granite-3.3-8b-instruct",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "How is perplexity measured for LLMs and why is it useful?",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }'
```

Response (201 Created):
```json
{
  "id": "4w7s80qby9rn80d1c6mbmr7p3g",
  "version": "ibm-granite/granite-3.3-8b-instruct",
  "status": "starting",
  "created_at": "2026-10-22T03:30:23.041Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "4w7s80qby9rn80d1c6mbmr7p3g",
  "status": "succeeded",
  "output": [
    "Perplexity is a common metric used to evaluate the performance of language models, including large language models (LLMs). It measures how well a model predicts a sample. A lower perplexity score indicates better performance, as it suggests the model is more confident in its predictions. \n\nTo calculate perplexity, we first generate probabilities for each word in a test set given the context of the previous words, according to the model's probability distribution. Then, we take the geometric mean of these probabilities, effectively averaging them. The perplexity score is the inverse of this geometric mean.\n\nMathematically, perplexity (P) is given by:\n\nP = 2^(-1/Σ(log2(p_i)))\n\nwhere p_i is the probability assigned by the model to the i-th word in the test set.\n\nPerplexity is useful for LLMs for several reasons:\n\n1. **Model comparison**: Perplexity allows for easy comparison of different language models. Lower perplexity implies a better model, as it demonstrates greater confidence in predicting the test data.\n\n2. **Training optimization**: During the training process, perplexity serves as a proxy for the likelihood of the training data. By monitoring perplexity, researchers can gauge the effectiveness of various hyperparameters, architectures, or training techniques. As the model improves, perplexity should decrease.\n\n3. **Generalization**: Perplexity on unseen data reflects how well the model generalizes to new, unseen examples. A good language model should have low perplexity on diverse and varied test sets, indicating its ability to capture and understand linguistic patterns and structures.\n\n4. **Evaluation of language understanding**: Perplexity is a quantitative measure of a language model's ability to capture the underlying structure and patterns of a language. A lower perplexity score implies that the model has a better grasp of the language, enabling it to generate more coherent and contextually relevant text.\n\nKeep in mind that perplexity has its limitations, as it doesn't directly measure the model's ability to perform specific tasks or understand semantics. Nonetheless, it remains a valuable and widely-used metric for assessing the overall performance of language models."
  ],
  "error": null,
  "created_at": "2026-10-22T03:30:23.041Z",
  "started_at": "2026-10-22T03:30:24.041Z",
  "completed_at": "2026-10-22T03:30:30.517Z"
}
```

`output` is a list of text chunks. Join them in order to get the full reply. The example above returned the reply as one chunk; longer replies can be split into several chunks.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `messages` value that is not a list or a `temperature` that is not a number.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "ibm-granite/granite-3.3-8b-instruct", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "How is perplexity measured for LLMs and why is it useful?",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "presence_penalty": 0,
  "frequency_penalty": 0
}).then((output) => console.log(output.join("")));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/improved-reasoning-instruction-following-ibm-granite-granite-3-3-8b-instruct"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "ibm-granite/granite-3.3-8b-instruct", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "How is perplexity measured for LLMs and why is it useful?",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "presence_penalty": 0,
    "frequency_penalty": 0
})
print("".join(output))
```

## FAQ
- **How do I get the full reply text?**
  - `output` is an array of text chunks. Join them in order with no separator: `output.join("")` in JavaScript or `"".join(output)` in Python.
- **Should I use prompt or messages?**
  - Use `prompt` (with an optional `system_prompt`) for a single question. Use `messages` for a multi-turn conversation, as a list of role and content objects.
- **How long can my input be?**
  - The model has a 128K token context length, shared between the input and the generated output.
- **Which length field should I use?**
  - Use `max_completion_tokens`. `max_tokens` still works but is deprecated.
- **How much does a request cost?**
  - $0.0384 per million input tokens and $0.32 per million output tokens, rounded up to a whole API unit ($0.0032) per request. The example request cost $0.0032.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Call IBM Granite 3.3 8B Instruct for chat, question answering and instruction following, with prompt or messages input and sampling controls.