Name: IBM Granite 3.2 8B Instruct API: Reasoning LLM
Brand: magicapi
SKU: api-magicapi-reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct
Availability: InStock
Author: API Market

Shashank Agarwal

# 🧠 Reasoning and Instruction LLM API (IBM Granite 3.2 8B Instruct)

## Overview
Granite 3.2 8B Instruct is an 8B-parameter language model from IBM with a 128K context length. Its developer describes it as fine-tuned for reasoning and instruction following.

Send a text `prompt` and an optional `system_prompt`; the model returns generated text. `max_tokens` (default 512) and `min_tokens` set the length of the answer, and `temperature`, `top_p`, `top_k`, `presence_penalty`, `frequency_penalty` and `stop_sequences` control sampling.

## Features
- **Long context**: a 128K context length for long prompts and documents pasted into the prompt.
- **System prompt**: `system_prompt` (default `You are a helpful assistant.`) is prepended to the prompt to guide behavior.
- **Length controls**: `max_tokens` (default 512) and `min_tokens` (default 0).
- **Sampling controls**: `temperature` (default 0.6), `top_p` (default 0.9) and `top_k` (default 50).
- **Stop sequences and penalties**: `stop_sequences`, `presence_penalty` and `frequency_penalty`.

## Use Cases
1. **Explanations**: step-by-step answers to technical and math questions.
2. **Document Q&A**: answer questions about long texts pasted into the prompt.
3. **Writing and editing**: drafts, rewrites and summaries.
4. **Assistants**: instruction-following chat with a role set by `system_prompt`.

## Visual Examples
### Example Input
Prompt: "How is perplexity measured for LLMs and why is it useful?"

System prompt: `You are a helpful assistant.` Settings: `max_tokens` 512, `min_tokens` 0, `temperature` 0.6, `top_p` 0.9, `presence_penalty` 0, `frequency_penalty` 0.

### Example Output
The chunks of this run joined into one answer (shortened here; the full answer ran to the `max_tokens` limit of 512):

````markdown
Perplexity is a common metric used to evaluate the performance of language models, including large language models (LLMs). It's a measure of how well a probability model predicts a sample. In the context of natural language processing, it's often used to gauge how well a language model predicts a test set.

Here's a simplified explanation of how it's calculated:

1. For each word in the test set, the model calculates the probability of that word given all the previous words.
2. The product of these probabilities for the entire sentence is then computed.
3. Perplexity is then the exponentiation of the average number of words that need to be predicted before making a mistake. 

Mathematically, if P is the probability assigned by the model to the test set, the perplexity (PP) is calculated as:

PP = 2^(-1/N * Σ log2(P(wi|w1...wi-1)))

Where:
- N is the number of words in the test set.
- wi is each word in the test set.
- P(wi|w1...wi-1) is the probability assigned by the model to word wi given the previous words w1 to wi-1.

The "perplexity" is the exponentiation of the average negative log probability. A lower perplexity score indicates better performance, as it means the model is better at predicting the test data.

Perplexity is useful for several reasons:

1. **Model Comparison**: It provides a standardized way to compare the performance of different models. A lower perplexity score indicates a model that is better at predicting the test data.

2. **Model Improvement**: It helps in identifying areas where the model is struggling. High perplexity on certain types of sentences or phrases can indicate that the model needs more training data or different training strategies for those specific cases.

3. **Understanding Model Capabilities**: It gives insights into how well the model understands the language and its nuances. 

4. **User Experience**: In practical applications like speech recognition or machine translation, a lower perplexity often translates to a better user experience, as the system will make fewer errors.

...
````

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | `""` |  | Prompt |
| `system_prompt` | string | No | `You are a helpful assistant.` |  | System prompt to send to the model. It is prepended to the prompt and helps guide the model's behavior. |
| `min_tokens` | integer | No | `0` |  | The minimum number of tokens the model should generate as output. |
| `max_tokens` | integer | No | `512` |  | The maximum number of tokens the model should generate as output. |
| `temperature` | number | No | `0.6` |  | The value used to modulate the next token probabilities. |
| `top_p` | number | No | `0.9` |  | A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). |
| `top_k` | integer | No | `50` |  | The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering). |
| `presence_penalty` | number | No | `0` |  | Presence penalty |
| `frequency_penalty` | number | No | `0` |  | Frequency penalty |
| `stop_sequences` | string | No | none |  | A comma-separated list of sequences to stop generation at. For example, `<end>,<stop>` stops generation at the first instance of `<end>` or `<stop>`. |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0032 per API unit.
- **1 API unit = 10,000 weighted tokens**; an output token counts 1, an input token counts 0.12.
- That is **$0.0384 per million input tokens** and **$0.32 per million output tokens**.
- Each request is rounded up to a whole API unit, so the smallest charge per request is $0.0032.
- **Example cost**: the example request used 22 input tokens and 513 output tokens. 22 x 0.12 + 513 = 515.64 weighted tokens, rounded up to 1 API unit = **$0.0032**.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "ibm-granite/granite-3.2-8b-instruct",
    "input": {
      "top_p": 0.9,
      "prompt": "How is perplexity measured for LLMs and why is it useful?",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are a helpful assistant.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }'
```

Response (201 Created):
```json
{
  "id": "q99j1xrvksrmw0d1d0zb8bf8sg",
  "version": "ibm-granite/granite-3.2-8b-instruct",
  "status": "starting",
  "created_at": "2026-10-23T10:11:03.356Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "q99j1xrvksrmw0d1d0zb8bf8sg",
  "status": "succeeded",
  "output": [
    "Perplexity is a common metric used to evaluate the performance of language models, including large language models (LLMs). It's a measure of how well a probability model predicts a sample. In the context of natural language processing, it's often used to gauge how well a language model predicts a test set.\n\nHere's a simplified explanation of how it's calculated:\n\n1. For each word in the test set, the model calculates the probability of that word given all the previous words.\n2. The product of these probabilities for the entire sentence is then computed.\n3. Perplexity is then the exponentiation of the average number of words that need to be predicted before making a mistake. \n\nMathematically, if P is the probability assigned by the model to the test set, the perplexity (PP) is calculated as:\n\nPP = 2^(-1/N * Σ log2(P(wi|w1...wi-1)))\n\nWhere:\n- N is the number of words in the test set.\n- wi is each word in the test set.\n- P(wi|w1...wi-1) is the probability assigned by the model to word wi given the previous words w1 to wi-1.\n\nThe \"perplexity\" is the exponentiation of the average negative log probability. A lower perplexity score indicates better performance, as it means the model is better at predicting the test data.\n\nPerplexity is useful for several reasons:\n\n1. **Model Comparison**: It provides a standardized way to compare the performance of different models. A lower perplexity score indicates a model that is better at predicting the test data.\n\n2. **Model Improvement**: It helps in identifying areas where the model is struggling. High perplexity on certain types of sentences or phrases can indicate that the model needs more training data or different training strategies for those specific cases.\n\n3. **Understanding Model Capabilities**: It gives insights into how well the model understands the language and its nuances. \n\n4. **User Experience**: In practical applications like speech recognition or machine translation, a lower perplexity often translates to a better user experience, as the system will make fewer errors.\n\n..."
  ],
  "error": null,
  "created_at": "2026-10-23T10:11:03.356Z",
  "started_at": "2026-10-23T10:12:56.331Z",
  "completed_at": "2026-10-23T10:13:00.219Z"
}
```

`output` is an array of text chunks. Join them in order to get the full answer. The example above shows the chunks from this run already joined into one string. The answer is shortened here.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `max_tokens` value that is not an integer.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "ibm-granite/granite-3.2-8b-instruct", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output.join("");
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "top_p": 0.9,
    "prompt": "How is perplexity measured for LLMs and why is it useful?",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are a helpful assistant.",
    "presence_penalty": 0,
    "frequency_penalty": 0
  }).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/reasoning-instruction-language-api-ibm-granite-granite-3-2-8b-instruct"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "ibm-granite/granite-3.2-8b-instruct", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return "".join(prediction["output"])

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "top_p": 0.9,
    "prompt": "How is perplexity measured for LLMs and why is it useful?",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are a helpful assistant.",
    "presence_penalty": 0,
    "frequency_penalty": 0
})
print(output)
```

## FAQ
- **Why does `output` come back as a list?**
  - The answer is returned as an array of text chunks. Join them in order (`output.join("")` in JavaScript, `"".join(output)` in Python).
- **Why did my answer stop mid-sentence?**
  - It reached `max_tokens` (default 512). Raise `max_tokens` for longer answers; the example run stopped at this limit.
- **How do I set the model's role or tone?**
  - Use `system_prompt`. It is prepended to your prompt; the default is `You are a helpful assistant.`
- **How is a request billed?**
  - By weighted tokens at $0.0032 per 10,000: input tokens count 0.12 and output tokens count 1, rounded up to a whole unit per request. Most short requests cost $0.0032.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Run IBM Granite 3.2 8B Instruct, an 8B model with 128K context fine-tuned for reasoning and instruction following. Pay as you go, no monthly fee.