Name: IBM Granite 4.1 8B API: Long-Context Instruct LLM
Brand: magicapi
SKU: api-magicapi-long-context-instruct-model-ibm-granite-granite-4-1-8b
Availability: InStock
Author: API Market

Shashank Agarwal

# 💬 Long-Context Instruct LLM API (IBM Granite 4.1 8B)

## Overview
Granite 4.1 8B is an 8B-parameter long-context instruct model from IBM, fine-tuned from Granite 4.1 8B Base on open-source instruction datasets with permissive licenses and on IBM's own synthetic datasets.

Send a single `prompt` (with an optional `system_prompt`) or a `messages` array for chat. You can also pass `documents` and `tools` to the chat template and constrain the answer with `response_format`. The model returns generated text.

## Features
- **8B parameters, long context**: a smaller long-context Granite model for everyday instruction tasks.
- **Prompt or chat**: send a single `prompt` (with optional `system_prompt`) or a `messages` array for multi-turn chat.
- **Documents and tools**: `documents`, `tools` and `tool_choice` are passed to the model's chat template for grounded answers and function calling.
- **Structured output**: `response_format` specifies the format the model must output.
- **Length controls**: `max_completion_tokens` caps the generated tokens; `min_tokens` sets a minimum. `max_tokens` is deprecated.
- **Sampling controls**: `temperature`, `top_p`, `top_k`, `presence_penalty`, `frequency_penalty`, `repetition_penalty`, `stop` and `seed`.

## Use Cases
1. **Question answering**: explanations and summaries in plain text.
2. **Retrieval-augmented answers**: pass retrieved passages in `documents` and ask about them.
3. **Function calling**: describe your functions in `tools` and let the model choose with `tool_choice`.
4. **Structured extraction**: request a fixed output shape with `response_format`.

## Visual Examples
### Example Input
Prompt: "What is Pi?"

Settings: `temperature` 0, `top_p` 0.9, `top_k` 50, `min_tokens` 0, `stream` `false`, `add_generation_prompt` `true`; `messages`, `documents`, `tools` and `stop` empty.

### Example Output
The chunks of this run joined into one answer:

````markdown
Pi (π) is a mathematical constant that represents the ratio of a circle's circumference to its diameter. It is an irrational number, meaning it cannot be expressed as a simple fraction and its decimal representation goes on forever without repeating. The value of pi is approximately 3.14159, but it has been calculated to trillions of decimal places for various computational purposes.

Key points about pi:

1. **Irrationality**: Pi cannot be exactly expressed as a fraction of two integers. Its decimal expansion is infinite and non-repeating.

2. **Symbol**: The symbol π was first used to represent this constant by Welsh mathematician William Jones in 1706, and it was popularized by Swiss mathematician Leonhard Euler.

3. **Approximations**: Common approximations include 22/7 and 3.14, though these are not exact. More precise approximations are used in calculations requiring higher accuracy.

4. **Applications**: Pi is fundamental in many areas of mathematics, including geometry (for calculating areas and volumes of circles and spheres), trigonometry, and calculus. It also appears in various formulas across physics, engineering, and statistics.

5. **Historical significance**: The concept of pi has been studied for thousands of years, with ancient civilizations like the Babylonians and Egyptians having approximations of it. Archimedes of Syracuse is credited with one of the first rigorous calculations of pi.

6. **Computational challenges**: Calculating pi to high precision has been a challenge and a benchmark for computational algorithms and supercomputers. As of 2023, pi has been computed to over 100 trillion digits.

Pi's ubiquity in mathematics and science makes it one of the most well-known and studied constants in all of mathematics.
````

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | none |  | Completion API user prompt. |
| `messages` | array of objects | No | none |  | Chat messages, as message objects (for example `{"role": "user", "content": "..."}`). Use instead of `prompt` for multi-turn chat. |
| `documents` | array of objects | No | none |  | Documents for the request. Passed to the chat template. |
| `tools` | array of objects | No | none |  | Tools for the request. Passed to the chat template. |
| `tool_choice` | string | No | none |  | Tool choice for the request. If the choice is a specific function, pass it as a JSON string. |
| `response_format` | object | No | none |  | An object specifying the format that the model must output. |
| `system_prompt` | string | No | none |  | Completion API system prompt. The chat template provides a good default. |
| `chat_template` | string | No | none |  | A template to format the prompt with. If not specified, the chat template provided by the model will be used. |
| `add_generation_prompt` | boolean | No | `true` |  | Add generation prompt. Passed to the chat template. |
| `chat_template_kwargs` | object | No | none |  | Additional arguments to be passed to the chat template. |
| `min_tokens` | integer | No | `0` |  | The minimum number of tokens the model should generate as output. |
| `max_tokens` | integer | No | none |  | Deprecated in favor of `max_completion_tokens`. |
| `max_completion_tokens` | integer | No | none |  | An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens. |
| `temperature` | number | No | none |  | The value used to modulate the next token probabilities. |
| `top_p` | number | No | none |  | A probability threshold for generating the output. If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). |
| `top_k` | integer | No | none |  | The number of highest probability tokens to consider for generating the output. If > 0, only keep the top k tokens with highest probability (top-k filtering). |
| `presence_penalty` | number | No | none |  | Presence penalty |
| `frequency_penalty` | number | No | none |  | Frequency penalty |
| `repetition_penalty` | number | No | none |  | Repetition penalty |
| `stop` | array of strings | No | none |  | A list of sequences to stop generation at. For example, `["<end>","<stop>"]` stops generation at the first instance of `<end>` or `<stop>`. |
| `seed` | integer | No | none |  | Random seed. Leave unspecified to randomize the seed. |
| `stream` | boolean | No | none |  | Request a streaming response. Results are read by polling GET `/predictions/{id}`, so leave this `false`. |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0032 per API unit.
- **1 API unit = 10,000 weighted tokens**; an output token counts 1, an input token counts 0.24.
- That is **$0.0768 per million input tokens** and **$0.32 per million output tokens**.
- Each request is rounded up to a whole API unit, so the smallest charge per request is $0.0032.
- **Example cost**: the example request used 12 input tokens and 355 output tokens. 12 x 0.24 + 355 = 357.88 weighted tokens, rounded up to 1 API unit = **$0.0032**.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/long-context-instruct-model-ibm-granite-granite-4-1-8b`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/long-context-instruct-model-ibm-granite-granite-4-1-8b/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "ibm-granite/granite-4.1-8b",
    "input": {
      "stop": [],
      "tools": [],
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "What is Pi?",
      "stream": false,
      "messages": [],
      "documents": [],
      "min_tokens": 0,
      "temperature": 0,
      "chat_template_kwargs": {},
      "add_generation_prompt": true
    }
  }'
```

Response (201 Created):
```json
{
  "id": "vyyjvntmv1rne0d1d0zr6tfd18",
  "version": "ibm-granite/granite-4.1-8b",
  "status": "starting",
  "created_at": "2026-10-23T10:12:23.664Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/long-context-instruct-model-ibm-granite-granite-4-1-8b/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "vyyjvntmv1rne0d1d0zr6tfd18",
  "status": "succeeded",
  "output": [
    "Pi (π) is a mathematical constant that represents the ratio of a circle's circumference to its diameter. It is an irrational number, meaning it cannot be expressed as a simple fraction and its decimal representation goes on forever without repeating. The value of pi is approximately 3.14159, but it has been calculated to trillions of decimal places for various computational purposes.\n\nKey points about pi:\n\n1. **Irrationality**: Pi cannot be exactly expressed as a fraction of two integers. Its decimal expansion is infinite and non-repeating.\n\n2. **Symbol**: The symbol π was first used to represent this constant by Welsh mathematician William Jones in 1706, and it was popularized by Swiss mathematician Leonhard Euler.\n\n3. **Approximations**: Common approximations include 22/7 and 3.14, though these are not exact. More precise approximations are used in calculations requiring higher accuracy.\n\n4. **Applications**: Pi is fundamental in many areas of mathematics, including geometry (for calculating areas and volumes of circles and spheres), trigonometry, and calculus. It also appears in various formulas across physics, engineering, and statistics.\n\n5. **Historical significance**: The concept of pi has been studied for thousands of years, with ancient civilizations like the Babylonians and Egyptians having approximations of it. Archimedes of Syracuse is credited with one of the first rigorous calculations of pi.\n\n6. **Computational challenges**: Calculating pi to high precision has been a challenge and a benchmark for computational algorithms and supercomputers. As of 2023, pi has been computed to over 100 trillion digits.\n\nPi's ubiquity in mathematics and science makes it one of the most well-known and studied constants in all of mathematics."
  ],
  "error": null,
  "created_at": "2026-10-23T10:12:23.664Z",
  "started_at": "2026-10-23T10:12:23.693Z",
  "completed_at": "2026-10-23T10:12:27.949Z"
}
```

`output` is an array of text chunks. Join them in order to get the full answer. The example above shows the chunks from this run already joined into one string.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example `messages` that is not an array.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/long-context-instruct-model-ibm-granite-granite-4-1-8b";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "ibm-granite/granite-4.1-8b", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output.join("");
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "stop": [],
    "tools": [],
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "What is Pi?",
    "stream": false,
    "messages": [],
    "documents": [],
    "min_tokens": 0,
    "temperature": 0,
    "chat_template_kwargs": {},
    "add_generation_prompt": true
  }).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/long-context-instruct-model-ibm-granite-granite-4-1-8b"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "ibm-granite/granite-4.1-8b", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return "".join(prediction["output"])

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "stop": [],
    "tools": [],
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "What is Pi?",
    "stream": False,
    "messages": [],
    "documents": [],
    "min_tokens": 0,
    "temperature": 0,
    "chat_template_kwargs": {},
    "add_generation_prompt": True
})
print(output)
```

## FAQ
- **Why does `output` come back as a list?**
  - The answer is returned as an array of text chunks. Join them in order (`output.join("")` in JavaScript, `"".join(output)` in Python).
- **Should I use `prompt` or `messages`?**
  - Use `prompt` (and optionally `system_prompt`) for a single question. Use `messages` to send a whole conversation as message objects with a `role` and `content`.
- **How do I limit the answer length?**
  - Set `max_completion_tokens`. `max_tokens` still exists but is deprecated in favor of `max_completion_tokens`.
- **How is a request billed?**
  - By weighted tokens at $0.0032 per 10,000: input tokens count 0.24 and output tokens count 1, rounded up to a whole unit per request. Most short requests cost $0.0032.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Run IBM Granite 4.1 8B, a long-context instruct model with chat messages, documents, tools and structured output. Pay as you go, no monthly fee.