Name: IBM Granite 3.1 2B Instruct Instruction-Following LLM API
Brand: magicapi
SKU: api-magicapi-instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct
Availability: InStock
Author: API Market

Shashank Agarwal

# 🧠 Instruction-Following LLM API (IBM Granite 3.1 2B Instruct)

## Overview
Granite-3.1-2B-Instruct is a lightweight, open-source language model from IBM with 2 billion parameters. IBM describes it as designed for instruction-following tasks such as summarization, problem-solving, text translation, reasoning and code tasks.

Send a `prompt` (and optionally a `system_prompt`). The model answers in text. The output arrives as a list of text chunks; join them in order to get the full answer.

## Features
- **Instruction following**: answer questions, summarize, rewrite, translate and explain from a plain-text prompt.
- **System prompt**: `system_prompt` (default "You are a helpful assistant.") is prepended to the prompt to guide behaviour.
- **Length control**: `max_tokens` (default 512) caps the answer; `min_tokens` (default 0) sets a minimum.
- **Sampling controls**: `temperature` (default 0.6), `top_p` (default 0.9), `top_k` (default 50), `presence_penalty` and `frequency_penalty` (default 0).
- **Stop sequences**: `stop_sequences` takes a comma-separated list such as `<end>,<stop>`.
- **Small and inexpensive**: billed on tokens, with input tokens weighted at 0.12 of an output token.

## Use Cases
1. **Short explanations and answers**: help text, FAQs and product copy.
2. **Summaries**: condense notes, tickets or articles into a few sentences.
3. **Text rewriting and translation**: change tone, length or language of a passage.
4. **Lightweight assistants**: chat features where a small, fast model is enough.

## Visual Examples
### Example Input
System prompt: "You are a helpful assistant that explains technology clearly and briefly."

Prompt: "Explain in three short sentences how a heat pump keeps a house warm in winter."

Settings: `max_tokens` `512`, `min_tokens` `0`, `temperature` `0.6`, `top_p` `0.9`, `top_k` `50`, `presence_penalty` `0`, `frequency_penalty` `0`.

### Example Output
The chunks joined together:

> A heat pump functions by extracting heat from the outside air, even in cold winter conditions, and transferring it indoors. It works as an air-source heat pump, using a refrigerant to absorb heat from the ambient air inside the outdoor unit, which is then compressed and released into the house, providing warmth. This process is more energy-efficient than traditional heating methods because it transfers heat rather than generating it.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | `""` |  | Prompt |
| `system_prompt` | string | No | `You are a helpful assistant.` |  | System prompt sent to the model. It is prepended to the prompt and helps guide system behaviour |
| `min_tokens` | integer | No | `0` |  | The minimum number of tokens the model should generate as output |
| `max_tokens` | integer | No | `512` |  | The maximum number of tokens the model should generate as output |
| `temperature` | number | No | `0.6` |  | The value used to modulate the next token probabilities |
| `top_p` | number | No | `0.9` |  | A probability threshold for generating the output. If below 1.0, only keep the top tokens with cumulative probability >= `top_p` (nucleus filtering) |
| `top_k` | integer | No | `50` |  | The number of highest probability tokens to consider for generating the output. If above 0, only keep the top k tokens with highest probability (top-k filtering) |
| `presence_penalty` | number | No | `0` |  | Presence penalty |
| `frequency_penalty` | number | No | `0` |  | Frequency penalty |
| `stop_sequences` | string | No | none |  | A comma-separated list of sequences to stop generation at. For example, `<end>,<stop>` stops generation at the first instance of `<end>` or `<stop>` |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0032 per API unit.
- **1 API unit = 10,000 weighted tokens; an output token counts 1, an input token counts 0.12.**
- That works out to **$0.0384 per million input tokens** and **$0.32 per million output tokens**.
- Each request rounds up to a whole unit, so every successful request costs at least $0.0032.
- **Example cost**: the example request used 35 input tokens and 95 output tokens: 35 x 0.12 + 95 x 1 = 99.2 weighted tokens, which rounds up to 1 unit, so $0.0032.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "7b0098e3d43b2c13c23a78ee680b1797e1149b8b0ea492dd8b2b3ccc78f8a94b",
    "input": {
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "Explain in three short sentences how a heat pump keeps a house warm in winter.",
      "max_tokens": 512,
      "min_tokens": 0,
      "temperature": 0.6,
      "system_prompt": "You are a helpful assistant that explains technology clearly and briefly.",
      "presence_penalty": 0,
      "frequency_penalty": 0
    }
  }'
```

Response (201 Created):
```json
{
  "id": "vffk22ekqxrmw0d1f1g859q5bc",
  "version": "7b0098e3d43b2c13c23a78ee680b1797e1149b8b0ea492dd8b2b3ccc78f8a94b",
  "status": "starting",
  "created_at": "2026-10-26T13:22:54.157Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK, `output` shortened):
```json
{
  "id": "vffk22ekqxrmw0d1f1g859q5bc",
  "status": "succeeded",
  "output": ["", "A", " heat", " pump", " functions", " by", " extracting", " heat", " from", " the", " outside", " air", ",", " even", " in", " cold", " w", "inter", " conditions", "...", " rather", " than", " generating", " it", ".", ""],
  "error": null,
  "created_at": "2026-10-26T13:22:54.157Z",
  "started_at": "2026-10-26T13:24:21.237Z",
  "completed_at": "2026-10-26T13:24:23.669Z"
}
```

`output` is an array of text chunks (in the example, 95 of them). Join them in order, with no separator, to get the full answer shown under Example Output.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `max_tokens` that is not an integer.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
Requests are rate limited to keep usage fair. If you receive a 429 response, wait and retry with exponential backoff.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "7b0098e3d43b2c13c23a78ee680b1797e1149b8b0ea492dd8b2b3ccc78f8a94b", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "Explain in three short sentences how a heat pump keeps a house warm in winter.",
  "max_tokens": 512,
  "min_tokens": 0,
  "temperature": 0.6,
  "system_prompt": "You are a helpful assistant that explains technology clearly and briefly.",
  "presence_penalty": 0,
  "frequency_penalty": 0
}).then((chunks) => console.log(chunks.join("")));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/instruction-following-ai-api-ibm-granite-granite-3-1-2b-instruct"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "7b0098e3d43b2c13c23a78ee680b1797e1149b8b0ea492dd8b2b3ccc78f8a94b", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


chunks = create_and_wait_for_prediction({
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "Explain in three short sentences how a heat pump keeps a house warm in winter.",
    "max_tokens": 512,
    "min_tokens": 0,
    "temperature": 0.6,
    "system_prompt": "You are a helpful assistant that explains technology clearly and briefly.",
    "presence_penalty": 0,
    "frequency_penalty": 0
})
print("".join(chunks))
```

## FAQ
- **Why is the output a list instead of one string?**
  - The answer is produced as a sequence of text chunks. Join them in order with no separator (`chunks.join("")` in JavaScript, `"".join(chunks)` in Python).
- **How long can an answer be?**
  - Up to `max_tokens` tokens (default 512). Raise it for longer answers; output tokens are what most of the price is made of.
- **Can I send a multi-turn chat or call tools?**
  - This API takes one `prompt` and one `system_prompt` per request and has no tools parameter. To continue a conversation, include the earlier turns in the prompt text yourself.
- **What does a request cost?**
  - Input tokens count 0.12 and output tokens count 1 toward 10,000-token units of $0.0032, rounded up per request: $0.0384 per million input tokens and $0.32 per million output tokens, with a minimum of $0.0032 per request.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.


Run IBM Granite 3.1 2B Instruct for summarization, translation, reasoning and code. $0.0384/M input and $0.32/M output tokens, pay as you go.