Name: Llama 2 13B Chat API: Open Source Chat LLM by Meta
Brand: magicapi
SKU: api-magicapi-ai-chat-completion-api-meta-llama-2-13b-chat
Availability: InStock
Author: API Market

Shashank Agarwal

# 💬 Chat LLM API (Meta Llama 2 13B Chat)

## Overview
Llama 2 13B Chat is a 13-billion-parameter language model from Meta, fine-tuned for chat completions.

Send your message in `prompt` and, optionally, instructions in `system_prompt`. Both are placed into the Llama 2 chat format by `prompt_template`, so plain text works. `max_tokens` caps the length of the reply, and `temperature`, `top_p`, `top_k`, `stop_sequences` and the penalties control the output. The reply comes back as a list of text chunks that you join.

## Features
- **Chat-tuned replies**: the model follows instructions and answers questions.
- **System prompt**: `system_prompt` sets the assistant's role and rules. The default is a helpful, honest and safe assistant prompt.
- **Length controls**: `max_tokens` (default 512) and `min_tokens`; `stop_sequences` ends generation at the strings you list.
- **Sampling controls**: `temperature` 0 to 5 (default 0.7), `top_p` 0 to 1 (default 0.95) and `top_k` (default 0).
- **Repetition and length penalties**: `presence_penalty` and `length_penalty` (0 to 5, default 1).
- **Custom prompt template**: `prompt_template` wraps your prompt; it must contain `{prompt}`.
- **Repeatable runs**: set `seed` to reproduce a result.

## Use Cases
1. **Assistants and chatbots**: answer user questions under your own system prompt.
2. **Creative writing**: stories, poems and scripts from a short brief.
3. **Drafting and rewriting**: emails, summaries and product descriptions.
4. **Explanations**: plain-language answers to general questions.

## Visual Examples
### Example Input
Prompt: "Write a short, friendly welcome message for new members of a weekend hiking club, with three tips for their first hike."

Settings: the default `system_prompt` (sent explicitly), `max_new_tokens` 500 (the older name for `max_tokens`), `temperature` 0.75, `top_p` 1.

### Example Output
The chunks of this run joined into one reply:

```text
 Hello and welcome to our weekend hiking club! We're so excited to have you join us on our adventures. As a helpful, respectful, and honest assistant, I'd like to offer three tips for your first hike with us:

1. Make sure to bring plenty of water and snacks to keep you hydrated and energized throughout your hike. It's also a good idea to pack a first aid kit and any necessary medications.
2. Wear comfortable, breathable clothing and sturdy hiking shoes to ensure you're prepared for the terrain and any unexpected obstacles. Don't forget to bring a hat, sunglasses, and sunscreen to protect yourself from the elements.
3. Take your time and enjoy the scenery! Hiking is all about enjoying nature and the company of your fellow hikers, so be sure to pause and take in the views, and don't be afraid to ask questions or seek help if you need it.

Remember, safety always comes first, so be sure to check the weather forecast and trail conditions before we head out, and let's all stay together and keep an eye out for each other during our hike. We can't wait to explore the great outdoors together!

I hope these tips are helpful, and I'm here if you have any other questions. Let's have a great time on our first hike together!
```

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | Yes |  |  | Prompt to send to the model |
| `system_prompt` | string | No | the Llama 2 default assistant prompt (shown below the table) |  | System prompt inserted into the prompt template to guide the model's behavior |
| `max_tokens` | integer | No | `512` | 1 or more | Maximum number of tokens to generate. A word is generally 2 to 3 tokens. |
| `min_tokens` | integer | No | none | -1 or more | Minimum number of tokens to generate. Set to -1 to disable. A word is generally 2 to 3 tokens. |
| `temperature` | number | No | `0.7` | 0 to 5 | Randomness of the output: 0 is deterministic, above 1 is very random; 0.75 is a good starting value |
| `top_p` | number | No | `0.95` | 0 to 1 | Samples from the top p share of most likely tokens; lower it to ignore less likely tokens |
| `top_k` | integer | No | `0` | -1 or more | Samples from the top k most likely tokens; lower it to ignore less likely tokens |
| `stop_sequences` | string | No | none |  | Comma-separated list of sequences to stop generation at, for example '<end>,<stop>' |
| `length_penalty` | number | No | `1` | 0 to 5 | Controls output length: below 1 favors shorter outputs, above 1 favors longer outputs |
| `presence_penalty` | number | No | `0` |  | Penalizes repeated tokens regardless of how often they appear; higher values make repetition less likely |
| `seed` | integer | No | none |  | Random seed. Leave blank to randomize. |
| `prompt_template` | string | No | `<s>[INST] <<SYS>> {system_prompt} <</SYS>> {prompt} [/INST]` (with line breaks) |  | Template for formatting the prompt. Any string that contains the substring {prompt}. |
| `log_performance_metrics` | boolean | No | `false` |  | Log performance metrics for the request |
| `max_new_tokens` | integer | No | none | 1 or more | Older name for max_tokens, kept for backwards compatibility. Use max_tokens; do not send both. |
| `min_new_tokens` | integer | No | none | -1 or more | Older name for min_tokens, kept for backwards compatibility. Use min_tokens; do not send both. |

Default `system_prompt`:

```text
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
```

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.001 per API unit.
- **1 API unit = 1,000 weighted tokens**; an output token counts 1, an input token counts 0.2.
- That is **$0.2 per million input tokens** and **$1 per million output tokens**.
- Each request is rounded up to a whole API unit, so the smallest charge per request is $0.001.
- **Example cost**: the request above used 29 input tokens and 318 output tokens. 29 x 0.2 + 318 = 323.8 weighted tokens, rounded up to 1 API unit = **$0.001**.
- The system prompt counts as input tokens, so a long system prompt costs more per request.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/ai-chat-completion-api-meta-llama-2-13b-chat`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/ai-chat-completion-api-meta-llama-2-13b-chat/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "meta/llama-2-13b-chat",
    "input": {
      "top_p": 1,
      "prompt": "Write a short, friendly welcome message for new members of a weekend hiking club, with three tips for their first hike.",
      "temperature": 0.75,
      "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don'\''t know the answer to a question, please don'\''t share false information.",
      "max_new_tokens": 500
    }
  }'
```

Response (201 Created):
```json
{
  "id": "pfbymyg7q9rne0d1d23vszdtj0",
  "version": "meta/llama-2-13b-chat",
  "status": "starting",
  "created_at": "2026-10-23T11:30:42.662Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/ai-chat-completion-api-meta-llama-2-13b-chat/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "pfbymyg7q9rne0d1d23vszdtj0",
  "status": "succeeded",
  "output": [
    " Hello and welcome to our weekend hiking club! We're so excited to have you join us on our adventures. As a helpful, respectful, and honest assistant, I'd like to offer three tips for your first hike with us:\n\n1. Make sure to bring plenty of water and snacks to keep you hydrated and energized throughout your hike. It's also a good idea to pack a first aid kit and any necessary medications.\n2. Wear comfortable, breathable clothing and sturdy hiking shoes to ensure you're prepared for the terrain and any unexpected obstacles. Don't forget to bring a hat, sunglasses, and sunscreen to protect yourself from the elements.\n3. Take your time and enjoy the scenery! Hiking is all about enjoying nature and the company of your fellow hikers, so be sure to pause and take in the views, and don't be afraid to ask questions or seek help if you need it.\n\nRemember, safety always comes first, so be sure to check the weather forecast and trail conditions before we head out, and let's all stay together and keep an eye out for each other during our hike. We can't wait to explore the great outdoors together!\n\nI hope these tips are helpful, and I'm here if you have any other questions. Let's have a great time on our first hike together!"
  ],
  "error": null,
  "created_at": "2026-10-23T11:30:42.662Z",
  "started_at": "2026-10-23T11:32:56.515Z",
  "completed_at": "2026-10-23T11:33:00.513Z"
}
```

`output` is an array of text chunks. Join them in order to get the full reply. The example above shows the chunks from this run already joined into one string.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `prompt` or a `temperature` above 5.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/ai-chat-completion-api-meta-llama-2-13b-chat";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "meta/llama-2-13b-chat", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "top_p": 1,
    "prompt": "Write a short, friendly welcome message for new members of a weekend hiking club, with three tips for their first hike.",
    "temperature": 0.75,
    "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
    "max_new_tokens": 500
  }).then((output) => console.log(output.join("")));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/ai-chat-completion-api-meta-llama-2-13b-chat"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "meta/llama-2-13b-chat", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print("".join(create_and_wait_for_prediction({
    "top_p": 1,
    "prompt": "Write a short, friendly welcome message for new members of a weekend hiking club, with three tips for their first hike.",
    "temperature": 0.75,
    "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
    "max_new_tokens": 500
})))
```

## FAQ
- **How do I change the assistant's behavior?**
  - Send your own `system_prompt`. It replaces the default prompt shown above.
- **Why did the reply stop mid-sentence?**
  - It reached the token limit. Raise `max_tokens` (or the older `max_new_tokens`) or ask for a shorter answer.
- **Should I use `max_tokens` or `max_new_tokens`?**
  - Use `max_tokens`. `max_new_tokens` is the older name and still works, but do not send both.
- **How much does a typical request cost?**
  - Each request is billed in units of 1,000 weighted tokens at $0.001 per unit. The example (29 input and 318 output tokens) cost $0.001.
- **How do I read the output?**
  - `output` is a list of text chunks. Join them in order, for example `output.join("")` in JavaScript or `"".join(output)` in Python.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Get chat replies from Meta's Llama 2 13B Chat with a custom system prompt, stop sequences and sampling controls. Pay as you go, no monthly fee.