Name: Llama 2 7B Chat API: Chat Completion by Meta
Brand: magicapi
SKU: api-magicapi-chat-completions-api-meta-llama-2-7b-chat
Availability: InStock
Author: API Market

Shashank Agarwal

# 💬 Chat Completion API (Meta Llama 2 7B Chat)

## Overview
Llama 2 7B Chat is a 7-billion-parameter language model from Meta, fine-tuned for chat completions.

Send a user message in `prompt` and, optionally, a `system_prompt` that sets the assistant's behavior. The model answers as an assistant and returns the reply as a list of text chunks that you join in order. If you leave `system_prompt` out, a default safety-focused system prompt is used.

## Features
- **Chat replies**: send a user message in `prompt` and get an assistant answer.
- **System prompt**: `system_prompt` sets the assistant's role and rules. The default is a long safety-focused prompt (shown below the parameter table).
- **Prompt template**: `prompt_template` places `{system_prompt}` and `{prompt}` in the model's chat format. The default already uses the Llama 2 chat format, so most requests do not need to change it.
- **Length control**: `max_tokens` (default 512) caps the reply and `min_tokens` sets a minimum. `length_penalty` (0 to 5, default 1) nudges replies shorter or longer.
- **Sampling controls**: `temperature` (0 to 5, default 0.7), `top_p` (0 to 1, default 0.95) and `top_k` (default 0).
- **Stop sequences and repetition**: `stop_sequences` ends the reply at a string you choose; `presence_penalty` makes repeated tokens less likely.
- **Reproducible runs**: set `seed` to repeat a result.

## Use Cases
1. **Assistants and FAQ bots**: answer user questions in a role you set with `system_prompt`.
2. **Advice and how-to answers**: short explanations and suggestions, as in the example.
3. **Low-cost chat features**: add chat to an app at a low price per token.
4. **Drafting replies**: draft answers to customer messages for a person to review.

## Examples
### Example Input
A how-to question with the default system prompt sent explicitly, `max_new_tokens` 800 and `temperature` 0.75.

```json
{
  "top_p": 1,
  "prompt": "Tell me how to tailor a men's suit so I look fashionable.",
  "temperature": 0.75,
  "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
  "max_new_tokens": 800
}
```

### Example Output
An excerpt from the middle of the joined reply (the full reply was 396 tokens):

~~~text
... When it comes to styling a suit for a fashionable look, there are a few key things to keep in mind. Firstly, it's important to consider the fit of the suit. A well-fitting suit should skim your body without being too tight or too loose. Look for a suit that is tailored to your body shape, with a clean line from shoulder to hem.

Next, consider the fabric of the suit. A high-quality suit will be made from a luxurious fabric that drapes well and moves with the body. Some popular fabrics for suits include wool, cotton, and linen. ...
~~~

With the default safety-focused system prompt the reply opens with a long caveat before the advice; a more specific `system_prompt` gives more direct answers.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | Yes |  |  | User message to send to the model |
| `system_prompt` | string | No | default safety prompt (see below) |  | System prompt inserted into `prompt_template`; it sets the assistant's behavior. The default is shown below the table |
| `max_tokens` | integer | No | `512` | `1` or more | Maximum number of tokens to generate. A word is generally 2 to 3 tokens |
| `min_tokens` | integer | No | none | `-1` or more | Minimum number of tokens to generate. Set it to -1 to disable |
| `temperature` | number | No | `0.7` | `0` to `5` | Randomness of the output: 0 is deterministic and values above 1 are very random. 0.75 is a good starting value |
| `top_p` | number | No | `0.95` | `0` to `1` | Samples only from the most likely tokens that together make up the top p probability; lower it to ignore less likely tokens |
| `top_k` | integer | No | `0` | `-1` or more | Samples only from the k most likely tokens; lower it to ignore less likely tokens |
| `stop_sequences` | string | No | none |  | Comma-separated list of sequences that stop generation, for example `<end>,<stop>` stops at the first `<end>` or `<stop>` |
| `length_penalty` | number | No | `1` | `0` to `5` | Controls output length: below 1 the model tends to write shorter outputs, above 1 longer ones |
| `presence_penalty` | number | No | `0` |  | Penalizes tokens that already appeared, however often; higher values make repetition less likely |
| `seed` | integer | No | none |  | Random seed. Leave it out to use a random seed |
| `prompt_template` | string | No | `"<s>[INST] <<SYS>>\n{system_prompt}\n<</SYS>>\n\n{prompt} [/INST]"` |  | Template for formatting the prompt. Any string that contains `{prompt}`; the default also places `{system_prompt}` |
| `log_performance_metrics` | boolean | No | `false` |  | Debug option that logs performance metrics; it does not change the output |
| `max_new_tokens` | integer | No | none | `1` or more | Older name for `max_tokens`, kept for backwards compatibility. Use `max_tokens`; do not send both |
| `min_new_tokens` | integer | No | none | `-1` or more | Older name for `min_tokens`, kept for backwards compatibility. Use `min_tokens`; do not send both |

Default `system_prompt`:

~~~text
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
~~~

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0032 per API unit.
- **1 API unit = 10,000 weighted tokens**; an output token counts 1, an input token counts 0.2.
- That is **$0.064 per million input tokens** and **$0.32 per million output tokens**.
- Each request is rounded up to a whole API unit, so the smallest charge for a request is $0.0032.
- **Example cost**: the example request used 156 input tokens and 396 output tokens: 156 x 0.2 + 396 x 1 = 427.2 weighted tokens, rounded up to 1 API unit = **$0.0032**.
- Output tokens make up most of the cost, so `max_tokens` is the main way to cap the price of a request. A long `system_prompt` adds input tokens to every request.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/chat-completions-api-meta-llama-2-7b-chat`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/chat-completions-api-meta-llama-2-7b-chat/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "meta/llama-2-7b-chat",
    "input": {
      "top_p": 1,
      "prompt": "Tell me how to tailor a men'\''s suit so I look fashionable.",
      "temperature": 0.75,
      "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don'\''t know the answer to a question, please don'\''t share false information.",
      "max_new_tokens": 800
    }
  }'
```

Response (201 Created):
```json
{
  "id": "1vbb536ybxrna0d1d20vdc86h0",
  "version": "meta/llama-2-7b-chat",
  "status": "starting",
  "created_at": "2026-10-23T11:25:04.155Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/chat-completions-api-meta-llama-2-7b-chat/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "1vbb536ybxrna0d1d20vdc86h0",
  "status": "succeeded",
  "output": ["When", " it", " comes", " to", " sty", "ling", " a", " suit", " for", " a", " fashion", "able", " look", ",", " there", " are", " a", " few", " key", " things", " to", " keep", " in", " mind", ".", " First", "ly", ",", " it", "'", "s", " important", " to", " consider", " the", " fit", " of", " the", " suit", ".", " A", " well", "-", "f", "itting", " suit", " should", " sk", "im", " your", " body", " without", " being", " too", " tight", " or", " too", " loose", ".", " Look", " for", " a", " suit", " that", " is", " tail", "ored", " to", " your", " body", " shape", ",", " with", " a", " clean", " line", " from", " shoulder", " to", " hem", ".", "\n", "\n", "Next", ",", " consider", " the", " fabric", " of", " the", " suit", ".", " A", " high", "-", "quality", " suit", " will", " be", " made", " from", " a", " lux", "ur", "ious", " fabric", " that", " d", "rap", "es", " well", " and", " moves", " with", " the", " body", ".", " Some", " popular", " fab", "rics", " for", " su", "its", " include", " w", "ool", ",", " cot", "ton", ",", " and", " lin", "en", "."],
  "error": null,
  "created_at": "2026-10-23T11:25:04.155Z",
  "started_at": "2026-10-23T11:27:06.503Z",
  "completed_at": "2026-10-23T11:27:09.041Z"
}
```

`output` is a list of text chunks in the order the model produced them (only the chunks of the excerpt are shown above). Join them with no separator to get the full reply.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `prompt`, a `temperature` above 5 or both `max_tokens` and `max_new_tokens` in one request.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/chat-completions-api-meta-llama-2-7b-chat";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "meta/llama-2-7b-chat", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  "top_p": 1,
  "prompt": "Tell me how to tailor a men's suit so I look fashionable.",
  "temperature": 0.75,
  "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
  "max_new_tokens": 800
}).then((output) => console.log(output.join("")));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/chat-completions-api-meta-llama-2-7b-chat"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "meta/llama-2-7b-chat", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


output = create_and_wait_for_prediction({
    "top_p": 1,
    "prompt": "Tell me how to tailor a men's suit so I look fashionable.",
    "temperature": 0.75,
    "system_prompt": "You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.",
    "max_new_tokens": 800
})
print("".join(output))
```

## FAQ
- **How do I get the full reply as one string?**
  - `output` is a list of text chunks. Join them in order with no separator: `output.join("")` in JavaScript or `"".join(output)` in Python.
- **Why are the answers so cautious?**
  - The default `system_prompt` asks the model to be careful and safe. Send your own `system_prompt` that describes the assistant you want to get more direct answers.
- **Can I send a multi-turn conversation?**
  - There is no messages field. Put the earlier turns into `prompt` yourself, or write your own `prompt_template`, which must contain `{prompt}`.
- **Should I use `max_tokens` or `max_new_tokens`?**
  - Use `max_tokens`. `max_new_tokens` and `min_new_tokens` are older names kept for backwards compatibility; do not send both forms in one request.
- **How much does a request cost?**
  - $0.064 per million input tokens and $0.32 per million output tokens, billed in units of $0.0032 and rounded up per request. The example cost $0.0032.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Generate assistant replies with Meta's Llama 2 7B Chat, a 7-billion-parameter model fine-tuned for chat. Pay as you go, no monthly fee.