Name: Prompt Classifier API: Text-to-Image Prompt Toxicity Check
Brand: magicapi
SKU: api-magicapi-text-to-image-toxicity-checker-fofr-prompt-classifier
Availability: InStock
Author: API Market

Shashank Agarwal

# 🛡️ Prompt Toxicity Classifier API (fofr Prompt Classifier)

## Overview
Prompt Classifier is a fine-tune of the Llama 13B language model, published by fofr, that rates how toxic a text-to-image prompt is. It returns a safety ranking between 0 (safe) and 10 (toxic).

You wrap the prompt you want to check in a fixed template and the model completes it with a number:

```
[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]
```

Set `stop_sequences` to `[/SAFETY_RANKING]` so generation stops right after the number. The output is a list of text chunks; join them and trim the whitespace to read the ranking.

## Features
- **Safety ranking from 0 to 10**: 0 means safe and 10 means toxic.
- **Built for text-to-image prompts**: screen prompts before you send them to an image model.
- **Fast**: the example request took 0.27 seconds of processing.
- **Generation controls**: `max_new_tokens`, `min_new_tokens`, `temperature`, `top_p`, `top_k`, `stop_sequences` and `seed`.

## Use Cases
1. **Prompt moderation**: block or flag image prompts above a ranking you choose before generating an image.
2. **Routing**: send low-ranked prompts straight through and higher-ranked prompts to manual review.
3. **Logging and analytics**: record a ranking for every prompt your users submit.

## Visual Examples
### Example Input
```json
{
  "debug": false,
  "top_k": 50,
  "top_p": 0.9,
  "prompt": "[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]",
  "temperature": 0.75,
  "max_new_tokens": 128,
  "min_new_tokens": -1,
  "stop_sequences": "[/SAFETY_RANKING]"
}
```

### Example Output
The model returned the chunks `[" ", "0", " "]`. Joined and trimmed, the result is:

```
0
```

A ranking of 0 means the prompt "a photo of a cat" was rated safe.

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | Yes |  |  | Prompt to send to the model, in the form `[PROMPT] your prompt [/PROMPT] [SAFETY_RANKING]` |
| `max_new_tokens` | integer | No | `128` | minimum 1 | Maximum number of tokens to generate. A word is generally 2-3 tokens |
| `min_new_tokens` | integer | No | `-1` | minimum -1 | Minimum number of tokens to generate. To disable, set to -1. A word is generally 2-3 tokens |
| `temperature` | number | No | `0.75` | 0.01 to 5 | Adjusts randomness of outputs, greater than 1 is random and 0 is deterministic, 0.75 is a good starting value |
| `top_p` | number | No | `0.9` | 0 to 1 | When decoding text, samples from the top p percentage of most likely tokens; lower to ignore less likely tokens |
| `top_k` | integer | No | `50` | minimum 0 | When decoding text, samples from the top k most likely tokens; lower to ignore less likely tokens |
| `stop_sequences` | string | No | none |  | A comma-separated list of sequences to stop generation at. For example, '<end>,<stop>' will stop generation at the first instance of '<end>' or '<stop>' |
| `seed` | integer | No | none |  | Random seed. Leave blank to randomize the seed |
| `debug` | boolean | No | `false` |  | Provide debugging output in logs |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.0022 per API unit.
- **1 API unit = 1 successful request with up to 2 seconds of processing.** A request that takes longer is billed one more unit for each started block of 2 seconds (for example, 3 seconds of processing is 2 units, $0.0044).
- Keep `stop_sequences` set to `[/SAFETY_RANKING]` so the model stops after the ranking and the request stays short.
- **Example cost**: the example request above took 0.27 seconds of processing, so it used 1 API unit and cost $0.0022.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/text-to-image-toxicity-checker-fofr-prompt-classifier`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/text-to-image-toxicity-checker-fofr-prompt-classifier/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "1ffac777bf2c1f5a4a5073faf389fefe59a8b9326b3ca329e9d576cce658a00f",
    "input": {
      "debug": false,
      "top_k": 50,
      "top_p": 0.9,
      "prompt": "[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]",
      "temperature": 0.75,
      "max_new_tokens": 128,
      "min_new_tokens": -1,
      "stop_sequences": "[/SAFETY_RANKING]"
    }
  }'
```

Response (201 Created):
```json
{
  "id": "5wsqa8retxrmy0d1ccq8rm448c",
  "version": "1ffac777bf2c1f5a4a5073faf389fefe59a8b9326b3ca329e9d576cce658a00f",
  "status": "starting",
  "created_at": "2026-10-22T10:35:25.527Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/text-to-image-toxicity-checker-fofr-prompt-classifier/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "5wsqa8retxrmy0d1ccq8rm448c",
  "status": "succeeded",
  "output": [" ", "0", " "],
  "error": null,
  "created_at": "2026-10-22T10:35:25.527Z",
  "started_at": "2026-10-22T10:35:25.535Z",
  "completed_at": "2026-10-22T10:35:25.808Z"
}
```

`output` is a list of text chunks. Join them into one string and trim the whitespace to get the ranking, here `0`.

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a missing `prompt` or a `temperature` outside 0.01 to 5.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- If the joined output is not a number from 0 to 10, check that the prompt follows the `[PROMPT] ... [/PROMPT] [SAFETY_RANKING]` template and that `stop_sequences` is `[/SAFETY_RANKING]`.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/text-to-image-toxicity-checker-fofr-prompt-classifier";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "1ffac777bf2c1f5a4a5073faf389fefe59a8b9326b3ca329e9d576cce658a00f", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output.join("").trim();
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
  debug: false,
  top_k: 50,
  top_p: 0.9,
  prompt: "[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]",
  temperature: 0.75,
  max_new_tokens: 128,
  min_new_tokens: -1,
  stop_sequences: "[/SAFETY_RANKING]"
}).then((ranking) => console.log(`Safety ranking: ${ranking}`));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/text-to-image-toxicity-checker-fofr-prompt-classifier"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "1ffac777bf2c1f5a4a5073faf389fefe59a8b9326b3ca329e9d576cce658a00f", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return "".join(prediction["output"]).strip()

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


ranking = create_and_wait_for_prediction({
    "debug": False,
    "top_k": 50,
    "top_p": 0.9,
    "prompt": "[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]",
    "temperature": 0.75,
    "max_new_tokens": 128,
    "min_new_tokens": -1,
    "stop_sequences": "[/SAFETY_RANKING]"
})
print(f"Safety ranking: {ranking}")
```

## FAQ
- **How do I format the prompt I want to check?**
  - Put it between `[PROMPT]` and `[/PROMPT]` and end with `[SAFETY_RANKING]`, for example `[PROMPT] a photo of a cat [/PROMPT] [SAFETY_RANKING]`.
- **What do the numbers mean?**
  - The ranking runs from 0 (safe) to 10 (toxic). Pick the threshold that fits your product, for example review anything above 5.
- **Why is the output a list?**
  - The model returns its text in chunks. Join the list into one string and trim the whitespace to read the number.
- **Do I need to send all the generation settings?**
  - Only `prompt` is required. Setting `stop_sequences` to `[/SAFETY_RANKING]` keeps the answer to the number alone.
- **How is a request billed?**
  - $0.0022 per request for up to 2 seconds of processing, plus one more unit for each further started block of 2 seconds. The example took 0.27 seconds and cost $0.0022.
- **Is the 201 response from POST an error?**
  - No. 201 Created means the prediction was accepted. Use the returned `id` to poll GET `/predictions/{id}` until `status` is `succeeded`, `failed` or `canceled`.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Screen text-to-image prompts before generation: a Llama 13B fine-tune returns a safety ranking from 0 (safe) to 10 (toxic). $0.0022 per request, pay as you go.