Name: GPU Chat API: Simple, Powerful LLM Completions & Prompts
Brand: quickpod
SKU: api-quickpod-aiquick-chat
Availability: InStock
Author: API Market

Anil Singh

# GPU Chat API

## Overview
The GPU Chat API provides a streamlined interface for interacting with a remote GPU chat cluster, offering access to various large language models. Designed for rapid integration and compatibility, it features both a simple prompt endpoint for quick text generation and an OpenAI-style chat completions endpoint for more complex conversational AI workflows. This API simplifies model selection and interaction, making it ideal for building intelligent applications that leverage powerful chat models. It acts as a robust wrapper, powered by an upstream chat worker and pre-configured with a capable default model, ensuring high availability and performance for your AI initiatives.

## Features
*   **Simple Prompt Interface:** Easily send a single system prompt and a user prompt for quick text generation tasks.
*   **OpenAI-Style Chat Completions:** Interact with chat models using a familiar message-based format, compatible with OpenAI's API specification.
*   **Dynamic Model Listing:** Retrieve a directory of available chat models, including their IDs, ownership, and context window lengths (`max_model_len`).
*   **Model Metadata Access:** Inspect model properties to compare capabilities and choose the most suitable model for your application.
*   **GPU-Powered Performance:** Leverage a remote GPU cluster for efficient and scalable chat model inference.
*   **Health Endpoints:** Monitor service and upstream chat cluster health to ensure operational continuity.

## Use Cases
*   **Intelligent Virtual Assistants:** Develop AI chatbots for customer support, lead qualification, or internal knowledge bases by integrating with the OpenAI-style chat completions endpoint.
*   **Content Generation & Summarization:** Automate the creation of articles, marketing copy, social media posts, or summarize lengthy documents using the simple prompt interface for various industries like media, e-commerce, and publishing.
*   **Interactive Educational Platforms:** Build dynamic learning tools that provide personalized feedback, answer student questions, and generate practice materials in real-time.
*   **Developer Tooling:** Create model pickers, admin dashboards, or intelligent routing layers within your applications by leveraging the `List available chat models` endpoint to inspect and manage accessible models.
*   **Automated Data Extraction & Structuring:** Design workflows to extract specific information from unstructured text and format it into structured JSON objects, useful for data analysis in finance, healthcare, and research.


## API Documentation

### Base URL
All API requests should be made to: `https://prod.api.market/api/v1/quickpod/aiquick-chat`

### Authentication
Authentication is handled via API key. Include your API key in the request header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```


### Endpoints

#### Service health
**GET /api/v1/quickpod/aiquick-chat/health**

Checks the health status of the service.

**Request Example:**
```bash
curl -X GET "https://prod.api.market/api/v1/quickpod/aiquick-chat/health" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

**Response Example:**
```json
{
  "ok": true,
  "service": "quickpod-aiquick-chat",
  "default_model": "Qwen/Qwen3-0.6B",
  "max_inflight": 10
}
```

#### Upstream chat health via model listing
**GET /api/v1/quickpod/aiquick-chat/chat/health**

Checks the health of the upstream chat service by attempting to list models.

**Request Example:**
```bash
curl -X GET "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/health" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

**Response Example:**
```json
{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen3-0.6B",
      "object": "model",
      "owned_by": "vllm",
      "max_model_len": 4096
    }
  ]
}
```

#### List available chat models
**GET /api/v1/quickpod/aiquick-chat/chat/models**

Retrieves a list of all chat models available to your account, along with their metadata. This is useful for building model pickers or routing logic.

**Request Example:**
```bash
curl -X GET "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/models" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

**Response Example:**
```json
{
  "object": "list",
  "data": [
    {
      "id": "Qwen/Qwen3-0.6B",
      "object": "model",
      "owned_by": "vllm",
      "max_model_len": 4096
    }
  ]
}
```

#### Send a simple prompt
**POST /api/v1/quickpod/aiquick-chat/chat/simple**

Offers a simplified chat interface for interacting with a chat model using one system prompt and one user prompt.

**Request Body Example (Short greeting):**
```json
{
  "model": "Qwen/Qwen3-0.6B",
  "system_prompt": "You are a helpful assistant.",
  "prompt": "Write a short hello message.",
  "temperature": 0.7,
  "max_tokens": 64
}
```

**Request Example (Short greeting):**
```bash
curl -X POST "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/simple" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "system_prompt": "You are a helpful assistant.",
    "prompt": "Write a short hello message.",
    "temperature": 0.7,
    "max_tokens": 64
  }'
```

**Response Example (Short greeting):**
```json
{
  "model": "Qwen/Qwen3-0.6B",
  "content": "Hello! How can I help?",
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 9,
    "total_tokens": 33
  }
}
```

#### OpenAI-style chat completions
**POST /api/v1/quickpod/aiquick-chat/chat/completions**

Provides a compatible wrapper around the upstream OpenAI-style chat completions endpoint, allowing for multi-turn conversations and structured message inputs.

**Request Body Example (Basic chat completion):**
```json
{
  "model": "Qwen/Qwen3-0.6B",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Write a short hello message."
    }
  ],
  "temperature": 0.7,
  "max_tokens": 128
}
```

**Request Example (Basic chat completion):**
```bash
curl -X POST "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Write a short hello message."
      }
    ],
    "temperature": 0.7,
    "max_tokens": 128
  }'
```

**Response Example (Basic chat completion):**
```json
{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "created": 1784670715,
  "model": "Qwen/Qwen3-0.6B",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 27,
    "completion_tokens": 10,
    "total_tokens": 37
  }
}
```

## Status Codes
*   **200 OK**: The request was successful, and the response body contains the requested data or the result of the operation.
*   **201 Created**: (Not explicitly used by current endpoints, but generally indicates successful resource creation).
*   **400 Bad Request**: The request could not be understood or was missing required parameters. Check your request body, query parameters, and headers for correctness.
*   **401 Unauthorized**: Authentication failed because of a missing or invalid API key. Ensure `x-api-market-key` header is present and contains a valid API key.
*   **403 Forbidden**: (Not explicitly used by current endpoints, but generally indicates valid API key with insufficient permissions).
*   **404 Not Found**: The requested resource could not be found. Check the endpoint path and any identifiers in the URL.
*   **422 Unprocessable Entity**: The request was well-formed but could not be processed due to semantic errors, often related to validation failures (e.g., incorrect data types or out-of-range values in the request body).
*   **429 Too Many Requests**: You have exceeded the API's rate limits. Slow down your requests and implement a retry mechanism with exponential backoff.
*   **500 Internal Server Error**: An unexpected error occurred on the API server. This is typically a temporary issue; retrying after a short delay might resolve it.
*   **503 Service Unavailable**: The service is temporarily unable to handle the request, possibly due to maintenance or overload. This is typically a transient error; try again later.

## Rate Limits
The API implements rate limiting to ensure fair usage and service stability. Specific limits are not disclosed in the schema, but you should expect responses with a `429 Too Many Requests` status code if limits are exceeded. Implement a retry strategy with exponential backoff to handle these responses gracefully.

## Error Handling
Effective error handling is crucial for building robust applications with the GPU Chat API.
*   **Interpret Error Responses:** Always check the HTTP status code of the response. For non-2xx status codes, the response body will usually contain a JSON object with `detail` or other fields explaining the error.
*   **Common Errors:**
    *   **401 Unauthorized**: Ensure your `x-api-market-key` header is correctly set with a valid API key.
    *   **422 Unprocessable Entity**: Review the `requestBody` of your POST requests against the OpenAPI schema for required fields, data types, and value constraints (e.g., `temperature` between 0 and 2, `max_tokens` within range).
    *   **503 Service Unavailable**: This often indicates an issue with the upstream chat service.
*   **Retry Strategies:** For transient errors like `429 Too Many Requests` or `503 Service Unavailable`, implement a retry mechanism with exponential backoff. This means retrying the request after an increasing delay (e.g., 1s, 2s, 4s, 8s) up to a maximum number of retries or a total timeout. Do not retry for `400` or `422` errors without modifying the request first.

**Error Response Example (Unauthorized):**
```json
{
  "detail": "Invalid or missing proxy secret"
}
```

**Error Response Example (Validation Error - 422):**
```json
{
  "detail": [
    {
      "loc": [
        "body",
        "messages"
      ],
      "msg": "field required",
      "type": "value_error.missing"
    }
  ]
}
```

**Error Response Example (Upstream Unavailable - 503):**
```json
{
  "detail": "Upstream chat service is currently unavailable. Please try again later."
}
```

## Implementation Guide

### Example Code

Since this API is **synchronous**, the examples below demonstrate making a single request and directly processing the response. There are no polling loops or prediction IDs required.

#### JavaScript

```javascript
// JavaScript example for synchronous API call
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/quickpod/aiquick-chat";

async function sendSimpleChatRequest(prompt, systemPrompt = "You are a helpful assistant.", model = "Qwen/Qwen3-0.6B", temperature = 0.7, maxTokens = 64) {
  try {
    const response = await fetch(`${baseUrl}/chat/simple`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-api-market-key": apiKey
      },
      body: JSON.stringify({
        model: model,
        system_prompt: systemPrompt,
        prompt: prompt,
        temperature: temperature,
        max_tokens: maxTokens
      })
    });
    
    if (!response.ok) {
      const errorData = await response.json();
      throw new Error(`API error: ${response.status} ${response.statusText}. Details: ${JSON.stringify(errorData)}`);
    }
    
    const data = await response.json();
    return data;
  } catch (error) {
    console.error("Error sending chat request:", error);
    throw error;
  }
}

// Example Usage:
(async () => {
  try {
    const chatResult = await sendSimpleChatRequest("What is the capital of France?");
    console.log("Chat Response:", chatResult.content);
  } catch (error) {
    console.error("Failed to get chat response.");
  }
})();

async function sendChatCompletionsRequest(messages, model = "Qwen/Qwen3-0.6B", temperature = 0.7, maxTokens = 128) {
  try {
    const response = await fetch(`${baseUrl}/chat/completions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-api-market-key": apiKey
      },
      body: JSON.stringify({
        model: model,
        messages: messages,
        temperature: temperature,
        max_tokens: maxTokens
      })
    });
    
    if (!response.ok) {
      const errorData = await response.json();
      throw new Error(`API error: ${response.status} ${response.statusText}. Details: ${JSON.stringify(errorData)}`);
    }
    
    const data = await response.json();
    return data;
  } catch (error) {
    console.error("Error sending chat completions request:", error);
    throw error;
  }
}

// Example Usage for Chat Completions:
(async () => {
  try {
    const chatMessages = [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "Tell me a short story about a cat." }
    ];
    const completionResult = await sendChatCompletionsRequest(chatMessages);
    console.log("Chat Completions Response:", completionResult.choices[0].message.content);
  } catch (error) {
    console.error("Failed to get chat completions response.");
  }
})();
```

#### Python

```python
# Python example for synchronous API call
import requests
import json

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/quickpod/aiquick-chat"

def send_simple_chat_request(prompt, system_prompt="You are a helpful assistant.", model="Qwen/Qwen3-0.6B", temperature=0.7, max_tokens=64):
    headers = {
        "Content-Type": "application/json",
        "x-api-market-key": API_KEY
    }
    payload = {
        "model": model,
        "system_prompt": system_prompt,
        "prompt": prompt,
        "temperature": temperature,
        "max_tokens": max_tokens
    }
    
    try:
        response = requests.post(f"{BASE_URL}/chat/simple", headers=headers, json=payload)
        response.raise_for_status() # Raises HTTPError for bad responses (4xx or 5xx)
        return response.json()
    except requests.exceptions.HTTPError as err:
        print(f"API error: {err.response.status_code} {err.response.text}")
        raise
    except requests.exceptions.RequestException as err:
        print(f"Request error: {err}")
        raise

# Example Usage:
try:
    chat_result = send_simple_chat_request("Suggest a name for a pet dog.")
    print("Chat Response:", chat_result.get("content"))
except Exception as e:
    print("Failed to get chat response.")

def send_chat_completions_request(messages, model="Qwen/Qwen3-0.6B", temperature=0.7, max_tokens=128):
    headers = {
        "Content-Type": "application/json",
        "x-api-market-key": API_KEY
    }
    payload = {
        "model": model,
        "messages": messages,
        "temperature": temperature,
        "max_tokens": max_tokens
    }
    
    try:
        response = requests.post(f"{BASE_URL}/chat/completions", headers=headers, json=payload)
        response.raise_for_status() # Raises HTTPError for bad responses (4xx or 5xx)
        return response.json()
    except requests.exceptions.HTTPError as err:
        print(f"API error: {err.response.status_code} {err.response.text}")
        raise
    except requests.exceptions.RequestException as err:
        print(f"Request error: {err}")
        raise

# Example Usage for Chat Completions:
try:
    chat_messages = [
        {"role": "system", "content": "You are a concise summarizer."},
        {"role": "user", "content": "Summarize this: The quick brown fox jumps over the lazy dog."}
    ]
    completion_result = send_chat_completions_request(chat_messages)
    print("Chat Completions Response:", completion_result.get("choices")[0].get("message").get("content"))
except Exception as e:
    print("Failed to get chat completions response.")
```

#### cURL

```bash
# cURL example for synchronous API call

# Example for sending a simple prompt
curl -X POST "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/simple" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "system_prompt": "You are a helpful assistant.",
    "prompt": "Tell me a fun fact about cats.",
    "temperature": 0.7,
    "max_tokens": 64
  }'

# Example for OpenAI-style chat completions
curl -X POST "https://prod.api.market/api/v1/quickpod/aiquick-chat/chat/completions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "What is the capital of Japan?"
      }
    ],
    "temperature": 0.7,
    "max_tokens": 128
  }'
```

### Performance Optimization
*   **Choose the Right Model:** Utilize the `/chat/models` endpoint to list available models and select one that balances performance (e.g., lower `max_model_len` might indicate faster inference for certain tasks) with your specific requirements.
*   **Optimize `max_tokens`:** Set `max_tokens` to the minimum value necessary for your desired output. Requesting fewer tokens reduces processing time and cost.
*   **Adjust `temperature`:** Lower `temperature` values (e.g., 0.1-0.3) tend to produce more predictable and often shorter responses, while higher values lead to more creative but potentially longer and less focused output. Optimize this based on your use case.
*   **Batch Requests (if applicable):** While the current API design doesn't explicitly support batching for chat requests, for other APIs, sending multiple requests concurrently (within rate limits) rather than sequentially can improve overall throughput.
*   **Efficient Prompt Engineering:** Craft concise and clear prompts that guide the model efficiently to the desired output, minimizing unnecessary token generation.

## FAQ

### What is the default model used by the API?

The default model is `Qwen/Qwen3-0.6B`, but you can specify a different model from the available list using the `model` parameter in your requests. You can also check the current default model via the `/health` endpoint.

### How do I get a list of all available models?

You can retrieve a list of all available chat models by making a `GET` request to the `/api/v1/quickpod/aiquick-chat/chat/models` endpoint.

### Is this API synchronous or asynchronous?

This API is synchronous. Requests return the final result directly in the response, so no polling is required.

### What should I do if I receive a `429 Too Many Requests` error?

A `429 Too Many Requests` response indicates that you've exceeded the API's rate limits. Implement a retry mechanism with exponential backoff (waiting progressively longer before each retry) to handle these errors gracefully.

### How can I ensure my API key is secure?

Always keep your API key confidential. Do not expose it in client-side code, public repositories, or unsecured environments. Store it securely using environment variables or a secrets management system.
## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello@magicapi.com.

Integrate the GPU Chat API for powerful LLM interactions. Get simple prompts or OpenAI-style chat completions, perfect for AI workflows, model experimentation, and application development.

GPU Chat API: Simple, Powerful LLM Completions & Prompts

Free