Name: LLaVA 13B – Image Analysis & Q&A | $0.05/call | API.market
Brand: magicapi
SKU: api-magicapi-visual-language-instruction-ai-api
Availability: InStock
Author: API Market

Shashank Agarwal

# 🖼️ Advanced Visual-Language Instruction API (LLaVA-13B)

## Overview
The Advanced Visual-Language Instruction API (LLaVA-13B) enables developers to analyze images and generate GPT-4-level answers to questions regarding visual content. By combining natural language processing with image recognition, you can provide nuanced and context-aware responses about virtually any image.

Key advantages include:
• Rapid inference for image-related questions.  
• High-precision understanding of complex visual scenes.  
• Seamless text generation based on visual context.  

This API is ideal for creating smart image-driven applications such as virtual assistants, educational tools, or content moderation systems.

## Features
- GPT-4 level vision-language understanding  
- Handles diverse image queries (e.g., object detection, scene interpretation, contextual question answering)  
- Configurable text generation parameters (max tokens, temperature, top_p)  
- Straightforward asynchronous prediction workflow with just two main endpoints  
- Scalable to large usage volumes with different pricing tiers  

## Use Cases
1. E-commerce Image Captioning  
   Automatically generate descriptive product captions and responses to user queries about item features or usage scenarios.

2. Educational Content Creation  
   Provide context-aware explanations of diagrams, photos, or historical images to enhance learning materials with AI-generated descriptions.

3. Smart Surveillance & Security  
   Detect objects or people in camera footage and generate warnings or notifications about observed scenarios, augmenting security workflows.

4. Social Media Content Analysis  
   Analyze user-uploaded images to detect context, sentiment, or brand presence, helping automate moderation or marketing campaigns.

5. Travel & Tourism Applications  
   Enable travel apps to provide historical or cultural context about a landmark image, respond to frequently asked questions, and suggest travel tips.

## Visual Examples

Below is an example of how the API can analyze and respond to an image. The image link is from the request examples in the OpenAPI schema. Imagine a scenario where a user uploads the following lake/pier image and asks, “Are you allowed to swim here?”

### Input Image
![A peaceful lake with a wooden pier. The user’s question is: “Are you allowed to swim here?”](https://replicate.delivery/pbxt/KRULC43USWlEx4ZNkXltJqvYaHpEx2uJ4IyUQPRPwYb8SzPf/view.jpg)

### Example Output
The API might generate a textual answer along the lines of:  
“Yes, you are allowed to swim in the lake near the pier… However, it is always important to be cautious…”

## Pricing
• 1 API unit = 1 image processed.  
• GET requests (checking prediction status) do not consume API units.  
• Multiple pricing tiers are available to accommodate different usage levels.

## API Documentation

### Base URL
All API requests should be made to:
```
https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api
```

### Authentication
Include your API key in the “x-magicapi-key” header:
```
x-magicapi-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
This API uses an asynchronous approach for image analysis and text generation:

1. Send a POST request to create a prediction.  
2. You receive a prediction ID.  
3. Poll the GET endpoint using this prediction ID to check its status while it processes.  
4. When the status is “succeeded,” the output field contains your result.

Guidelines:  
- Poll at 1-second intervals.  
- Maximum timeout is 600 seconds (10 minutes).  
- Stop polling if the status is “failed” or after reaching the timeout.

### Endpoints

#### 1. Create a Prediction
POST /api/v1/magicapi/visual-language-instruction-ai-api/predictions

• Sends an image and text prompt for processing.  
• Returns a unique prediction ID.

Example request (cURL):
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api/predictions" \
  -H "Content-Type: application/json" \
  -H "x-magicapi-key: YOUR_API_MARKET_KEY" \
  -d '{
    "input": {
      "image": "https://example.com/image.jpg",
      "prompt": "Are you allowed to swim here?",
      "max_tokens": 1024,
      "temperature": 0.2,
      "top_p": 1
    }
  }'
```

Example JSON response (HTTP 201):
```json
{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "80537f9eead1a5bfa72d5ac6ea6414379be41d4d4f6679fd776e9535d1eb58bb",
  "status": "starting",
  "created_at": "2025-05-18T08:21:06.873Z"
}
```

#### 2. Get Prediction Status and Results
GET /api/v1/magicapi/visual-language-instruction-ai-api/predictions/{id}

• Retrieves the status of a prediction and its output once complete.  
• {id} is the unique prediction ID from the creation step.

Example request (cURL):
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api/predictions/ufawqhfynnddngldkgtslldrkq" \
  -H "x-magicapi-key: YOUR_API_MARKET_KEY"
```

Example JSON response (HTTP 200):
```json
{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "status": "succeeded",
  "output": [
    "Yes, ",
    "you ",
    "are ",
    "allowed ",
    "to ",
    "swim ",
    "in ",
    "the ",
    "lake ",
    "near ",
    "the ",
    "pier. ",
    "..."
  ],
  "created_at": "2023-01-01T12:00:00.000Z",
  "started_at": "2023-01-01T12:00:05.000Z",
  "completed_at": "2023-01-01T12:01:00.000Z"
}
```

## Status Codes
- 200 OK: Request successful.  
- 201 Created: Prediction successfully created.  
- 400 Bad Request: Input parameters invalid or request not well-formed.  
- 401 Unauthorized: Invalid or missing API key.  
- 403 Forbidden: Authentication succeeded, but not authorized to perform the request.  
- 404 Not Found: Resource or prediction ID not found.  
- 429 Too Many Requests: Rate limit exceeded; back off and retry.  
- 500 Internal Server Error: Unexpected server issue.  
- 503 Service Unavailable: Service temporarily overloaded or in maintenance.

## Rate Limits
Requests may be subject to rate limiting. If you encounter a 429 response, wait briefly before retrying. We recommend an exponential backoff strategy for best results under high load.

## Error Handling
When an error occurs, the API returns a JSON payload with details:
```json
{
  "error": {
    "code": "invalid_parameter",
    "message": "The parameter 'image' is required",
    "status": 400
  }
}
```
Best practices:
- Check the response body for specific error details.  
- Correct input or retry for transient errors.  
- Implement a backoff strategy for rate limit or service errors.

## Implementation Guide

### Example Code

#### JavaScript (Node.js)
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api";

async function createAndWaitForPrediction(inputData) {
  try {
    // Step 1: Create the prediction
    const response = await fetch(`${baseUrl}/predictions`, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        "x-magicapi-key": apiKey
      },
      body: JSON.stringify({ input: inputData })
    });
    
    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${response.statusText}`);
    }
    
    const prediction = await response.json();
    const predictionId = prediction.id;
    
    // Step 2: Poll for results with timeout
    return await pollForResults(predictionId);
  } catch (error) {
    console.error("Error:", error);
    throw error;
  }
}

function delay(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function pollForResults(predictionId) {
  const MAX_TIMEOUT = 600000; // 600 seconds (10 minutes)
  const POLL_INTERVAL = 1000; // 1 second interval
  
  const startTime = Date.now();
  
  while (Date.now() - startTime < MAX_TIMEOUT) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: {
        "x-magicapi-key": apiKey
      }
    });
    
    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${response.statusText}`);
    }
    
    const prediction = await response.json();
    
    if (prediction.status === "succeeded") {
      return prediction.output;
    }
    
    if (prediction.status === "failed") {
      throw new Error(`Prediction failed: ${prediction.error || "Unknown error"}`);
    }
    
    await delay(POLL_INTERVAL);
  }
  
  throw new Error("Prediction timed out after 10 minutes");
}
```

#### Python
```python
import requests
import time

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api"

def create_and_wait_for_prediction(input_data):
    # Step 1: Create the prediction
    headers = {
        "Content-Type": "application/json",
        "x-magicapi-key": API_KEY
    }
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"input": input_data}
    )
    
    if response.status_code != 200 and response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")
    
    prediction = response.json()
    prediction_id = prediction["id"]
    
    # Step 2: Poll for results with timeout
    return poll_for_results(prediction_id)

def poll_for_results(prediction_id):
    MAX_TIMEOUT = 600  # 600 seconds (10 minutes)
    POLL_INTERVAL = 1  # 1 second
    
    headers = {"x-magicapi-key": API_KEY}
    start_time = time.time()
    
    while time.time() - start_time < MAX_TIMEOUT:
        response = requests.get(
            f"{BASE_URL}/predictions/{prediction_id}",
            headers=headers
        )
        
        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")
        
        prediction = response.json()
        
        if prediction["status"] == "succeeded":
            return prediction["output"]
        
        if prediction["status"] == "failed":
            error_msg = prediction.get("error", "Unknown error")
            raise Exception(f"Prediction failed: {error_msg}")
        
        time.sleep(POLL_INTERVAL)
    
    raise Exception("Prediction timed out after 10 minutes")
```

#### cURL
```bash
# 1. Create a prediction
curl -X POST "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api/predictions" \
  -H "Content-Type: application/json" \
  -H "x-magicapi-key: YOUR_API_MARKET_KEY" \
  -d '{
    "input": {
      "image": "https://example.com/image.jpg",
      "prompt": "Describe the image"
    }
  }'

# You will receive a response containing a prediction ID, like {"id": "abcd1234", "status": "starting"}

# 2. Poll the status until it shows "succeeded"
curl -X GET "https://prod.api.market/api/v1/magicapi/visual-language-instruction-ai-api/predictions/abcd1234" \
  -H "x-magicapi-key: YOUR_API_MARKET_KEY"
```

### Performance Optimization
- Reduce max_tokens for shorter responses and faster execution.  
- Use lower temperature (e.g., 0.2) to get more deterministic outputs, which can reduce the chance of multiple re-runs.  
- Reuse Prediction IDs responsibly if polling multiple times.  

## FAQ

• How long do predictions take?  
  Typically a few seconds, but can take up to 10 minutes for complex images or high-volume workloads.

• Do status checks consume API units?  
  No, status checks (GET requests) are free and do not count toward unit usage.

• Can I process multiple images at once?  
  Each POST request processes one image. You can send multiple requests in parallel if your plan allows it.

• How is the output formatted?  
  The output is returned as an array of tokens (strings). You can join them for a complete answer.

## Contact Us
If you have any questions or need assistance, you can reach out to us through the embedded chat on the bottom right corner of the page or send an email to hello[at]api.market.

Analyze images and answer questions with GPT-4 level AI. Get high-precision understanding of complex visual scenes. Try it free on API.market.