Name: SD 1.5 Canny ControlNet LoRA API: Edge-Guided Images
Brand: magicapi
SKU: api-magicapi-canny-controlnet-lora-api-pnyompen-sd-controlnet-lora
Availability: InStock
Author: API Market

Shashank Agarwal

# 🖼️ Edge-Guided Image Generator API (pnyompen SD 1.5 Canny ControlNet LoRA)

## Overview
This model runs Stable Diffusion 1.5 with a Canny edge ControlNet and LoRA support. Its developer describes it as "SD1.5 Canny controlnet with LoRA support".

Send a reference `image` and a `prompt`. The model extracts the edges of the image and generates a new picture that follows them, styled by the prompt; `condition_scale` sets how strictly. Turn on `img2img` to also start from the reference image (`strength` sets how much changes). Extra controls: `ip_adapter_scale` for the IP-Adapter, `auto_generate_caption` to add a BLIP caption of the image to the prompt, and `remove_bg` to remove the input's background. Each request returns 1 to 4 images (`num_outputs`).

## Features
- **Edge-guided generation**: a Canny ControlNet keeps the outlines and composition of the reference image.
- **img2img mode**: `img2img` uses the image as both the control image and the starting image; `strength` (default `0.8`) sets how much changes.
- **Automatic captions**: `auto_generate_caption` adds a BLIP caption of the input image to the prompt, weighted by `generated_caption_weight`.
- **IP-Adapter**: `ip_adapter_scale` (default `1`) sets the IP-Adapter's influence.
- **Background removal**: `remove_bg` removes the background of the input image first.
- **LoRA support**: load LoRA weights with `lora_weights` and set their influence with `lora_scale`.
- **1 to 4 images per request** with `num_outputs`, 7 schedulers, 1 to 500 steps.

## Use Cases
1. **Restyling images**: keep the composition of a picture and render it as an oil painting, sketch or other style.
2. **Sketch to image**: turn line drawings or outlines into finished images.
3. **Product shots**: remove the background of a product photo and generate it in a new style.
4. **Consistent variations**: generate several images that share the same structure.

## Visual Examples
### Example Input
The reference image below, restyled with the prompt "An astronaut riding a rainbow unicorn, In the style of oil painting". The request uses `img2img` with `strength` `1`, an automatic caption with weight `0.1`, `ip_adapter_scale` `0.1`, the `KarrasDPM` scheduler and 15 steps.

![Example input: the reference image](https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-input-api-market.jpg)

```json
{
  "image": "https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-input-api-market.jpg",
  "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
  "img2img": true,
  "strength": 1,
  "scheduler": "KarrasDPM",
  "lora_scale": 0.95,
  "num_outputs": 1,
  "guidance_scale": 7.5,
  "condition_scale": 0.5,
  "negative_prompt": "worst quality, photorealistic ",
  "ip_adapter_scale": 0.1,
  "num_inference_steps": 15,
  "auto_generate_caption": true,
  "generated_caption_weight": 0.1
}
```

### Example Output
![SD 1.5 Canny ControlNet output: an oil painting style image following the reference image's outlines](https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-output-etrd5mvt-api-market.png)

## Input Parameters
| Field | Type | Required | Default | Allowed values | Description |
| --- | --- | --- | --- | --- | --- |
| `prompt` | string | No | `An astronaut riding a rainbow unicorn` |  | Text prompt |
| `image` | string (URL) | No | none |  | Image URL (jpeg, png, gif or webp). Its Canny edges guide the generation; in img2img mode it is also the starting image |
| `img2img` | boolean | No | `false` |  | Use the img2img pipeline: the image is used both as the control image and as the base image |
| `auto_generate_caption` | boolean | No | `false` |  | Generate a caption of the input image with BLIP and add it to the prompt |
| `generated_caption_weight` | number | No | `0.5` | `0` or more | Weight of the generated caption |
| `condition_scale` | number | No | `1.1` | `0` to `2` | How strongly the ControlNet edges steer the result. Higher values follow the edges more closely |
| `strength` | number | No | `0.8` | `0` to `1` | Denoising strength when img2img is on. 1 discards the input image completely |
| `ip_adapter_scale` | number | No | `1` | `0` or more | Scale for the IP-Adapter |
| `negative_prompt` | string | No | `""` |  | Negative prompt: what to avoid in the image |
| `num_inference_steps` | integer | No | `30` | `1` to `500` | Number of denoising steps |
| `num_outputs` | integer | No | `1` | `1` to `4` | Number of images to return |
| `scheduler` | string | No | `K_EULER` | `DDIM`, `DPMSolverMultistep`, `HeunDiscrete`, `KarrasDPM`, `K_EULER_ANCESTRAL`, `K_EULER`, `PNDM` | Sampling scheduler |
| `guidance_scale` | number | No | `7.5` | `1` to `50` | Scale for classifier-free guidance |
| `seed` | integer | No | none |  | Random seed. Leave blank for a random seed |
| `lora_scale` | number | No | `0.95` | `0` to `1` | LoRA additive scale. Applies only when LoRA weights are loaded |
| `lora_weights` | string | No | none |  | URL of LoRA weights to load. Leave blank to use the default weights |
| `remove_bg` | boolean | No | `false` |  | Remove the background from the input image |

## Pricing
- **Pay As You Go (PAYG) pricing**: you only pay for what you use.
- **Price**: $0.018 per API unit.
- **1 API unit = 1 request with up to 72 seconds of processing time.** A request that takes longer is billed one unit per started 72-second block (for example, 100 seconds of processing is 2 units, $0.036).
- A typical request fits in one unit; many steps or 4 images take longer and can use more.
- **Example cost**: the example (1 image, 15 steps) took 5.07 seconds of processing: 1 API unit = **$0.018**.
- **Failed or cancelled predictions are not billed.**
- **GET requests to check prediction status** are free and don't count toward API unit usage.
- **No monthly subscription fee.**

## API Documentation

### Base URL
All API requests go to: `https://prod.api.market/api/v1/magicapi/canny-controlnet-lora-api-pnyompen-sd-controlnet-lora`

### Authentication
Send your api.market API key in the `x-api-market-key` header:
```
x-api-market-key: YOUR_API_MARKET_KEY
```

### Asynchronous Processing
**IMPORTANT**: This API is asynchronous. The flow is:
1. Send a POST request to `/predictions`. A successful request returns **201 Created** with a prediction `id` and a `status` of `starting` or `processing`.
2. Poll GET `/predictions/{id}` with that ID.
3. Keep polling while `status` is `starting` or `processing`.
4. Stop when `status` is `succeeded` (read `output`), `failed` or `canceled` (read `error`).

A polling interval of 1 second and an overall timeout of 600 seconds (10 minutes) work well.

### Endpoints

#### Create a Prediction
```bash
curl -X POST "https://prod.api.market/api/v1/magicapi/canny-controlnet-lora-api-pnyompen-sd-controlnet-lora/predictions" \
  -H "Content-Type: application/json" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY" \
  -d '{
    "version": "37ceaf8d4df13e7fe7f1320413189c641c825210df7e40bc21072634a10029bb",
    "input": {
      "image": "https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-input-api-market.jpg",
      "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
      "img2img": true,
      "strength": 1,
      "scheduler": "KarrasDPM",
      "lora_scale": 0.95,
      "num_outputs": 1,
      "guidance_scale": 7.5,
      "condition_scale": 0.5,
      "negative_prompt": "worst quality, photorealistic ",
      "ip_adapter_scale": 0.1,
      "num_inference_steps": 15,
      "auto_generate_caption": true,
      "generated_caption_weight": 0.1
    }
  }'
```

Response (201 Created):
```json
{
  "id": "etrd5mvtq1rgc0d1cd686sxrn0",
  "version": "37ceaf8d4df13e7fe7f1320413189c641c825210df7e40bc21072634a10029bb",
  "status": "starting",
  "created_at": "2026-10-22T11:08:39.131Z"
}
```

#### Check Prediction Status
```bash
curl -X GET "https://prod.api.market/api/v1/magicapi/canny-controlnet-lora-api-pnyompen-sd-controlnet-lora/predictions/{id}" \
  -H "x-api-market-key: YOUR_API_MARKET_KEY"
```

Response when the prediction has finished (200 OK):
```json
{
  "id": "etrd5mvtq1rgc0d1cd686sxrn0",
  "status": "succeeded",
  "output": [
    "https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-output-etrd5mvt-api-market.png"
  ],
  "error": null,
  "created_at": "2026-10-22T11:08:39.131Z",
  "started_at": "2026-10-22T11:11:36.323Z",
  "completed_at": "2026-10-22T11:11:41.397Z"
}
```

`output` is a list of image URLs, one per requested image (`num_outputs`).

### Status Codes
- **201 Created**: prediction created (POST `/predictions`). This is the normal success response.
- **200 OK**: prediction details returned (GET `/predictions/{id}`).
- **400 Bad Request**: invalid input, for example a `condition_scale` above 2, a `num_outputs` above 4 or a `scheduler` that is not in the list.
- **401 Unauthorized**: missing or invalid API key.
- **404 Not Found**: no prediction with that ID.
- **429 Too Many Requests**: rate limit exceeded.
- **500 Internal Server Error** or **503 Service Unavailable**: temporary server problem; retry later.

## Rate Limits
All plans allow 10 requests per second. Requests above that return `429 Too Many Requests`; back off before retrying and honour any `Retry-After` header. Polling a prediction with GET is free but still counts toward the per-second limit, so poll every 1 to 5 seconds rather than in a tight loop.

## Error Handling
- Check the HTTP status code of every request and read the response body for the error message.
- A prediction can also end with `status` set to `failed`; the `error` field then explains why. Failed predictions are not billed.
- Retry 429, 500 and 503 responses with exponential backoff. Fix the input before retrying a 400.

## Implementation Guide
### Example Code

#### JavaScript
```javascript
const apiKey = "YOUR_API_MARKET_KEY";
const baseUrl = "https://prod.api.market/api/v1/magicapi/canny-controlnet-lora-api-pnyompen-sd-controlnet-lora";

async function createAndWaitForPrediction(input) {
  const response = await fetch(`${baseUrl}/predictions`, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "x-api-market-key": apiKey
    },
    body: JSON.stringify({ version: "37ceaf8d4df13e7fe7f1320413189c641c825210df7e40bc21072634a10029bb", input })
  });

  if (response.status !== 201) {
    throw new Error(`API error: ${response.status} ${await response.text()}`);
  }

  const prediction = await response.json();
  return pollForResults(prediction.id);
}

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

async function pollForResults(predictionId) {
  const maxTimeout = 600000;
  const pollInterval = 1000;
  const startTime = Date.now();

  while (Date.now() - startTime < maxTimeout) {
    const response = await fetch(`${baseUrl}/predictions/${predictionId}`, {
      headers: { "x-api-market-key": apiKey }
    });

    if (!response.ok) {
      throw new Error(`API error: ${response.status} ${await response.text()}`);
    }

    const prediction = await response.json();

    if (prediction.status === "succeeded") {
      return prediction.output;
    }

    if (prediction.status === "failed" || prediction.status === "canceled") {
      throw new Error(`Prediction ${prediction.status}: ${prediction.error || "Unknown error"}`);
    }

    await delay(pollInterval);
  }

  throw new Error("Prediction timed out after 10 minutes");
}

createAndWaitForPrediction({
    "image": "https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-input-api-market.jpg",
    "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
    "img2img": true,
    "strength": 1,
    "scheduler": "KarrasDPM",
    "lora_scale": 0.95,
    "num_outputs": 1,
    "guidance_scale": 7.5,
    "condition_scale": 0.5,
    "negative_prompt": "worst quality, photorealistic ",
    "ip_adapter_scale": 0.1,
    "num_inference_steps": 15,
    "auto_generate_caption": true,
    "generated_caption_weight": 0.1
  }).then((output) => console.log(output));
```

#### Python
```python
import time
import requests

API_KEY = "YOUR_API_MARKET_KEY"
BASE_URL = "https://prod.api.market/api/v1/magicapi/canny-controlnet-lora-api-pnyompen-sd-controlnet-lora"


def create_and_wait_for_prediction(input_data):
    headers = {"Content-Type": "application/json", "x-api-market-key": API_KEY}
    response = requests.post(
        f"{BASE_URL}/predictions",
        headers=headers,
        json={"version": "37ceaf8d4df13e7fe7f1320413189c641c825210df7e40bc21072634a10029bb", "input": input_data},
    )

    if response.status_code != 201:
        raise Exception(f"API error: {response.status_code} {response.text}")

    return poll_for_results(response.json()["id"])


def poll_for_results(prediction_id):
    max_timeout = 600
    poll_interval = 1
    headers = {"x-api-market-key": API_KEY}
    start_time = time.time()

    while time.time() - start_time < max_timeout:
        response = requests.get(f"{BASE_URL}/predictions/{prediction_id}", headers=headers)

        if response.status_code != 200:
            raise Exception(f"API error: {response.status_code} {response.text}")

        prediction = response.json()

        if prediction["status"] == "succeeded":
            return prediction["output"]

        if prediction["status"] in ("failed", "canceled"):
            raise Exception(f"Prediction {prediction['status']}: {prediction.get('error') or 'Unknown error'}")

        time.sleep(poll_interval)

    raise Exception("Prediction timed out after 10 minutes")


print(create_and_wait_for_prediction({
    "image": "https://blog.api.market/wp-content/uploads/2026/10/sd-controlnet-lora-input-api-market.jpg",
    "prompt": "An astronaut riding a rainbow unicorn, In the style of oil painting",
    "img2img": True,
    "strength": 1,
    "scheduler": "KarrasDPM",
    "lora_scale": 0.95,
    "num_outputs": 1,
    "guidance_scale": 7.5,
    "condition_scale": 0.5,
    "negative_prompt": "worst quality, photorealistic ",
    "ip_adapter_scale": 0.1,
    "num_inference_steps": 15,
    "auto_generate_caption": True,
    "generated_caption_weight": 0.1
}))
```

## FAQ
- **What is the difference between the default mode and `img2img`?**
  - By default only the edges of `image` guide the new picture. With `img2img` set to `true`, the image is also the starting point and `strength` sets how much of it is replaced (1 replaces it completely).
- **What does `auto_generate_caption` do?**
  - It writes a caption of the input image with BLIP and adds it to your prompt. `generated_caption_weight` (default `0.5`) sets how much it counts.
- **Can I use my own LoRA?**
  - `lora_weights` takes a URL of LoRA weights to load, and `lora_scale` (0 to 1) sets their influence. Leave `lora_weights` blank to use the default weights.
- **How many images can one request return?**
  - 1 to 4, set with `num_outputs`.
- **How is a request priced?**
  - $0.018 per started 72 seconds of processing. The example took 5.07 seconds, so it cost 1 unit.
- **What if my prediction fails?**
  - Failed predictions are not billed. Read the `error` field, adjust the input and try again.

## Contact Us
If you have any questions, reach out to us through the chat on the bottom right of the page. You can also email our support team at hello@magicapi.com.


Generate images that follow the edges of your input picture with Stable Diffusion 1.5, Canny ControlNet and an optional LoRA. Pay per request.