Edit model card

ControlNet Canny for Flux

This repository provides a checkpoint with trained ControlNet Canny model for FLUX.1-dev model by Black Forest Labs

ComfyUI

See our github for comfy ui workflows. Example Picture 1

Training details

XLabs AI team is happy to publish fune-tuning Flux scripts, including:

  • LoRA πŸ”₯
  • ControlNet πŸ”₯

See our github for train script and train configs.

Training dataset

Dataset has the following format for the training process:

β”œβ”€β”€ images/
β”‚    β”œβ”€β”€ 1.png
β”‚    β”œβ”€β”€ 1.json
β”‚    β”œβ”€β”€ 2.png
β”‚    β”œβ”€β”€ 2.json
β”‚    β”œβ”€β”€ ...

A .json file contains "caption" field with a text prompt.

Inference

To test our checkpoints, use commands presented below.

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a handsome viking man with white hair, cinematic, MM full HD"

Example Picture 1

python3 demo_controlnet_inference.py \
    --checkpoint controlnet.safetensors \
    --control_image "input_image.jpg" \
    --prompt "a dark evil mysterius house with ghosts, cinematic, MM full HD"

Example Picture 2

License

controlnet.safetensors falls under the FLUX.1 [dev] Non-Commercial License

Downloads last month
2,158
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for XLabs-AI/flux-controlnet-canny

Finetuned
(259)
this model

Spaces using XLabs-AI/flux-controlnet-canny 9