# Skill: Use ControlNet in ComfyUI to control pose, depth, edges, and composition

> Lets an AI agent add ControlNet guidance to a ComfyUI workflow -- choosing the right mode (pose, depth, canny, scribble), wiring the preprocessor and Apply ControlNet node, and setting strength -- whenever a prompt alone cannot reliably control spatial layout or character pose.

Source: sodigi·learn -- comfyui/controlnet · https://sodigi.io/learn/comfyui/controlnet

## When to use
- User needs a specific character pose that prompting alone does not reliably produce.
- User wants to preserve scene composition or volume while changing style.
- User has a rough sketch and wants to generate a finished image from it.
- User is making a series of images that must share the same layout.

## Core rules
- ControlNet adds a reference diagram to the generation; the prompt still controls content and style -- ControlNet controls geometry only.
- The pipeline is always: reference image -> preprocessor -> diagram -> Apply ControlNet (+ ControlNet model + conditioning) -> KSampler.
- Preprocessor type and ControlNet model type must match exactly: pose goes with pose, canny with canny, depth with depth.
- Four core modes: pose/OpenPose (skeleton of joints), depth (brightness = distance), canny (sharp outlines), scribble (rough doodles).
- Preprocessor nodes usually come from the "ControlNet Auxiliary Preprocessors" pack, installed via ComfyUI Manager.
- ControlNet model files are separate from checkpoint files; place them in `models/controlnet/`.
- Strength in Apply ControlNet sets how tightly the diagram constrains generation: high = rigid, low = loose. Start at 0.7-0.9 and adjust.

## Procedure
1. Obtain or create a reference image containing the pose, composition, or outlines you need.
2. Install the ControlNet Auxiliary Preprocessors pack via ComfyUI Manager if not already present.
3. Download a ControlNet model matching your chosen mode (e.g., OpenPose model for pose) and place it in `models/controlnet/`.
4. Load the reference image with a Load Image node.
5. Connect the image to the appropriate preprocessor node; the output is a diagram (skeleton, depth map, or edge map).
6. Add an Apply ControlNet node. Feed it: (a) Load ControlNet Model output, (b) the preprocessor diagram, (c) the conditioning from CLIP Text Encode.
7. Route Apply ControlNet's output conditioning into KSampler's positive conditioning input.
8. Run the generation; adjust strength if the pose is too rigid (lower) or too loose (raise).

## Ready-to-use prompt
```
I want to add ControlNet pose control to my ComfyUI workflow. Guide me through:

1. Choosing the right ControlNet mode for my reference: [pose / depth / canny / scribble].
2. Installing the preprocessor pack if needed.
3. Wiring the full chain: Load Image -> Preprocessor -> Apply ControlNet -> KSampler.
4. Placing the ControlNet model file in the correct folder.
5. Setting a starting strength and knowing which direction to adjust it.

My reference image contains: [describe the pose or composition].
My base checkpoint family: [e.g., SDXL, SD 1.5].
```

## Pitfalls
- Feeding a raw photo directly into Apply ControlNet without running it through the preprocessor first -- the model receives unintelligible input.
- Mismatching preprocessor type and ControlNet model type (e.g., depth map fed to a canny model) -- produces mush.
- Trying to install ControlNet without the Auxiliary Preprocessors pack -- the preprocessor nodes simply will not appear in the node search.
- Setting strength to the maximum (1.0) by default -- can over-constrain the model and degrade quality; tune from a mid-range starting point.
- Expecting ControlNet to handle style and content as well as geometry -- it holds structure only; prompt still drives everything else.
- Using a low-quality or blurry reference -- the preprocessor extracts a poor skeleton and the pose transfers poorly.