# Img2img inpaint + controlnet openpose

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimg2img-inpaint-controlnet-openpose%2F58320%3E%5C
**Category:** Beginners\
**Created:** [October 12, 2023, 2:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimg2img-inpaint-controlnet-openpose%2F58320 "2023-10-12T14:43:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jaank](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jaank/32/20282_2.png) [@jaank](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjaank)%5C
**Post date:** [October 12, 2023, 2:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimg2img-inpaint-controlnet-openpose%2F58320%2F1 "2023-10-12T14:43:57Z")

</div>

Hi there,

I am trying to create a workflow with these inputs:

- prompt
- image
- mask\_image

- use ControlNet openpose

It needs to persist the masked part of the input image and generate new content around the masked area to fit in.

I tried to use StableDiffusionControlNetInpaintPipeline with lllyasviel/control\_v11p\_sd15\_openpose.

```auto
vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse")
controlnet = ControlNetModel.from_pretrained(
    "lllyasviel/control_v11p_sd15_openpose", torch_dtype=torch.float16
)
pipe = StableDiffusionControlNetInpaintPipeline.from_pretrained(
    #"runwayml/stable-diffusion-v1-5",
    "SG161222/Realistic_Vision_V5.1_noVAE", 
    controlnet=controlnet,
    vae=vae,
    safety_checker=None,
    torch_dtype=torch.float16
).to("cuda")

```

then:

```auto
        self.pipe = StableDiffusionControlNetInpaintPipeline.from_pretrained(
            CACHE_DIR, torch_dtype=torch.float16
        ).to("cuda")
        self.pipe.scheduler = EulerDiscreteScheduler.from_config(
            self.pipe.scheduler.config
        )
        self.pipe.enable_xformers_memory_efficient_attention()

        image = image.resize((512,512))
        mask_image = mask_image.resize((512,512))
        
        print(image)
        print(mask_image)
        out = self.pipe(
            prompt=prompt,
            negative_prompt=negative_prompt,
            image=image,
            mask_image=mask_image,
            width=width,
            height=height,
            guidance_scale=guidance_scale,
            generator=torch.Generator().manual_seed(seed),
            num_inference_steps=num_inference_steps,
        )

```

The image seems to be correct when printing then it is:

```auto
<PIL.Image.Image image mode=RGB size=512x512 at 0x7FE43E194E50>

```

But self.pipe() gives error:

```auto
TypeError: image must be passed and be one of PIL image, numpy array, torch tensor, list of PIL images, list of numpy arrays or list of torch tensors, but is <class 'NoneType'>

```

---

<div class="post-metadata">

**Author:** ![matischrod](https://avatars.discourse-cdn.com/v4/letter/m/90db22/32.png) [@matischrod](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmatischrod)%5C
**Post date:** [May 5, 2024, 12:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fimg2img-inpaint-controlnet-openpose%2F58320%2F3 "2024-05-05T12:12:06Z")

</div>

I had the same error and just solved adding the parameter **control\_image**.  
The error didn’t specified that very well.

> **[ControlNet](https://huggingface.co/docs/diffusers/main/en/api/pipelines/controlnet#diffusers.StableDiffusionControlNetInpaintPipeline. __call__ )**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.