# Custom dataset for Mask2Former finetuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-mask2former-finetuning%2F63052%3E%5C
**Category:** 🤗Datasets\
**Created:** [November 21, 2023, 8:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-mask2former-finetuning%2F63052 "2023-11-21T20:23:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![student101](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@student101](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstudent101)%5C
**Post date:** [November 21, 2023, 8:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-mask2former-finetuning%2F63052%2F1 "2023-11-21T20:23:05Z")

</div>

Hi,

I have started creating a synthetic dataset consisting of the original images and their segmentation masks (one black/white mask as a separate png image for each instance in the original image). My aim is to use my dataset for finetuning on Mask2Former.

As far as I understood according to [this link](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-do-i-create-a-image-segmentation-dataset%2F23123)%2C I need to merge all masks into one segmentation map for each original image. Is that right? If so, this shouldn’t be a problem.

For this step I don’t want to use segments.ai as suggested in this [blog](https://huggingface.co/blog/fine-tune-segformer), because I created my dataset with Blender and it’s quite simple to create segmentation maps from there with a python script.

So now I have original images and one segmentation map for each image.

What are the next steps for me? I somehow have to link the RGB colors of the masks to class names (in a JSON file? which format?), but how exactly does this need to look like?  
And what else do I need to do to create a hugging face dataset thats suited for finetuning Mask2Former?

As I am quite new to the world of AI/datasets/huggingface I might need you to explain it in small steps, even if the problem seems easy for you to solve 😊. Any help is appreciated. Thanks a lot!

---

<div class="post-metadata">

**Author:** ![panigrah](https://avatars.discourse-cdn.com/v4/letter/p/bb73d2/32.png) [@panigrah](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpanigrah)%5C
**Post date:** [November 22, 2023, 4:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-mask2former-finetuning%2F63052%2F2 "2023-11-22T16:49:34Z")

</div>

Convert all the masks into single channel gray scale images. Add up the masks for each image such that you end up with a single mask for each image. Each pixel in the combined mask would represent a single number and that number would correspond to an unique class name.

I think you have already done this based on your post.

I am assuming you have fewer than 256 labels, so all your classes can be described by a single channel mask image.

You can then follow this [recipe](https://github.com/huggingface/transformers/tree/main/examples/pytorch/semantic-segmentation#note-on-custom-data) to create the dataset.

Also create the id2label file which is just the mapping of your grayscale value in the mask to the class.

---

<div class="post-metadata">

**Author:** ![student101](https://avatars.discourse-cdn.com/v4/letter/s/a9adbd/32.png) [@student101](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstudent101)%5C
**Post date:** [November 23, 2023, 4:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcustom-dataset-for-mask2former-finetuning%2F63052%2F3 "2023-11-23T16:30:48Z")

</div>

Hi panigrah,

thank you very much. That helped a lot. I tried to implement it today and I think I got it now.