# Building an imagefolder dataset takes too long

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbuilding-an-imagefolder-dataset-takes-too-long%2F22606%3E%5C
**Category:** 🤗Datasets\
**Created:** [September 6, 2022, 4:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbuilding-an-imagefolder-dataset-takes-too-long%2F22606 "2022-09-06T04:53:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vanewu](https://avatars.discourse-cdn.com/v4/letter/v/77aa72/32.png) [@vanewu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvanewu)%5C
**Post date:** [September 6, 2022, 4:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbuilding-an-imagefolder-dataset-takes-too-long%2F22606%2F1 "2022-09-06T04:53:08Z")

</div>

There are about 20,000+ images and text information in the local folder, and it took about 30 minutes to build an imagefolder dataset. The build process appears to be traversing the folders doing a series of confirmations. If there are billions of data, how to deal with it?  
code:

```auto
dataset = load_dataset('imagefolder',
                        data_dir='/home/data/ms_coco/val2017/',
                        streaming=True,
                        ignore_verifications=True,
                        cache_dir='/home/data/ms_coco/huggingface/valid'
            )

```

Please help me 😀.

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [September 6, 2022, 11:19am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbuilding-an-imagefolder-dataset-takes-too-long%2F22606%2F2 "2022-09-06T11:19:29Z")

</div>

Hi! Can you interrupt (CTRL + C or CMD + C) the process while waiting for it to finish and paste the returned error stack trace here to help us debug the issue? Also, what’s the output of the `datasets-cli env` command?