# Extremely slow data loading of imagefolder

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextremely-slow-data-loading-of-imagefolder%2F60131%3E%5C
**Category:** 🤗Datasets\
**Created:** [October 27, 2023, 7:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextremely-slow-data-loading-of-imagefolder%2F60131 "2023-10-27T07:44:04Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![kopyl](https://avatars.discourse-cdn.com/v4/letter/k/3bc359/32.png) [@kopyl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkopyl)%5C
**Post date:** [December 19, 2023, 6:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextremely-slow-data-loading-of-imagefolder%2F60131%2F6 "2023-12-19T06:29:23Z")

</div>

@panigrah thank you very much. Maybe you also know if it’s possible to download a dataset in multi-processed way? For some reason [setting num\_proc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-can-i-multithreadedly-download-a-huggingface-dataset%2F56178%2F2) does not work at all… My dataset has 58 parquet files and i was hoping passing `num_proc` to `load_dataset` would spawn 58 Python processes each downloading its own parquet so I can load my dataset in 1 minutes instead of 50…

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fextremely-slow-data-loading-of-imagefolder%2F60131)._