# Load shards as one dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-shards-as-one-dataset%2F73422%3E%5C
**Category:** 🤗Datasets\
**Created:** [February 16, 2024, 3:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-shards-as-one-dataset%2F73422 "2024-02-16T03:47:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![chenchaozhao](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@chenchaozhao](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchenchaozhao)%5C
**Post date:** [February 16, 2024, 3:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-shards-as-one-dataset%2F73422%2F1 "2024-02-16T03:47:35Z")

</div>

I processed the datasets into several shards. If I want to load them as one piece I can do concatenation but it will take some time to index all of the files. Is there a quicker way to load the dataset like a memory mapping from several dataset shards?