# Unable to Load Dataset Using \`load\_dataset\`

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-dataset-using-load-dataset%2F144579%3E%5C
**Category:** 🤗Datasets\
**Created:** [March 7, 2025, 8:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-dataset-using-load-dataset%2F144579 "2025-03-07T08:28:58Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 7, 2025, 12:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-dataset-using-load-dataset%2F144579%2F5 "2025-03-07T12:55:40Z")

</div>

Hmm…  
Maybe it would be better to shard the data set.

> [@How to load a large hf dataset efficiently?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-load-a-large-hf-dataset-efficiently%2F69288)%3A
>
> I am trying to load a dataset [axiong/pmc\_oa · Datasets at Hugging Face](https://huggingface.co/datasets/axiong/pmc_oa) The dataset size is around 22 gb and I have ram ~10 GB, the dataset object is stuck at extracting file point I also tried streaming mode but that’s giving another error. from datasets import load\_dataset dataset = load\_dataset("axiong/pmc\_oa", 'pmc\_oa', split='train', streaming=True) print(next(iter(dataset))) Any suggestion on how to deal with…

> [@How to load a large hf dataset efficiently?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-load-a-large-hf-dataset-efficiently%2F69288)%3A
>
> I am trying to load a dataset [axiong/pmc\_oa · Datasets at Hugging Face](https://huggingface.co/datasets/axiong/pmc_oa) The dataset size is around 22 gb and I have ram ~10 GB, the dataset object is stuck at extracting file point I also tried streaming mode but that’s giving another error. from datasets import load\_dataset dataset = load\_dataset("axiong/pmc\_oa", 'pmc\_oa', split='train', streaming=True) print(next(iter(dataset))) Any suggestion on how to deal with…

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-dataset-using-load-dataset%2F144579)._