# Does huggingface support load raw text dataset from hdfs?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233%3E%5C
**Category:** 🤗Datasets\
**Created:** [December 30, 2021, 5:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233 "2021-12-30T05:57:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pipi](https://avatars.discourse-cdn.com/v4/letter/p/e9bcb4/32.png) [@pipi](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpipi)%5C
**Post date:** [December 30, 2021, 5:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233%2F1 "2021-12-30T05:57:54Z")

</div>

I noticed that both **load\_from\_disk** and **save\_to\_disk** support the **‘fs’** parameters, but not **load\_dataset**.

I want to know if it is possible to load text data from **hdfs** via **load\_dataset**

Any suggestion will be appreciately.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [January 5, 2022, 2:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233%2F2 "2022-01-05T14:47:14Z")

</div>

Hi ! Currently `load_dataset` only supports reading files from local paths or HTTP. We may add support for other filesystems (hdfs, cloud storages…) later 🙂

---

<div class="post-metadata">

**Author:** ![pipi](https://avatars.discourse-cdn.com/v4/letter/p/e9bcb4/32.png) [@pipi](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpipi)%5C
**Post date:** [January 6, 2022, 5:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233%2F3 "2022-01-06T05:39:49Z")

</div>

Thanks for your reply, another question is Does **load\_from\_disk** support **stream** mode?

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [January 9, 2022, 9:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-huggingface-support-load-raw-text-dataset-from-hdfs%2F13233%2F4 "2022-01-09T21:16:18Z")

</div>

No it doesn’t, only lozd\_dataset supports streaming mode