# Load\_dataset() doesn't load ONE of the Subset
**URL:**
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 17, 2023, 12:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F1 "2023-10-17T12:57:35Z")
So I have a dataset: [stepkurniawan/qa-rag-llama · Datasets at Hugging Face](https://huggingface.co/datasets/stepkurniawan/qa-rag-llama/viewer/Llama-2-13b-chat-hf)
in that dataset, I made a couple of subsets:
1. “default”,
2. “Llama-2-13b-chat-hf”, and
3. “Llama-2-7b-chat-hf”.
I load using : `qa_dataset = load_dataset("stepkurniawan/qa-rag-llama","Llama-2-13b-chat-hf")`
When I load “Llama-2-7b-chat-hf”, everything works well, however, when I try to load “Llama-2-13b-chat-hf”, it gave me the dataset of the “default” one.
Why?
And how can I fix it?
Thank you!
Update: now I changed the “Llama-2-7b-chat-hf”, the changes are seen in the Dataset Viewer, but when I do in the code
```auto
HF_HUB_QA_LLAMA = "stepkurniawan/qa-rag-llama"
qa_dataset = load_dataset(HF_HUB_QA_LLAMA, "Llama-2-7b-chat-hf")
```
it still get me the old dataset… I dont understand…
Update 2:
The difference is just the number of rows… Previously, they have only 3-5 rows… But after I change, everything becomes 50 rows. Does HF have rows limit ?
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 18, 2023, 1:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F2 "2023-10-18T13:36:24Z")
pushing for visibility
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 20, 2023, 9:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F3 "2023-10-20T09:15:03Z")
Anyone?
is this a bug?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 20, 2023, 10:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F4 "2023-10-20T10:10:54Z")
Hi ! Maybe you tried to run `push_to_hub` twice at the same time and therefore it caused conflicts ? Feel free to retry and push subsets one at a time.
> Does HF have rows limit ?
No, you can have as many rows as you want
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 20, 2023, 10:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F5 "2023-10-20T10:18:41Z")
Hi @lhoestq , thanks for answering
Im sure I didnt do double pushing at the same time.
I tried with 2 subsets (“Llama2-13b” & “Llama2-7b”) and both of them are behaving weird the same way.
can you confirm what I am doing is correct?
1. push\_to\_hub: `result_dataset.push_to_hub(HF_HUB_QA_LLAMA, token=hf_token, config_name=subset_name)`
2. load\_dataset : `qa_dataset = load_dataset(HF_HUB_QA_LLAMA, subset_name)`
And another question is, by best practice, do you recommend push\_to\_hub(), or directly uploading to HF using terminal?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 20, 2023, 10:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F6 "2023-10-20T10:31:36Z")
Ok cool ! What you’re doing is correct.
> And another question is, by best practice, do you recommend push\_to\_hub(), or directly uploading to HF using terminal?
`push_to_hub()` is the recommended way. It optimizes for I/O efficiency, allows resuming, offers a better experience for the dataset viewer, and also allows defining multiple subsets.
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 20, 2023, 10:33am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F7 "2023-10-20T10:33:07Z")
Can you try to reupload your subsets and ping me if you still have an issue when you reload the data using `load_dataset` ?
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 20, 2023, 10:33am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F8 "2023-10-20T10:33:38Z")
ok…
that is a bad news…
since now I dont know what I did wrong and its still unsolved ☹
It normally works with datasets without subset, however im struggling to do with with subset…
I can try do it once more and ping you if the bad news still persist, sure.
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 20, 2023, 4:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F9 "2023-10-20T16:34:23Z")
@lhoestq
Hey, Ive tested it again.
It turns out that using another laptop will give me 50 rows, and using my laptop, I still get 3 rows.
Does it mean I have to clear cache somehow/ somewhere?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 20, 2023, 5:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F10 "2023-10-20T17:05:36Z")
The cache is automatically refreshed as long as you have an internet connection: it will get the latest files from the HF website. Otherwise it can use the lcoal cache and show a WARNING log message to say that it loaded the dataset from your local cache
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 20, 2023, 5:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F11 "2023-10-20T17:49:20Z")
I certainly don’t get the warning.
```auto
qa_dataset = load_dataset("stepkurniawan/qa-rag-llama", "Llama-2-13b-chat-hf")
print("something")
```
and this is in my terminal:
```auto
(base) [xxx@ml3-gpu2 RAG-comparation]$ /usr/bin/env /home/xxx/python /home/xxx/RAG-comparation/rag_ragas.py
something
```
btw, how to do I delete the cache?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 24, 2023, 8:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F12 "2023-10-24T08:58:44Z")
Actually it’s at “info” log level
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 24, 2023, 6:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F13 "2023-10-24T18:29:23Z")
@lhoestq
I turned on INFO log level using
```auto
transformers.logging.set_verbosity_info()
```
But i still didnt get the cache warning. Is the code above correct?
It could be that the problem is not the cache.
We could check that by deleting the cache maybe?
How to do that?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 25, 2023, 10:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F14 "2023-10-25T10:12:19Z")
The cache is at `~/.cache/huggingface/datasets` by default, feel free to delete it (make sure you’re not using `datasets` when deleting it)
---
**Author:**  [@stepkurniawan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstepkurniawan)%5C
**Post date:** [October 25, 2023, 10:20am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F15 "2023-10-25T10:20:07Z")
Thank you! it works after I delete the cache!!!
But it is weird since I dont want my code to use cache…
Is there any way i can force the program to NOT use the cache?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [October 25, 2023, 10:41am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-dataset-doesnt-load-one-of-the-subset%2F58874%2F16 "2023-10-25T10:41:00Z")
yes you can do
```python
import datasets
datasets.disable_caching()
```
and
```python
load_dataset(..., download_mode="force_redownload")
```