# Datasets not using the cache dir

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-not-using-the-cache-dir%2F63469%3E%5C
**Category:** 🤗Datasets\
**Created:** [November 25, 2023, 5:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-not-using-the-cache-dir%2F63469 "2023-11-25T05:39:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sebastienwood](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@sebastienwood](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsebastienwood)%5C
**Post date:** [November 25, 2023, 5:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-not-using-the-cache-dir%2F63469%2F1 "2023-11-25T05:39:07Z")

</div>

Hi !  
I have a really simple script:

```auto
import datasets
import os

dset = datasets.load_dataset("imagenet-1k", cache_dir=os.environ["DATASET_STORE"])

```

which should use a cached version of the Imagenet1k dataset at $DATASET\_STORE, which exists:

 ![image](https://us1.discourse-cdn.com/hellohellohello/original/3X/4/4/44950196e3ffba19ba2afe0bc345dc999c3389e2.png)  
And as the screen shows, it always tries to download the dataset online rather than using the cache.  
Am I doing it wrong ? Thanks !

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [November 28, 2023, 3:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-not-using-the-cache-dir%2F63469%2F2 "2023-11-28T15:48:20Z")

</div>

The `datasets`’ cache structure is incompatible with manually downloaded files, so you should use `Dataset.from_generator` instead to parse those files.

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [November 29, 2023, 5:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-not-using-the-cache-dir%2F63469%2F3 "2023-11-29T17:15:59Z")

</div>

I think it should work no ? `imagenet-1k` doesn’t need manually downloaded files.

Maybe it fails to find the loading script in the cache, which is in a different cache directory `dynamic_modules_cache` ?