# In-memory dataset to disk for caching operations

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fin-memory-dataset-to-disk-for-caching-operations%2F17413%3E%5C
**Category:** 🤗Datasets\
**Created:** [May 2, 2022, 7:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fin-memory-dataset-to-disk-for-caching-operations%2F17413 "2022-05-02T07:37:20Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sanderbaduk](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sanderbaduk/32/4648_2.png) [@Sanderbaduk](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSanderbaduk)%5C
**Post date:** [May 2, 2022, 7:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fin-memory-dataset-to-disk-for-caching-operations%2F17413%2F1 "2022-05-02T07:37:20Z")

</div>

I am creating datasets from pandas dataframes a lot, as it’s simply an easier way to preserve columns that are arrays and such. However, I notice my .map operations aren’t getting cached.  
How can I turn an in memory dataset to a disk-based one for caching (or directly load a dataframe as such)

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [May 2, 2022, 12:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fin-memory-dataset-to-disk-for-caching-operations%2F17413%2F2 "2022-05-02T12:16:00Z")

</div>

Hi! In-memory datasets create a temporary cache bound to a python session. To cache operations permanently, save the dataset to disk with `.save_to_disk("path/to/save/dir")` and reload it with `datasets.load_from_disk("path/to/save/dir")` to get the version backed by an arrow file and then execute the ops on it again.