# Load\_from\_disk and read-only filesystem
**URL:**
**Author:**  [@tsobolev](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftsobolev)%5C
**Post date:** [September 10, 2023, 1:59pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F1 "2023-09-10T13:59:13Z")
Hello,
I attempted to use my own dataset on Kaggle, which I created using `save_to_disk` and uploaded as a Kaggle dataset. I can load this dataset with `load_from_disk('/kaggle/input/my_dataset_dir')`, but I can’t perform operations like ‘train\_test\_split’ because of the read-only filesystem:
```auto
OSError: [Errno 30] Read-only file system: '/kaggle/input/my_dataset_dir/tmp5x3dj6no'
```
When I try loading it with `load_dataset(streaming=True)`, I encounter an error:
```auto
ValueError: You are trying to load a dataset that was saved using `save_to_disk`. Please use `load_from_disk` instead.
```
What should I do to use my datasets on a read-only filesystem?
---
**Author:**  [@ReatKay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FReatKay)%5C
**Post date:** [September 10, 2023, 4:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F2 "2023-09-10T16:22:25Z")
A complete read-only filesystem? So no possibility of any form of disk based caching on the whole environment? Not sure if it truly works, but, logically this approach should work:
```auto
# Disable datasets caching
from datasets import set_caching_enabled
set_caching_enabled(False)
# Load from disk and keep fully in memory
load_from_disk(path, keep_in_memory=True)
```
If you have to do extensive preparations / mapping and run into mem issues (beside doing the splitting), process/map the dataset to final usability and remove any not required columns BEFORE saving to disk and uploading to the readonly filesystem.
If that doesnt work: Split and then save and upload.
---
**Author:**  [@tsobolev](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftsobolev)%5C
**Post date:** [September 10, 2023, 4:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F3 "2023-09-10T16:45:08Z")
The entire filesystem is in read-write mode, while the datasets directory at /kaggle/input/\*, which is likely mounted via NFS, is in read-only mode.
---
**Author:**  [@ReatKay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FReatKay)%5C
**Post date:** [September 10, 2023, 4:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F4 "2023-09-10T16:50:04Z")
> [@tsobolev](#):
>
> tory at /kaggle/input/\*, which is likely mounted via NFS, is in read-only mode.
Try it with disabled caching and keep in memory. I can’t think of any other option, beside loading an already fully processed/mapped dataset.
---
**Author:**  [@tsobolev](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftsobolev)%5C
**Post date:** [September 11, 2023, 11:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F5 "2023-09-11T23:28:26Z")
This was a bad idea to use this format for storing the dataset on a kaggle drive. Loading is very slow. Using `load_dataset()` with data\_files works much better.
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [September 21, 2023, 10:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fload-from-disk-and-read-only-filesystem%2F54312%2F6 "2023-09-21T10:34:41Z")
You can specify where the train/test row indices are saved on disk by specifying the `train_indices_cache_file_name` and `test_indices_cache_file_name` arguments to `train_test_split`.
Feel free to choose a location where you have write access.
This is the same for other methods like map, filter etc. where you can specify `cache_file_name`