# Audio dataset without uploading the data to the hub
**URL:**
**Author:**  [@vimey](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvimey)%5C
**Post date:** [March 17, 2023, 9:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F1 "2023-03-17T09:36:44Z")
Hey there!
I am trying to create a custom audio dataset with local files: I have the audiofiles (mp3) and the corresponding metadata (json). I can’t upload the data to the huggingface hub, because it’s confidential. Does anyone know if there is a way of creating the audio set without uploading the data to the hub?
Thanks!
---
**Author:**  [@stevhliu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fstevhliu)%5C
**Post date:** [March 17, 2023, 5:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F2 "2023-03-17T17:05:44Z")
Hi! You can use the `AudioFolder` to quickly create the dataset, and make sure you have the metadata file in the same directory (check out the docs [here](https://huggingface.co/docs/datasets/main/en/audio_dataset#audiofolder) for more details!) 😀
```py
from datasets import load_dataset
dataset = load_dataset("audiofolder", data_dir="/path/to/data")
```
---
**Author:**  [@sriniu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsriniu)%5C
**Post date:** [March 19, 2023, 7:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F3 "2023-03-19T19:10:59Z")
Hi @vimey ,
were you able to acheive this?
if yes, Could you please guide how you achieved it ?
---
**Author:**  [@vimey](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvimey)%5C
**Post date:** [March 20, 2023, 7:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F4 "2023-03-20T07:27:48Z")
Hey @stevhliu and @sriniu,
unfortunately this solution didn’t work for me. I’ve tried this before, but I only got this as a result:
```auto
DatasetDict({
test: Dataset({
features: ['audio'],
num_rows: 1
})
})
```
But it should be two columns (“file\_name”, “transcription”) instead of “audio”.
---
**Author:**  [@sriniu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsriniu)%5C
**Post date:** [March 20, 2023, 8:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F5 "2023-03-20T08:28:48Z")
I did some checks yesterday and sort of worked. I am currently at training step and the training process fails due to PYTORCH\_CUDA\_ALLOC\_CONF error (which I am trying to resolve).
What did I do:
1. Arrange my data in a local folder with the names audio\_0.wav, audio\_1.wav… , audio\_N.wav.
2.
# df train was a pandas dataframe in my case with information like label for each audio file
audio\_dict = {}
audio\_dict[‘audio’] = [“path/to/audio\_1”, “path/to/audio\_2”, …, “path/to/audio\_n”]
audio\_dict[‘label’] = [x for x in df\_train.label]
audio\_dict[‘split’] = len(df\_train)\*[‘train’]
audio\_dataset = Dataset.from\_dict(audio\_dict).cast\_column(“audio”, Audio())
#note: for now i am not using the split column anywhere in my code, it can be omitted i think
once i got my audio\_dataset object, i replaced the “dataset” object with “audio\_Dataset” in this notebook, [Google Colab](https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/audio_classification.ipynb#scrollTo=niniUAnb5IrR)
---
**Author:**  [@vimey](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvimey)%5C
**Post date:** [March 20, 2023, 10:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F6 "2023-03-20T10:34:28Z")
Thanks for your approach! It looks really good 🙂
However, I solved my initial issue: I saved the metadata in a json file, which is wrong. You have to save it in a jsonl file. After I changed the file type, _audiofolder_ worked for me.
---
**Author:**  [@sriniu](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsriniu)%5C
**Post date:** [March 20, 2023, 12:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faudio-dataset-without-uploading-the-data-to-the-hub%2F34000%2F7 "2023-03-20T12:55:03Z")
Thats great to know!