# Uploading image dataset to Huggingface Hub

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fuploading-image-dataset-to-huggingface-hub%2F24348%3E%5C
**Category:** 🤗Datasets\
**Created:** [October 12, 2022, 4:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fuploading-image-dataset-to-huggingface-hub%2F24348 "2022-10-12T16:12:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ejcho623](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ejcho623](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fejcho623)%5C
**Post date:** [October 12, 2022, 4:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fuploading-image-dataset-to-huggingface-hub%2F24348%2F1 "2022-10-12T16:12:32Z")

</div>

Hi, I am trying to create an image dataset (training only) and upload it on HuggingFace Hub. The data has two columns: 1) the image, and 2) the description text, aka, label. Essentially I’m trying to upload something similar like [this](https://huggingface.co/datasets/lambdalabs/pokemon-blip-captions).

I am using the ImageFolder approach and have my data folder structured as such:

> `metadata.jsonl`  
> `data/train/image_1.png`  
> `data/train/image_2.png`  
> `data/train/image_3.png`  
> `data/train/image_4.png`  
> `...`

In the `metadata.jsonl` file I’ve added the labels for the images as mentioned [here](https://huggingface.co/docs/datasets/image_dataset):

> {“file\_name”: “image\_1.png”, “text”: “some description about image 1”}  
> {“file\_name”: “image\_2.png”, “text”: “some description about image 2”}  
> {“file\_name”: “image\_3.png”, “text”: “some description about image 3”}  
> {“file\_name”: “image\_4.png”, “text”: “some description about image 4”}

My script to upload is simple, and looks something like this:

> from datasets import load\_dataset  
> dataset = load\_dataset(“imagefolder”, data\_dir=“data”, split=“train”)  
> dataset.push\_to\_hub(“ejcho623/undraw-raw”)

When I run the script, strangely enough it seems to only push 1 image? I see a `data/train-xxxx.parquet` and a `dataset_infos.json` file generated on my repo, but clearly (given the size) it has not uploaded the full dataset in my local directory (1000+ images). Here is the result of the command

> ejcho@ejs-macbook-pro undraw-raw % python3 push\_images.py  
> Using custom data configuration default-e70837628f6a2c62  
> Found cached dataset imagefolder (/Users/ejcho/.cache/huggingface/datasets/imagefolder/default-e70837628f6a2c62/0.0.0/37fbb85cc714a338bea574ac6c7d0b5be5aff46c1862c1989b20e0771199e93f)  
> 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00\<00:00, 68.35ba/s]  
> Pushing dataset shards to the dataset hub: 100%|████████████████████████████████████████████████████████| 1/1 [00:02\<00:00, 2.27s/it]

Would anyone be able to help to see what’s going on?

Thanks

---

<div class="post-metadata">

**Author:** ![ejcho623](https://avatars.discourse-cdn.com/v4/letter/e/b3f665/32.png) [@ejcho623](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fejcho623)%5C
**Post date:** [October 12, 2022, 6:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fuploading-image-dataset-to-huggingface-hub%2F24348%2F2 "2022-10-12T18:32:46Z")

</div>

It seems like there were some files names that the `load_dataset` didn’t like and would fail on. Once I removed those files (e.g. a file named “personal\_training.png”) it worked.

Not sure if there is some issue in the parser and conflicts those text with certain keywords

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [October 14, 2022, 1:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fuploading-image-dataset-to-huggingface-hub%2F24348%2F3 "2022-10-14T13:26:18Z")

</div>

Hi! Very strange indeed. Do you mind sharing the image folder with us (with dummy images if you want to keep them private), so we can get to the root of this problem?