# Dataset too large error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-too-large-error%2F33844%3E%5C
**Category:** 🤗Datasets\
**Created:** [March 15, 2023, 8:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-too-large-error%2F33844 "2023-03-15T08:34:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![barry556652](https://avatars.discourse-cdn.com/v4/letter/b/b77776/32.png) [@barry556652](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbarry556652)%5C
**Post date:** [March 15, 2023, 8:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-too-large-error%2F33844%2F1 "2023-03-15T08:34:38Z")

</div>

![image](https://us1.discourse-cdn.com/hellohellohello/original/2X/c/ca31b880a60636481a85fcc8091a1a0d77facef1.png)  
If my images dataset more than 20000, always give me this error.  
ArrowInvalid: JSON parse error: Column() changed from object to string in row 0  
Can somebody help me?QAQ

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [March 15, 2023, 2:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-too-large-error%2F33844%2F2 "2023-03-15T14:11:12Z")

</div>

Hi! I think the problem is the format of the metadata file rather than the dataset size. You can test this with the following code:

```python
import pyarrow.json as paj
table = paj.read_json("path/to/metadata_file")

```

You can find the correct metadata format [here](https://huggingface.co/docs/datasets/image_dataset#imagefolder).