# Create custom splits

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251%3E%5C
**Category:** 🤗Datasets\
**Created:** [February 22, 2023, 8:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251 "2023-02-22T08:32:14Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sl02](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@sl02](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsl02)%5C
**Post date:** [February 22, 2023, 8:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251%2F1 "2023-02-22T08:32:14Z")

</div>

I was looking at the `imdb` dataset script, and I noticed that it uses a custom split - “unsupervised”. However, in my custom dataset, a custom split throws a TypeError. How do I resolve this?

```auto
 Traceback (most recent call last):
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/bin/datasets-cli", line 8, in <module>
    sys.exit(main())
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/commands/datasets_cli.py", line 39, in main
    service.run()
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/commands/test.py", line 141, in run
    builder.download_and_prepare(
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/builder.py", line 860, in download_and_prepare
    self._download_and_prepare(
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/builder.py", line 953, in _download_and_prepare
    self._prepare_split(split_generator, **prepare_split_kwargs)
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/builder.py", line 1669, in _prepare_split
    split_info = self.info.splits[split_generator.name]
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/splits.py", line 530, in __getitem__
    instructions = make_file_instructions(
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/arrow_reader.py", line 112, in make_file_instructions
    name2filenames = {
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/arrow_reader.py", line 113, in <dictcomp>
    info.name: filenames_for_dataset_split(
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/naming.py", line 71, in filenames_for_dataset_split
    prefix = filename_prefix_for_split(dataset_name, split)
  File "/Users/home/.local/share/virtualenvs/JR745-rx6Pdnnr/lib/python3.8/site-packages/datasets/naming.py", line 54, in filename_prefix_for_split
    if os.path.basename(name) != name:
  File "/Users/home/anaconda3/lib/python3.8/posixpath.py", line 142, in basename
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

```

---

<div class="post-metadata">

**Author:** ![polinaeterna](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/polinaeterna/32/19055_2.png) [@polinaeterna](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpolinaeterna)%5C
**Post date:** [February 22, 2023, 6:59pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251%2F2 "2023-02-22T18:59:42Z")

</div>

hi @sl02 ! this looks similar to this issue: [Adding new splits to a dataset script with existing old splits info in metadata's `dataset_info` fails · Issue #5315 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues/5315)

have you generated `README.md` file for your dataset?

---

<div class="post-metadata">

**Author:** ![sl02](https://avatars.discourse-cdn.com/v4/letter/s/ba9def/32.png) [@sl02](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsl02)%5C
**Post date:** [February 23, 2023, 6:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251%2F3 "2023-02-23T06:14:46Z")

</div>

@polinaeterna  
yes I did generate a README.md in the first iteration.

After removing the README.md, and updating the script, it runs fine.  
Thank you!

---

<div class="post-metadata">

**Author:** ![JoaoLages](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@JoaoLages](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJoaoLages)%5C
**Post date:** [June 1, 2023, 8:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-custom-splits%2F32251%2F4 "2023-06-01T08:02:01Z")

</div>

I solved this issue by setting a different `datasets.DatasetInfo.description` depending on the splits (or other arguments) I might have for that dataset.