# Limitations of iterable datasets

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flimitations-of-iterable-datasets%2F16794%3E%5C
**Category:** 🤗Datasets\
**Created:** [April 13, 2022, 1:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flimitations-of-iterable-datasets%2F16794 "2022-04-13T13:54:45Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![conceptofmind](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/conceptofmind/32/55565_2.png) [@conceptofmind](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fconceptofmind)%5C
**Post date:** [May 12, 2022, 11:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flimitations-of-iterable-datasets%2F16794%2F5 "2022-05-12T23:25:07Z")

</div>

Hi @adrienchaton ,

I had noticed something similar as well with spiking convergence when training with streamed data and an Iterable Dataset vs a non-streamed non-iterable local dataset.

> [@adrienchaton](#):
>
> Then I am afraid there is no reshuffling of the dataset during training … am I right ?

It may be worth checking out whether using `ShufflerIterDataPipe()` to shuffle the batches in the Iterable data loader will help to resolve your issue.

For example something like this:

```auto
from torch.utils.data.datapipes.iter.combinatorics import ShufflerIterDataPipe

shuffled_batches = ShufflerIterDataPipe(your_torch_dataset)

train_dataloader = DataLoader(shuffle_batches, shuffle = True, batch_size = 8)

```

I have been working through it with the Hugging Face team and documenting my results in this thread: [Streaming Dataset of Sequence Length 2048 - #7 by loubnabnl](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fstreaming-dataset-of-sequence-length-2048%2F17649%2F7)

Hope this will help.

Best.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flimitations-of-iterable-datasets%2F16794)._