# Datasets.map is not consistent with IterableDataset?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-map-is-not-consistent-with-iterabledataset%2F154912%3E%5C
**Category:** 🤗Datasets\
**Created:** [May 13, 2025, 3:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-map-is-not-consistent-with-iterabledataset%2F154912 "2025-05-13T15:00:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Agnellino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/agnellino/32/29000_2.png) [@Agnellino](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAgnellino)%5C
**Post date:** [May 13, 2025, 3:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-map-is-not-consistent-with-iterabledataset%2F154912%2F1 "2025-05-13T15:00:19Z")

</div>

Hi, I encounter an issue and I don’t know if I use badly the library or if there is a problem in it.

Here is the code :

```python
from datasets import load_dataset

instruction = "Write the LaTeX representation for this image."

def convert_to_conversation(sample):
    conversation = [
        { "role": "user",
        "content" : [
            {"type" : "text", "text" : instruction},
            {"type" : "image", "image" : sample["image"]} ]
        },
        { "role" : "assistant",
        "content" : [
            {"type" : "text", "text" : sample["text"]} ]
        },
    ]
    return { "messages" : conversation }

dataset = load_dataset(
        "unsloth/LaTeX_OCR", 
        split="train", 
        streaming=True
    )

item = next(iter(dataset))
item_converted = convert_to_conversation(item)
print(item_converted)

```

The result is the following :

```python
{'messages': [{'role': 'user', 'content': [{'type': 'text', 'text': 'Write the LaTeX representation for this image.'}, {'type': 'image', 'image': <PIL.PngImagePlugin.PngImageFile image mode=RGB size=160x40 at 0x7FE898A06D50>}]}, {'role': 'assistant', 'content': [{'type': 'text', 'text': '{ \\frac { N } { M } } \\in { \\bf Z } , { \\frac { M } { P } } \\in { \\bf Z } , { \\frac { P } { Q } } \\in { \\bf Z }'}]}]}

```

Whereas with the following code:

```python
dataset = load_dataset(
        "unsloth/LaTeX_OCR", 
        split="train", 
        streaming=True
    )
dataset = dataset.map(convert_to_conversation)

item = next(iter(dataset))
print(item)

```

Then the result shown is the following (I truncated the bytes…):

```python
{'image': {'bytes': b'\x89PNG\r\n\.....', 'path': '5208.png'}, 'text': '{ \\frac { N } { M } } \\in { \\bf Z } , { \\frac { M } { P } } \\in { \\bf Z } , { \\frac { P } { Q } } \\in { \\bf Z }', 'messages': [{'role': 'user', 'content': [{'type': 'text', 'text': 'Write the LaTeX representation for this image.'}, {'type': 'image', 'image': {'bytes': b'\x89PNG\r\n.....', 'path': '5208.png'}}]}, {'role': 'assistant', 'content': [{'type': 'text', 'text': '{ \\frac { N } { M } } \\in { \\bf Z } , { \\frac { M } { P } } \\in { \\bf Z } , { \\frac { P } { Q } } \\in { \\bf Z }'}]}]}

```

So basically, the image has been cast from a PIL Image into a datasets Image without being asked.

Is this normal ?

It seems that I can patch the type casting behavior by using `datasets.Image.decode_example` and remove additional keys from the dict using `remove_colums=["image", "text"]` in the `map` method.

Did I miss something ? What do I do wrong ?

Kind regards,

Tristan

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [May 14, 2025, 7:41am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdatasets-map-is-not-consistent-with-iterabledataset%2F154912%2F2 "2025-05-14T07:41:41Z")

</div>

I think that’s probably normal…  
[https://stackoverflow.com/questions/74251282/huggingface-datasets-storing-and-loading-image-data](https://stackoverflow.com/questions/74251282/huggingface-datasets-storing-and-loading-image-data)

> [@Turn of automatic Pil image generation in load\_dataset](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fturn-of-automatic-pil-image-generation-in-load-dataset%2F103222)%3A
>
> when loading a dataset, I have the following arrow format - label: int, image: struct\<bytes:binary, path:string\>. when using load\_dataset() method, the image is automatically converted to Pil image format, and the path is lost. is there a way to avoid that behavior? In my example - dataset = load\_dataset("chronopt-research/cropped-vggface2-224") for i in range(0, len(dataset['train']), batch\_size): batch = dataset['train'][i:i + batch\_size] images = batch['image'] # Original 224x224 i…

> [@How to return PIL.Image from dataset generator?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-return-pil-image-from-dataset-generator%2F31160)%3A
>
> I have a custom GeneratorBasedBuilder subclass that I’m trying to get running. It seems like I’m almost there, but my trainer is failing with a type error when trying to access a generated sample—to be more specific, I’m working with the train\_text\_to\_image.py script and it’s failing at the preprocess\_train(examples) function… From what I understand, this is because the actual data is a PIL.Image, but my generator’s features dict lists the features as "image": datasets.Value("string"), which is…