# BUGs on offset-mapping

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbugs-on-offset-mapping%2F88313%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [May 24, 2024, 4:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbugs-on-offset-mapping%2F88313 "2024-05-24T16:39:51Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ruiton](https://avatars.discourse-cdn.com/v4/letter/r/4af34b/32.png) [@Ruiton](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRuiton)%5C
**Post date:** [May 24, 2024, 4:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbugs-on-offset-mapping%2F88313%2F1 "2024-05-24T16:39:51Z")

</div>

Hi guys, I recently notice that the offset mapping returned by tokenizer seems to be problematic. I am working with the llama3-8b-Instruct model with dtype=fp16.

```auto
text='''Suppose A represents a certain relation. Infer the relation based on certain examples.'''
token_ranges = tokenizer(text,return_offsets_mapping=True)['offset_mapping'] 

```

and what it returns is:

```auto
[(0, 0), (0, 3), (3, 7), (7, 7), (9, 9), (20, 20), (22, 22), (30, 30), (39, 39), (40, 40), (46, 46), (50, 50), (59, 59), (65, 65), (68, 68), (76, 76), (85, 85)]

```

I assume that what it should return is the starting and end position of the token. Is there a bug or something that I misunderstand?