# Regarding input\_ids and labels while grouping texts

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373%3E%5C
**Category:** 🤗Optimum\
**Created:** [April 23, 2024, 6:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373 "2024-04-23T18:04:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![palashai](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@palashai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpalashai)%5C
**Post date:** [April 23, 2024, 6:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373%2F1 "2024-04-23T18:04:50Z")

</div>

Hi @regisss

I see input\_ids and labels to be of same tensors [here](https://github.com/huggingface/optimum-habana/blob/081130d53a1237e5b8b5d7ac1c64195300579303/examples/language-modeling/run_clm.py#L569)

Is it a mistake? Or internally in GaudiTrainer the shifting of labels happens?

---

<div class="post-metadata">

**Author:** ![regisss](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/regisss/32/22467_2.png) [@regisss](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fregisss)%5C
**Post date:** [April 23, 2024, 8:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373%2F2 "2024-04-23T20:11:53Z")

</div>

Hi @palashai, it happens in the forward of the model. For instance for Llama: [optimum-habana/optimum/habana/transformers/models/llama/modeling\_llama.py at 8c8ec55d8030f1df05288407b2e687a7664d7395 · huggingface/optimum-habana · GitHub](https://github.com/huggingface/optimum-habana/blob/8c8ec55d8030f1df05288407b2e687a7664d7395/optimum/habana/transformers/models/llama/modeling_llama.py#L943)

---

<div class="post-metadata">

**Author:** ![palashai](https://avatars.discourse-cdn.com/v4/letter/p/977dab/32.png) [@palashai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpalashai)%5C
**Post date:** [April 24, 2024, 5:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373%2F3 "2024-04-24T05:38:01Z")

</div>

Got it. Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [April 24, 2024, 5:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-input-ids-and-labels-while-grouping-texts%2F83373%2F4 "2024-04-24T17:38:12Z")

</div>

Closed on April 24, 2024, 5:38pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.