# DeBERTa absolute Positions

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeberta-absolute-positions%2F3671%3E%5C
**Category:** Beginners\
**Created:** [February 16, 2021, 8:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeberta-absolute-positions%2F3671 "2021-02-16T20:27:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![joaogui1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/joaogui1/32/1024_2.png) [@joaogui1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjoaogui1)%5C
**Post date:** [February 16, 2021, 8:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeberta-absolute-positions%2F3671%2F1 "2021-02-16T20:27:05Z")

</div>

Hi! I was taking a look at Hugging Face’s DeBERTa [implementation](https://github.com/huggingface/transformers/blob/master/src/transformers/models/deberta/modeling_deberta.py), but I couldn’t find where in the code are the absolute positions being added (they should come before the softmax layer), can someone point me towards the line where that is being done?  
Thank you for your time 🙂

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [February 16, 2021, 8:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeberta-absolute-positions%2F3671%2F2 "2021-02-16T20:30:21Z")

</div>

cc @lysandre maybe?

---

<div class="post-metadata">

**Author:** ![gurvinder](https://avatars.discourse-cdn.com/v4/letter/g/f08c70/32.png) [@gurvinder](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgurvinder)%5C
**Post date:** [April 15, 2021, 8:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdeberta-absolute-positions%2F3671%2F3 "2021-04-15T20:49:35Z")

</div>

same here, wondering about absolute positions. Also @lysandre can you explain what the [z\_steps](https://github.com/huggingface/transformers/blob/2550b41aa2ec34f05ddfd3ec5875ddb32ad78d58/src/transformers/models/deberta/modeling_deberta.py#L949) suppose to do in Debertav2.

Thanks a lot 🙂