# Different masks for encoder self and cross attention

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdifferent-masks-for-encoder-self-and-cross-attention%2F25749%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 8, 2022, 5:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdifferent-masks-for-encoder-self-and-cross-attention%2F25749 "2022-11-08T17:15:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![tarush14](https://avatars.discourse-cdn.com/v4/letter/t/e0b2c6/32.png) [@tarush14](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftarush14)%5C
**Post date:** [November 8, 2022, 5:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdifferent-masks-for-encoder-self-and-cross-attention%2F25749%2F1 "2022-11-08T17:15:43Z")

</div>

Hi, I was looking through the source code for the T5 model and I realized that the decoder uses the same encoder\_attention\_mask (for cross attention) as the input attention mask supplied for self attention in the encoder. Is it possible to supply different masks for these two cases as I am trying to build a knowledge mechanism that requires this.

Thank you.