# How to include token-level prior into bert?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-include-token-level-prior-into-bert%2F17341%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 29, 2022, 7:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-include-token-level-prior-into-bert%2F17341 "2022-04-29T19:33:54Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![josephgatto](https://avatars.discourse-cdn.com/v4/letter/j/b38774/32.png) [@josephgatto](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjosephgatto)%5C
**Post date:** [April 29, 2022, 7:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-include-token-level-prior-into-bert%2F17341%2F1 "2022-04-29T19:33:54Z")

</div>

Is there a way to explicitly tell BERT to pay more attention to certain tokens given some prior information I have about the data?