# \`BertEmbeddings\` contains positional embedding?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbertembeddings-contains-positional-embedding%2F28442%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 26, 2022, 10:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbertembeddings-contains-positional-embedding%2F28442 "2022-12-26T10:36:15Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![NimaBoscarino](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nimaboscarino/32/4358_2.png) [@NimaBoscarino](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNimaBoscarino)%5C
**Post date:** [December 26, 2022, 7:19pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbertembeddings-contains-positional-embedding%2F28442%2F2 "2022-12-26T19:19:09Z")

</div>

Hello there! `BertEmbeddings` has the positional embeddings for you: [transformers/modeling\_bert.py at 31d452c68b34c2567b62924ee0df40a83cbc52d5 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/blob/31d452c68b34c2567b62924ee0df40a83cbc52d5/src/transformers/models/bert/modeling_bert.py#L187)

We use a vanilla nn.Embedding layer instead of the sin/cos positional encoding (more on that here: [Why positional embeddings are implemented as just simple embeddings? - #6 by yjernite](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwhy-positional-embeddings-are-implemented-as-just-simple-embeddings%2F585%2F6)). If you want to override that and use sinusoidal embeddings though, you can follow the tip here: [How to use custom positional embedding while fine tuning Bert - #2 by JinKing](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-use-custom-positional-embedding-while-fine-tuning-bert%2F7992%2F2)

Hope that helps! Cheers!

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbertembeddings-contains-positional-embedding%2F28442)._