# Claritifcation about the \`max\_position\_embeddings\` argument

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fclaritifcation-about-the-max-position-embeddings-argument%2F16250%3E%5C
**Category:** 🤗Transformers\
**Created:** [March 30, 2022, 10:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fclaritifcation-about-the-max-position-embeddings-argument%2F16250 "2022-03-30T10:26:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![carted-ml](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/carted-ml/32/4381_2.png) [@carted-ml](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcarted-ml)%5C
**Post date:** [March 30, 2022, 10:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fclaritifcation-about-the-max-position-embeddings-argument%2F16250%2F1 "2022-03-30T10:26:28Z")

</div>

Referring to this Colab Notebook:

> **[Google Colaboratory](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb)**

While initializing the `RobertaConfig`:

```nohighlight
config = RobertaConfig(
    vocab_size=52_000,
    max_position_embeddings=514,
    num_attention_heads=12,
    num_hidden_layers=6,
    type_vocab_size=1,
)

```

Why do we set `max_position_embeddings` to 514 when the maximum sequence length is set to 512 in the notebook?

---

<div class="post-metadata">

**Author:** ![Rushikesh](https://avatars.discourse-cdn.com/v4/letter/r/6f9a4e/32.png) [@Rushikesh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRushikesh)%5C
**Post date:** [January 27, 2023, 8:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fclaritifcation-about-the-max-position-embeddings-argument%2F16250%2F2 "2023-01-27T08:45:16Z")

</div>

Hey this may be late, but the other two are sep\_tokens at the start and end of the tensor list.