# Should cls\_token be \[CLS\] or \<cls\>?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [October 11, 2023, 7:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140 "2023-10-11T07:48:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![raptorkwok](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/raptorkwok/32/26481_2.png) [@raptorkwok](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fraptorkwok)%5C
**Post date:** [October 11, 2023, 7:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140%2F1 "2023-10-11T07:48:50Z")

</div>

If I want to create a tokenizer from scratch, should the special tokens use square brackets with uppercase letters, or angle brackets with lowercase letters?

That is, should cls\_token be [CLS] or `<cls>`?

`BertTokenizerFast` uses the former one:

> <https://github.com/huggingface/transformers/blob/599db139f921f3af535052c860cb685cadae6fcd/src/transformers/tokenization_bert.py#L547>

while many other examples on the Internet uses the latter one.

---

<div class="post-metadata">

**Author:** ![Sandy1857](https://avatars.discourse-cdn.com/v4/letter/s/ecae2f/32.png) [@Sandy1857](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSandy1857)%5C
**Post date:** [October 11, 2023, 8:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140%2F2 "2023-10-11T08:34:53Z")

</div>

It could be anything you want it to be since you’re making the tokenizer from scratch.

---

<div class="post-metadata">

**Author:** ![raptorkwok](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/raptorkwok/32/26481_2.png) [@raptorkwok](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fraptorkwok)%5C
**Post date:** [October 11, 2023, 8:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140%2F3 "2023-10-11T08:44:05Z")

</div>

thanks. Also, is `bos_token` and `eos_token` necessary? In Bert’s implementation, these two tokens are not defined.

---

<div class="post-metadata">

**Author:** ![Sandy1857](https://avatars.discourse-cdn.com/v4/letter/s/ecae2f/32.png) [@Sandy1857](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSandy1857)%5C
**Post date:** [October 11, 2023, 9:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fshould-cls-token-be-cls-or-cls%2F58140%2F4 "2023-10-11T09:23:19Z")

</div>

It’s not needed for Bert, which does not generate text auto-regressively. So it depends on your application I guess.