# GPT-2 special tokens
**URL:**
**Author:**  [@theodp](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftheodp)%5C
**Post date:** [February 7, 2024, 4:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-special-tokens%2F72296%2F1 "2024-02-07T16:29:34Z")
Hello,
Currently working with GPT-2, I am fine-tuning a model on Next Token Generation task in order to perform text generation at inference from an image.
During training, I manually add special token at the beginning of the sentence (BOS) and at the end (EOS). So at inference, I start with (BOS) token and let the model generate.
input\_ids = [gpt2.bos\_token\_id] + tokens[‘input\_ids’] + [gpt2.eos\_token\_id]
However, I realized that [gpt2.bos\_token\_id] and [gpt2.eos\_token\_id] have the same ID (0 : ‘\<|endoftext|\>’) so there are the same tokens. Why is it done like it ? Not like in BERT with different tokens for BOS and EOS.
Morever, is it a problem for my generation at inference ?
Thank you !
---
**Author:**  [@theodp](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftheodp)%5C
**Post date:** [February 20, 2024, 1:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-special-tokens%2F72296%2F2 "2024-02-20T13:39:26Z")
Update: It is not a problem !
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [February 21, 2024, 1:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt-2-special-tokens%2F72296%2F3 "2024-02-21T01:39:32Z")
Closed on February 21, 2024, 1:39am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.