# Repetitive words in model output

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frepetitive-words-in-model-output%2F132085%3E%5C
**Category:** Models\
**Created:** [December 18, 2024, 11:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frepetitive-words-in-model-output%2F132085 "2024-12-18T11:46:11Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tempdeltavalue](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/tempdeltavalue/32/28454_2.png) [@tempdeltavalue](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftempdeltavalue)%5C
**Post date:** [December 18, 2024, 11:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frepetitive-words-in-model-output%2F132085%2F1 "2024-12-18T11:46:11Z")

</div>

```auto
import torch 
from transformers import GPT2Tokenizer, GPT2LMHeadModel

model_name = "gpt2" #output_dir #
tokenizer = GPT2Tokenizer.from_pretrained(model_name, padding_side='left', add_eos_token=True) #gpt2
llm_model = GPT2LMHeadModel.from_pretrained(model_name) # gpt2
device = torch.device('cuda')
llm_model.to(device)

tokenizer.pad_token = tokenizer.eos_token

# Configure the model
llm_model.config.pad_token_id = tokenizer.eos_token_id
llm_model.config.eos_token_id = tokenizer.eos_token_id
llm_model.config.vocab_size = llm_model.config.vocab_size + len(tokenizer.get_added_vocab())
llm_model.resize_token_embeddings(len(tokenizer))
llm_model.config.pad_token_id = tokenizer.pad_token_id

print(tokenizer.eos_token_id)

```

result

 ![Screenshot 2024-12-18 140918](https://us1.discourse-cdn.com/hellohellohello/original/3X/9/0/90c90cbc8b80b7fbb74ce17f5d825e615f4d28a3.png)

github issue

> <https://github.com/huggingface/tokenizers/issues/1704>
>
> code here:
> https://github.com/tempdeltavalue/temp\_l/blob/main/finetune\_seq2seq.…ipynb
> 
> !\[Screenshot 2024-12-18 150037\](https://github.com/user-attachments/assets/64197b78-215f-4271-9737-6a8d755c7afe)
> !\[Screenshot 2024-12-18 150203\](https://github.com/user-attachments/assets/10b803c1-2dd3-45b1-b831-c5258aa0667a)
> 
> 
> /static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frepetitive-words-in-model-output%2F132085%2F2

---

<div class="post-metadata">

**Author:** ![tempdeltavalue](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/tempdeltavalue/32/28454_2.png) [@tempdeltavalue](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftempdeltavalue)%5C
**Post date:** [December 18, 2024, 1:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frepetitive-words-in-model-output%2F132085%2F2 "2024-12-18T13:02:27Z")

</div>

![Screenshot 2024-12-18 150037](https://us1.discourse-cdn.com/hellohellohello/original/3X/7/6/76b570b878f81fdfd10c06c0c3cf2e0cffa87e5b.png)  
 ![Screenshot 2024-12-18 150203](https://us1.discourse-cdn.com/hellohellohello/original/3X/a/3/a36775e444e12150ab23e32d43329b87aaa6ff1c.png)