# Infinity output from gpt2 model?
**URL:**
**Author:**  [@HuangLED](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHuangLED)%5C
**Post date:** [June 22, 2024, 6:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finfinity-output-from-gpt2-model%2F93470%2F1 "2024-06-22T06:16:56Z")
New to hugging face. Trying to use gpt2 model to generate some text, why the following code yields infinity result?
chatgpt says it might due to EOS not correct. I tried a few but does not help.
> import torch
> from transformers import GPT2LMHeadModel, GPT2Tokenizer
>
> model = GPT2LMHeadModel.from\_pretrained(“gpt2”, torchscript=True).eval()
>
> # tokenizer
>
> tokenizer = GPT2Tokenizer.from\_pretrained(“gpt2”)
> in\_text = “Lionel Messi is something”
> in\_tokens = torch.tensor(tokenizer.encode(in\_text))
>
> # inference
>
> #token\_eos = torch.tensor([198]) # line break symbol
> token\_eos = torch.tensor([50256])
> out\_token = None
> i = 0
> with torch.no\_grad():
> while out\_token != token\_eos:
> logits, \_ = model(in\_tokens)
> out\_token = torch.argmax(logits[-1, :], dim=0, keepdim=True)
> in\_tokens = torch.cat((in\_tokens, out\_token), 0)
> text = tokenizer.decode(in\_tokens)
> print(f’step {i} input: {text}', flush=True)
> i += 1
>
> out\_text = tokenizer.decode(in\_tokens)
> print(f’ Input: {in\_text}‘)
> print(f’Output: {out\_text}’)
---
**Author:**  [@arnoldmatt](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Farnoldmatt)%5C
**Post date:** [June 22, 2024, 2:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finfinity-output-from-gpt2-model%2F93470%2F2 "2024-06-22T14:17:39Z")
Hey HuangLED,
It looks like your code might be getting stuck in an infinite loop because `out_token` may never match `token_eos`. You can try adding a maximum iteration limit to your loop to prevent it from running indefinitely. Here’s a modified version of your loop:
```auto
max_iterations = 100 # or any suitable number
i = 0
while out_token != token_eos and i < max_iterations:
logits, _ = model(in_tokens.unsqueeze(0)) # Add batch dimension
out_token = torch.argmax(logits[0, -1, :], dim=0, keepdim=True)
in_tokens = torch.cat((in_tokens, out_token), 0)
text = tokenizer.decode(in_tokens)
print(f'step {i} input: {text}', flush=True)
i += 1
```
---
**Author:**  [@HuangLED](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FHuangLED)%5C
**Post date:** [June 22, 2024, 6:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finfinity-output-from-gpt2-model%2F93470%2F3 "2024-06-22T18:28:54Z")
Thanks for the discussion, Matt.
I am not sure if it is the right thing to do. A hard cut off like this, will just lead to an incomplete sentence at last.
That does not seem a nature response to get I assume?