# BART max\_new\_tokens in generate function

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-max-new-tokens-in-generate-function%2F86242%3E%5C
**Category:** Models\
**Created:** [May 11, 2024, 1:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-max-new-tokens-in-generate-function%2F86242 "2024-05-11T13:34:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![vsocrates](https://avatars.discourse-cdn.com/v4/letter/v/90db22/32.png) [@vsocrates](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvsocrates)%5C
**Post date:** [May 11, 2024, 1:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-max-new-tokens-in-generate-function%2F86242%2F1 "2024-05-11T13:34:36Z")

</div>

I’m using a fine-tuned `BartForConditionalGeneration` model and trying to generate tokens.

```python
  outputs = model.generate(input_ids, attention_mask=attention_mask, num_beams=3, 
                           min_new_tokens=1500,
                           max_new_tokens=2500,
                           early_stopping=True)

```

However, I get an error that `This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.`

CUDA also throws a `RunTimeError`.

Why? Shouldn’t BART be able to arbitrarily long sequences autoregressively?

---

<div class="post-metadata">

**Author:** ![RaushanTurganbay](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@RaushanTurganbay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRaushanTurganbay)%5C
**Post date:** [May 11, 2024, 6:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-max-new-tokens-in-generate-function%2F86242%2F2 "2024-05-11T18:26:38Z")

</div>

Hey!

No, Bart cannot arbitrarily increase the max length since it has been trained with learned positional encodings, unlike the recent LLMs that use RoPE. In other words, Bart has a fixed size matrix that computes position embeddings.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [May 13, 2024, 4:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbart-max-new-tokens-in-generate-function%2F86242%2F3 "2024-05-13T16:57:36Z")

</div>

Closed on May 13, 2024, 4:57pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.