# Generate() and automatic truncation of context

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-and-automatic-truncation-of-context%2F91995%3E%5C
**Category:** 🤗Transformers\
**Created:** [June 13, 2024, 1:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-and-automatic-truncation-of-context%2F91995 "2024-06-13T13:08:47Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![DonThomasitos](https://avatars.discourse-cdn.com/v4/letter/d/a5b964/32.png) [@DonThomasitos](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDonThomasitos)%5C
**Post date:** [June 13, 2024, 1:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-and-automatic-truncation-of-context%2F91995%2F1 "2024-06-13T13:08:47Z")

</div>

Hey guys, I have a question regarding language modeling:

I have a decoder-only model (Llama) and want to generate a sequence. During the sequence generation, I want to have a continuously “full” context, equivalent to the model’s maximum context size during training. To my understanding, this would involve truncating the first token after every single token generation.

However, I don’t find this functionality in `generate().` Instead, the MaxLengthCriteria assumes that for the generation of N new tokens, my context length is at max `(CONTEXT_LEN - N)`.

The “manual” approach would be to generate one token at a time with generate() and then cut the first token in the sequence every time. But I’m sure this can be done more elegantly?

Thank you.