# How to apply decoding method and penalty

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-decoding-method-and-penalty%2F66765%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 22, 2023, 9:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-decoding-method-and-penalty%2F66765 "2023-12-22T21:32:48Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pivitin](https://avatars.discourse-cdn.com/v4/letter/p/ecccb3/32.png) [@Pivitin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPivitin)%5C
**Post date:** [December 22, 2023, 9:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-decoding-method-and-penalty%2F66765%2F1 "2023-12-22T21:32:48Z")

</div>

I’m trying to apply some parameters in my code below, anyone know how to apply them? Is it possible in google/flan-t5-xl model?

parameters I want to apply:

```auto
{
    "decoding_method": "greedy",
    "max_new_tokens": 5,
    "repetition_penalty": 1
}

```

Code from google/flan-t5-xl model:

```auto
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-xl", decoding_method="greedy",)
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xl")
input_ids = tokenizer(prompt_input, return_tensors="pt", ).input_ids
outputs = model.generate(input_ids, max_new_tokens=5)
print(f"Sentimental: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")

```

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 23, 2023, 11:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-apply-decoding-method-and-penalty%2F66765%2F2 "2023-12-23T11:23:53Z")

</div>

Hi,

Sure. For decoding, the `generate` method can be used, and it uses greedy decoding by default, so that’s ok. You can pass the additional arguments as keyword arguments:

```auto
from transformers import T5Tokenizer, T5ForConditionalGeneration
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-xl", decoding_method="greedy",)
model = T5ForConditionalGeneration.from_pretrained("google/flan-t5-xl")
input_ids = tokenizer(prompt_input, return_tensors="pt", ).input_ids

generation_kwargs = {"max_new_tokens": 5, "repetition_penalty": 1}
outputs = model.generate(input_ids, **generation_kwargs)
print(f"Sentimental: {tokenizer.decode(outputs[0], skip_special_tokens=True)}")

```
