# Freezing mt5 model for fine-tuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreezing-mt5-model-for-fine-tuning%2F45489%3E%5C
**Category:** Models\
**Created:** [July 3, 2023, 4:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreezing-mt5-model-for-fine-tuning%2F45489 "2023-07-03T16:57:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![soroushheidary](https://avatars.discourse-cdn.com/v4/letter/s/8c91f0/32.png) [@soroushheidary](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsoroushheidary)%5C
**Post date:** [July 3, 2023, 4:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreezing-mt5-model-for-fine-tuning%2F45489%2F1 "2023-07-03T16:57:16Z")

</div>

Hey, I wanted to fine tune a mt5-base model for my project (machine translation) and when I try to freeze the parameters except the language head, I’ll get errors: can any one helps me know why this is the case  
(also I read in the Docs that fine tuning transformers often yield better results when updating every parameter and not freezing, but I don’t have enough processing power for that, and is it even the case?)

this is how I freeze the parameters:

```auto
model = T5ForConditionalGeneration.from_pretrained("t5-small")
tokenizer = T5TokenizerFast.from_pretrained("t5-small")

for param in model.parameters():
    param.requires_grad = False
model.lm_head.requires_grad = True

training_args = TrainingArguments(
    output_dir="mt5-finetuned",
    num_train_epochs=3,
    per_device_train_batch_size=32, ## lower batch sizes
    per_device_eval_batch_size=32, ## lower batch sizes
    evaluation_strategy="epoch",
    learning_rate=5e-4,
    weight_decay=0.01,
    save_total_limit=3,
    # fp16=True, ## lower precision
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,#["train"],
    eval_dataset=dataset,#["validation"],
)

trainer.train()

```

But I get this error:

```auto
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

```

Thank everyone in advance 🤗

---

<div class="post-metadata">

**Author:** ![akdo00001](https://avatars.discourse-cdn.com/v4/letter/a/34f0e0/32.png) [@akdo00001](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fakdo00001)%5C
**Post date:** [July 15, 2023, 4:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreezing-mt5-model-for-fine-tuning%2F45489%2F2 "2023-07-15T16:05:23Z")

</div>

Hello,

Were you able to solve the issue? I am also getting the same error, when I run [nielsr/codet5-small-code-summarization-ruby · Hugging Face](https://huggingface.co/nielsr/codet5-small-code-summarization-ruby) @nielsr