# Fine-tuning T5 with Trainer for novel task

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-t5-with-trainer-for-novel-task%2F5501%3E%5C
**Category:** Models\
**Created:** [April 14, 2021, 4:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-t5-with-trainer-for-novel-task%2F5501 "2021-04-14T16:34:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jdoe](https://avatars.discourse-cdn.com/v4/letter/j/eb8c5e/32.png) [@jdoe](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjdoe)%5C
**Post date:** [April 14, 2021, 4:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-t5-with-trainer-for-novel-task%2F5501%2F1 "2021-04-14T16:34:13Z")

</div>

Hi guys,

I am trying to fine-tune T5 with Huggingface’s Trainer class, trying to recycle as much training code as possible.

Yet I am wondering what the `Trainer.train()` method actually does. In the [T5 paper](https://www.semanticscholar.org/paper/Exploring-the-Limits-of-Transfer-Learning-with-a-Raffel-Shazeer/3cfb319689f06bf04c2e28399361f414ca32c4b3) the authors mention three fine-tuning methods that they used (§3.5.1):

- training only additional adapter layers
- gradually unfreezing the model and training more and more layers
- training the whole model right away

What does the Huggingface `Trainer.train()` do? And is there a simple way of switching between strategies?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [September 1, 2021, 2:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-t5-with-trainer-for-novel-task%2F5501%2F2 "2021-09-01T14:00:35Z")

</div>

By default, `Trainer.train()` will train the entire model (i.e. all layers).

To freeze certain layers before training, you can do that as follows:

```auto
for name, param in model.named_parameters():
     if name == "...":
        param.requires_grad = False

```