# T5 training with Trainer, w/ AdaFactor

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-training-with-trainer-w-adafactor%2F31573%3E%5C
**Category:** 🤗Transformers\
**Created:** [February 12, 2023, 9:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-training-with-trainer-w-adafactor%2F31573 "2023-02-12T21:37:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ndvb](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ndvb/32/17239_2.png) [@ndvb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fndvb)%5C
**Post date:** [February 12, 2023, 9:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5-training-with-trainer-w-adafactor%2F31573%2F1 "2023-02-12T21:37:01Z")

</div>

All is well when I have my own training loop, however when I try to move to using Trainer - the loss doesn’t decrease.

Every step, it prints something like:  
{‘loss’: 11.3911, ‘learning\_rate’: 0.0, ‘epoch’: 0.01}

Now, I am using Adafactor. I do:

```auto
optimizer = Adafactor(model.parameters(), lr=0.001, eps=(1e-30, 1e-3), clip_threshold=1.0, decay_rate=-0.8,
						  beta1=None, weight_decay=0.0, scale_parameter=False, relative_step=False,
						  warmup_init=False)

```

Then I don’t specify the optimizer to the TrainingArguments (because I am specifying it later in the Trainer, hope I am not wrong here), and then in the Trainer() I pass:

`optimizers=(optimizer, lr_scheduler)`