# Learning rate setting

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flearning-rate-setting%2F2063%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 14, 2020, 4:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flearning-rate-setting%2F2063 "2020-11-14T04:16:57Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![guoziyuan](https://avatars.discourse-cdn.com/v4/letter/g/7ab992/32.png) [@guoziyuan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fguoziyuan)%5C
**Post date:** [November 14, 2020, 4:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flearning-rate-setting%2F2063%2F1 "2020-11-14T04:16:58Z")

</div>

Hey, I wonder is it possible to set different learning rates for different parts in the model ？This is ususlly considered as a trick in bert fine-tuning.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 16, 2020, 1:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flearning-rate-setting%2F2063%2F2 "2020-11-16T13:58:14Z")

</div>

You can pass your own optimizer to the Trainer, in which you can define your parameter groups as you like.
