# No benefit from turning on gradient\_checkpointing: True

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fno-benefit-from-turning-on-gradient-checkpointing-true%2F87232%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 17, 2024, 10:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fno-benefit-from-turning-on-gradient-checkpointing-true%2F87232 "2024-05-17T22:41:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivekkaul](https://avatars.discourse-cdn.com/v4/letter/v/5fc32e/32.png) [@vivekkaul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvivekkaul)%5C
**Post date:** [May 17, 2024, 10:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fno-benefit-from-turning-on-gradient-checkpointing-true%2F87232%2F1 "2024-05-17T22:41:39Z")

</div>

I was LORA finetuning LLama 70b model and I turned on gradient\_checkpointing: True in my training config but it has no affect on the memory consumption at all and I don’t see any affect of the flag where I put it False or true. Any idea why that will the case ?

---

<div class="post-metadata">

**Author:** ![NinjaNegate](https://avatars.discourse-cdn.com/v4/letter/n/cab0a1/32.png) [@NinjaNegate](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNinjaNegate)%5C
**Post date:** [October 24, 2024, 4:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fno-benefit-from-turning-on-gradient-checkpointing-true%2F87232%2F2 "2024-10-24T16:28:15Z")

</div>

Isn’t it because LORA has trainable parameters in few tens of millions order and is not significant memory wise?