# Question about gradient checkpointing and use\_cache
**URL:**
**Author:**  [@WenyangHui](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWenyangHui)%5C
**Post date:** [July 9, 2024, 6:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-gradient-checkpointing-and-use-cache%2F96301%2F1 "2024-07-09T06:02:46Z")
I want to use the past\_key\_values to compute some losses during training, but gradient checkpointing forbids use\_cache in CLMs. I wonder why there is such a conflict.
---
**Author:**  [@RaushanTurganbay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRaushanTurganbay)%5C
**Post date:** [July 9, 2024, 2:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-gradient-checkpointing-and-use-cache%2F96301%2F2 "2024-07-09T14:12:44Z")
I think it’s because gradient checkpointing forces us to do several forward passes to compute the gradients again. If we have use past\_key\_values in such a scenario, the cache will be filled with repeting keys/values from extra forward calls.
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [July 10, 2024, 4:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-gradient-checkpointing-and-use-cache%2F96301%2F3 "2024-07-10T04:11:40Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.