# Gradient checkpointing without training

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgradient-checkpointing-without-training%2F47251%3E%5C
**Category:** Beginners\
**Created:** [July 18, 2023, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgradient-checkpointing-without-training%2F47251 "2023-07-18T20:51:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mike5645](https://avatars.discourse-cdn.com/v4/letter/m/3e96dc/32.png) [@Mike5645](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMike5645)%5C
**Post date:** [July 18, 2023, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgradient-checkpointing-without-training%2F47251%2F1 "2023-07-18T20:51:46Z")

</div>

I have a LlamaForCausalLM model. I want to do a single run of backprop on a single sample (one forward pass, one backward pass) and record all the gradients that are computed in the process. I do not want to actually update the model weights- I just want to record the gradients. The model is pretty big and I only have a single GPU, so to be able to do this I need to use gradient checkpointing. Is there a way to use a Trainer to accomplish this? Thanks!