# Finetuning and single-GPU utilization

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-and-single-gpu-utilization%2F9353%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 19, 2021, 2:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-and-single-gpu-utilization%2F9353 "2021-08-19T14:10:16Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![shahjaidev](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shahjaidev/32/2531_2.png) [@shahjaidev](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshahjaidev)%5C
**Post date:** [August 19, 2021, 2:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-and-single-gpu-utilization%2F9353%2F1 "2021-08-19T14:10:16Z")

</div>

Hi all!  
I have a question related to fine-tuning:  
Is fine-tuning a pre-trained transformer model a easier model an ‘easier’ task than training a transformer from scratch (BERT, GPT-2) in terms of GPU needs and GPU memory usage?

To clarify further, I’ve read how to train most transformer models, one would require multi-GPU training. However, is it possible to fine-tune some of these models on a single-GPU?  
Why is this the case? Is it because the computational graph during backpropagation is smaller when a lot of the parameters are frozen?