# Can't use multi GPU in evaluation from Trainer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 12, 2023, 4:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692 "2023-09-12T16:14:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![betty12](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@betty12](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbetty12)%5C
**Post date:** [September 12, 2023, 4:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692%2F1 "2023-09-12T16:14:50Z")

</div>

Why is it that when I use Trainer, multiple GPUs are used for training, but only one GPU is used for evaluation? When I compared the GPU usage for training and evaluation, I found that: only the memory of GPU-0 is increased, and only its GPU-util is not 0.  
This causes `per_device_eval_batch_size` to be only 1 or it goes OOM. And causing the evaluation to be slow.

---

<div class="post-metadata">

**Author:** ![zuzannad1](https://avatars.discourse-cdn.com/v4/letter/z/dc4da7/32.png) [@zuzannad1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fzuzannad1)%5C
**Post date:** [September 14, 2023, 9:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692%2F2 "2023-09-14T09:35:46Z")

</div>

Have the same exact issue. Have you come to any conclusions / managed to proceed?

---

<div class="post-metadata">

**Author:** ![betty12](https://avatars.discourse-cdn.com/v4/letter/b/ea666f/32.png) [@betty12](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbetty12)%5C
**Post date:** [September 18, 2023, 7:22am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692%2F3 "2023-09-18T07:22:21Z")

</div>

Not yet 😭

---

<div class="post-metadata">

**Author:** ![capnchat](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/capnchat/32/31430_2.png) [@capnchat](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcapnchat)%5C
**Post date:** [December 6, 2023, 5:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcant-use-multi-gpu-in-evaluation-from-trainer%2F54692%2F4 "2023-12-06T17:16:49Z")

</div>

Any update on this? I am getting OOM for the same thing, only cuda:0 is being used…

I have heard about DataParallel or DistributedDataParallel, but this appears to require pretty extensive refactoring of my training script…