# Trainer.predict in parallel not supported!

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-predict-in-parallel-not-supported%2F25347%3E%5C
**Category:** 🤗Transformers\
**Created:** [November 1, 2022, 5:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-predict-in-parallel-not-supported%2F25347 "2022-11-01T17:22:40Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![djaym7](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@djaym7](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdjaym7)%5C
**Post date:** [November 1, 2022, 5:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-predict-in-parallel-not-supported%2F25347%2F1 "2022-11-01T17:22:40Z")

</div>

Currently doing any inference via trainer.predict only uses 1 gpu to do all the computations. Has someone done any parallelization for this ? Split the data among all available gpus and do inference, aggregate all metrics once all processes are done ?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 2, 2022, 1:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-predict-in-parallel-not-supported%2F25347%2F2 "2022-11-02T13:56:20Z")

</div>

It is supported and you can have a look at our [examples](https://github.com/huggingface/transformers/tree/main/examples) which all use distributed evaluation if you have several GPUs.

---

<div class="post-metadata">

**Author:** ![djaym7](https://avatars.discourse-cdn.com/v4/letter/d/b5a626/32.png) [@djaym7](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdjaym7)%5C
**Post date:** [November 2, 2022, 4:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-predict-in-parallel-not-supported%2F25347%2F3 "2022-11-02T16:49:33Z")

</div>

No, it doesn’t run it parallel. Trainer.predict() keeps using 1 gpu and rest of 7 sit idle.

Different from that, I made my own which divides data and trainer.predicts on all N gpus and then gathers output back. This reduces time N times vs default usage of trainer.predict()