# Zero Shot Classification using multiGPU

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fzero-shot-classification-using-multigpu%2F8079%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 6, 2021, 9:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fzero-shot-classification-using-multigpu%2F8079 "2021-07-06T09:42:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![subhamkumar](https://avatars.discourse-cdn.com/v4/letter/s/aeb1de/32.png) [@subhamkumar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsubhamkumar)%5C
**Post date:** [July 6, 2021, 9:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fzero-shot-classification-using-multigpu%2F8079%2F1 "2021-07-06T09:42:54Z")

</div>

Can we not use the Pipeline abstract class with multi GPU? I am trying do inference using pipeline(“zero-shot-classification”). It only utilises single GPU & throws CUDA out of memory memory error on 16GB 4x V100. How to go about using multiGPU(other than device=0)?

---

<div class="post-metadata">

**Author:** ![907Resident](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/907resident/32/24749_2.png) [@907Resident](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2F907Resident)%5C
**Post date:** [August 14, 2023, 11:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fzero-shot-classification-using-multigpu%2F8079%2F2 "2023-08-14T23:35:18Z")

</div>

I also have the same question. **Is there a way to use `pipeline` for inference with a “zero-shot-classification” model?** I cannot use more than one GPU at a time with `pipeline` for inference purposes.

This [conversation on GitHub](https://github.com/huggingface/transformers/issues/15799) is relevant but seems to point to the conclusion that it is not possible. However, if I am mistaken, please let me know.

Toward the end of the conversation, it says that the `accelerate` module will help, but when I run the following:

```python
from transformers import pipeline 

pipe = pipeline(
                task="zero-shot-learning", 
                model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli", 
                device_map="auto"
)

```

The error that I get is:

```sh
('Keyword argument not understood:', 'device_map')

```

I am using the following GPU compute set up:

```sh
GPU Info: 
  GPU 0: NVIDIA A10G
  GPU 1: NVIDIA A10G
  GPU 2: NVIDIA A10G
  GPU 3: NVIDIA A10G

```