# TrainingArguments changing the GPU by iteslf

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainingarguments-changing-the-gpu-by-iteslf%2F8001%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 4, 2021, 3:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainingarguments-changing-the-gpu-by-iteslf%2F8001 "2021-07-04T15:00:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanjaysingh23](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sanjaysingh23/32/2161_2.png) [@sanjaysingh23](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsanjaysingh23)%5C
**Post date:** [July 4, 2021, 3:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainingarguments-changing-the-gpu-by-iteslf%2F8001%2F1 "2021-07-04T15:00:30Z")

</div>

I have 4 gpus available, out of which i have selected the second one using

```auto
if torch.cuda.is_available():
    torch.cuda.set_device(2)

```

However when i compute the `TrainingArgument()` command :  
`training_args = TrainingArguments('mydirectory')`

then `torch.cuda.current_device()`. is giving 0

Any idea why this is happening?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [July 7, 2021, 12:19pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainingarguments-changing-the-gpu-by-iteslf%2F8001%2F2 "2021-07-07T12:19:13Z")

</div>

Yes, the training argument set the GPU corresponding to its `local_rank` value (for distributed training), so you have to make sure to pass along `local_rank=2` when you instantiate them.

Though to execute a script on a given GPU, you would be better off setting the global env variable `CUDA_VISIBLE_DEVICES`.