# Training on multiple GPUs with multi file script

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-on-multiple-gpus-with-multi-file-script%2F58716%3E%5C
**Category:** 🤗Accelerate\
**Created:** [October 16, 2023, 11:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-on-multiple-gpus-with-multi-file-script%2F58716 "2023-10-16T11:23:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sajan](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@Sajan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSajan)%5C
**Post date:** [October 16, 2023, 11:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-on-multiple-gpus-with-multi-file-script%2F58716%2F1 "2023-10-16T11:23:43Z")

</div>

I have a training script that takes the training arguments, creates a directory for the experiment run, processes the annotations from the files passed and trains a DETR model. My dataset class is custom and inherits the torch Dataset class. These things are handled from a main script which is the entrypoint.

I moved this configuration to use accelerate to train on multiple GPUs. I followed [this tutorial](https://huggingface.co/docs/transformers/accelerate) and changed the relevant parts. I started the training with `nohup python3 main.py --flags... &`. This uses only 1 GPU out of 4 and printing `accelerator.num_processes` returns 1.

I tried running with ` nohup accelerate launch main.py --flags... &` after running `accelerate config` and setting appropriate params. This created 4 experiment runs/directories which is not desirable.

My training loop is simple pytorch based loop and I am not using Trainer/torch lightning. What am I doing wrong? Is there a standard practice that I should follow?