# Accelerate Multi-GPU on several Nodes How to

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736%3E%5C
**Category:** 🤗Accelerate\
**Created:** [October 13, 2021, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736 "2021-10-13T09:01:05Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![demoacct](https://avatars.discourse-cdn.com/v4/letter/d/bbce88/32.png) [@demoacct](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdemoacct)%5C
**Post date:** [October 13, 2021, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736%2F1 "2021-10-13T09:01:05Z")

</div>

Hi,

I wonder how to setup Accelerate or possibly train a model if I have 2 physical machines sitting in the same network. Each machine has 4 GPUs.

Can I use Accelerate + DeepSpeed to train a model with this configuration ?

Can’t seem to be able to find any writeups or example how to perform the “accelerate config”.

Thanks.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [October 13, 2021, 11:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736%2F2 "2021-10-13T11:38:55Z")

</div>

I’m not sure what documentation you need, just type `accelerate config` in the terminal of both machines and follow the prompts.

---

<div class="post-metadata">

**Author:** ![demoacct](https://avatars.discourse-cdn.com/v4/letter/d/bbce88/32.png) [@demoacct](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdemoacct)%5C
**Post date:** [October 13, 2021, 12:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736%2F3 "2021-10-13T12:02:29Z")

</div>

Hi,

I have read the doc from [accelerate](https://huggingface.co/docs/accelerate/). This doc shows how I can perform training on a single multi-gpu machine (one machine) using the “accelerate config”.

I am looking for example, how to perform training on 2 multi-gpu machines. In other words, in my setup, I have 4 x GPU per machine.

What are the packages I needs to install ? For example:

- machine 1, I install accelerate & deepspeed. Run accelerate config
- machine 2, do I also just install accelerate & deepspeed ?

Is the training on multi-gpu using 2 machines possible ?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [October 13, 2021, 12:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerate-multi-gpu-on-several-nodes-how-to%2F10736%2F4 "2021-10-13T12:55:39Z")

</div>

Like I said, you need to run `accelerate config` on both machines (and yes you need to install everything you need on both of them), then run `accelerate launch training_script.py` on both machines as well.