# Accelerator.backward freeze

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-backward-freeze%2F142484%3E%5C
**Category:** DeepSpeed\
**Created:** [February 23, 2025, 12:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-backward-freeze%2F142484 "2025-02-23T12:51:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucky-lzh](https://avatars.discourse-cdn.com/v4/letter/l/50afbb/32.png) [@lucky-lzh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flucky-lzh)%5C
**Post date:** [February 23, 2025, 12:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-backward-freeze%2F142484%2F1 "2025-02-23T12:51:59Z")

</div>

```auto
        if image_list is not None:
            self.accelerator.backward(loss_ce+loss_diff)
            loss = loss_ce.detach()+loss_diff.detach()
        else:
            self.accelerator.backward(loss_ce)
            loss = loss_ce.detach()

```

I’m using multi-GPU training, and the code above causes the backward computation graphs on different GPUs to be different, leading to deadlocks. Is there any way to solve this issue?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 24, 2025, 8:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccelerator-backward-freeze%2F142484%2F2 "2025-02-24T08:04:34Z")

</div>

I think this is a similar problem, but it might be more reliable to raise an issue on the github page for the accelerate library…

> [@How to collect the accuracy when running multi GPU model with accelerate?](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-collect-the-accuracy-when-running-multi-gpu-model-with-accelerate%2F63971)%3A
>
> Dear all, When using accelerate for speeding up my training I have the problem, that the calculated accuracy far away from what I get when running the same model on one graphic card is. network, optimizer, trainData, validationData, scheduler = accelerator.prepare(network, optimizer, trainData, validationData, None) for epoch in range(args[“epochs”]): # Set the model in training mode network.train() trainRunningLoss = 0.0 trainRunningCorrect = 0 for i, (images, labels) in enumerate(trainD…

> [@Question about calculating training loss of multi-GPU with Accelerate](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fquestion-about-calculating-training-loss-of-multi-gpu-with-accelerate%2F71222)%3A
>
> I am currently training a model in Kaggle with Accelerate (2 T4 GPUs), and I’m confused about how to calculate or log the training loss correctly. According to some discussions ([https://github.com/huggingface/accelerate/issues/2109](https://github.com/huggingface/accelerate/issues/2109), [https://github.com/huggingface/accelerate/issues/226](https://github.com/huggingface/accelerate/issues/226)) and some official examples (complete\_nlp\_example), it appears that I may only need to log the training loss on the main process. I do some test in Kaggle with the following code: from accelerate import Acceler…