# Same number of optimizations steps with 1 GPU or 4 GPUs?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsame-number-of-optimizations-steps-with-1-gpu-or-4-gpus%2F33584%3E%5C
**Category:** 🤗Accelerate\
**Created:** [March 11, 2023, 3:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsame-number-of-optimizations-steps-with-1-gpu-or-4-gpus%2F33584 "2023-03-11T15:06:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BadDepartment](https://avatars.discourse-cdn.com/v4/letter/b/c57346/32.png) [@BadDepartment](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBadDepartment)%5C
**Post date:** [March 11, 2023, 3:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsame-number-of-optimizations-steps-with-1-gpu-or-4-gpus%2F33584%2F1 "2023-03-11T15:06:36Z")

</div>

I am getting the same number of optimization steps using `Trainer` and `Accelerate`; what might be going wrong with my settings?

```auto

def train_trainer_ddp():

    training_args = TrainingArguments(
        output_dir='./results', # output directory
        num_train_epochs=1, # total number of training epochs
        per_device_train_batch_size=16, # batch size per device during training
        per_device_eval_batch_size=64, # batch size for evaluation
        warmup_steps=500, # number of warmup steps for learning rate scheduler
        fp16=True,
        dataloader_num_workers=20,
        disable_tqdm=False,
        fsdp=False,
        weight_decay=0.01, # strength of weight decay
        logging_dir='./logs', # directory for storing logs
        logging_steps=1000,
    )

    model = AlbertForSequenceClassification.from_pretrained("albert-base-v2", num_labels=num_classes)

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_set,
        eval_dataset=val_set 
    )

    trainer.train()

notebook_launcher(train_trainer_ddp, args=(), num_processes=1, mixed_precision="fp16") # or 4

```