# Trainer stuck mid epoch

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-stuck-mid-epoch%2F103765%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 24, 2024, 1:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-stuck-mid-epoch%2F103765 "2024-08-24T01:52:15Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![FDM1](https://avatars.discourse-cdn.com/v4/letter/f/c57346/32.png) [@FDM1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FFDM1)%5C
**Post date:** [August 24, 2024, 1:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-stuck-mid-epoch%2F103765%2F1 "2024-08-24T01:52:15Z")

</div>

Good evening.

Trainer has a behavior I do not fully understand (I am a beginner) and it gets stuck for a lot of time after a fixed number of steps, and does not proceed in training.

Here code and output at which it gets stuck.

code snippet:

```auto
training_args = TrainingArguments(
    output_dir=f"{'dataset.coco'.replace(' ', '-')}-finetune",
    num_train_epochs=20,
    max_grad_norm=0.1,
    learning_rate=5e-5,
    #warmup_steps=300,
    per_device_train_batch_size=1,
    #gradient_accumulation_steps=4,
    dataloader_num_workers=0,
    metric_for_best_model="eval_map",
    greater_is_better=True,
    load_best_model_at_end=True,
    eval_strategy="epoch",
    save_strategy="epoch",
    save_total_limit=2,
    remove_unused_columns=False,
    eval_do_concat_batches=False,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=pytorch_dataset_train,
    eval_dataset=pytorch_dataset_valid,
    tokenizer=processor,
    data_collator=collate_fn,
    compute_metrics=eval_compute_metrics_fn,
)

trainer.train()

```

output:

5%|▍ | 500/10320 [02:01\<46:19, 3.53it/s]{‘loss’: 56.0595, ‘grad\_norm’: 163.28176879882812, ‘learning\_rate’: 4.757751937984497e-05, ‘epoch’: 0.97}  
5%|▌ | 516/10320 [02:06\<45:03, 3.63it/s]  
0%| | 0/19 [00:00\<?, ?it/s]  
11%|█ | 2/19 [00:00\<00:03, 4.37it/s]  
16%|█▌ | 3/19 [00:00\<00:05, 3.12it/s]  
21%|██ | 4/19 [00:01\<00:05, 2.73it/s]  
26%|██▋ | 5/19 [00:01\<00:05, 2.53it/s]  
32%|███▏ | 6/19 [00:02\<00:05, 2.41it/s]  
37%|███▋ | 7/19 [00:02\<00:05, 2.35it/s]  
42%|████▏ | 8/19 [00:03\<00:04, 2.32it/s]  
47%|████▋ | 9/19 [00:03\<00:04, 2.29it/s]  
53%|█████▎ | 10/19 [00:04\<00:03, 2.28it/s]  
58%|█████▊ | 11/19 [00:04\<00:03, 2.27it/s]  
63%|██████▎ | 12/19 [00:04\<00:03, 2.25it/s]  
68%|██████▊ | 13/19 [00:05\<00:02, 2.24it/s]  
74%|███████▎ | 14/19 [00:05\<00:02, 2.24it/s]  
79%|███████▉ | 15/19 [00:06\<00:01, 2.24it/s]  
84%|████████▍ | 16/19 [00:06\<00:01, 2.24it/s]  
89%|████████▉ | 17/19 [00:07\<00:00, 2.23it/s]  
95%|█████████▍| 18/19 [00:07\<00:00, 2.36it/s]  
100%|██████████| 19/19 [00:07\<00:00, 2.89it/s]

I thank you all in advance.