# Trainer epoch does not go through all training data?
**URL:**
**Author:**  [@Skylixia](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSkylixia)%5C
**Post date:** [January 22, 2021, 2:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-epoch-does-not-go-through-all-training-data%2F3339%2F1 "2021-01-22T14:21:17Z")
Hello
I’m training a model with transformers Trainer but when I set the number of epoch to eg: 1000 then it seems the training just does 1000 steps however an epoch is normally the number of times the model goes through the entire dataset. Thus, how can we use the trainer such that each epoch goes through the full training dataset (and that we see the progression of these)
Thanks!
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [January 22, 2021, 2:40pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-epoch-does-not-go-through-all-training-data%2F3339%2F2 "2021-01-22T14:40:44Z")
Hi there!
Please post the command/code you are executing as we can’t really help without that.
---
**Author:**  [@Skylixia](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSkylixia)%5C
**Post date:** [January 22, 2021, 2:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-epoch-does-not-go-through-all-training-data%2F3339%2F3 "2021-01-22T14:56:36Z")
Sure, sorry! I thought it was not a code specific question but rather about the parameters of the Trainer class 🙂
Here is the code I use:
from transformers import Trainer, TrainingArguments
training\_args = TrainingArguments(
output\_dir=’./out’,
num\_train\_epochs=1000,
per\_device\_train\_batch\_size=2,
per\_device\_eval\_batch\_size=2,
warmup\_steps=500,
weight\_decay=0.01,
logging\_dir=’./logs’,
logging\_steps=10,
)
trainer = Trainer(
model=model,
args=training\_args,
train\_dataset=training\_data,
eval\_dataset=validation\_data)
trainer.train()
I would expect that there are 1000 epoch that goes through the full train\_dataset but the output goes very fast and prints “epoch 1/1000, epoch 2/1000, epoch 3/1000, …” which give the impression that the epoch is simply a training step rather than an actual epoch. However, having 20k training instances I expect to see 20k steps for each epoch and this normally take some time.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [January 22, 2021, 5:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-epoch-does-not-go-through-all-training-data%2F3339%2F4 "2021-01-22T17:57:45Z")
I wanted to check your `num_train_epochs` wasn’t overridden by other parameters like `max_steps`. The code looks correct and the logs do indicate you are going through the epochs. Double check the length of your dataset to make sure it’s not reduced to something small.
---
**Author:**  [@Skylixia](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FSkylixia)%5C
**Post date:** [January 22, 2021, 7:24pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrainer-epoch-does-not-go-through-all-training-data%2F3339%2F5 "2021-01-22T19:24:33Z")
My dataset seem to have the right size. Using Wandb logger it seems that there is only 1 step performed in each epoch indeed… I don’t know what could be wrong.
My code is basically done as in [this post](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-pegasus%2F1433).