# Regarding the eval batch size for large models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-the-eval-batch-size-for-large-models%2F17729%3E%5C
**Category:** Models\
**Created:** [May 9, 2022, 8:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-the-eval-batch-size-for-large-models%2F17729 "2022-05-09T20:13:04Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![arunwzd](https://avatars.discourse-cdn.com/v4/letter/a/90db22/32.png) [@arunwzd](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Farunwzd)%5C
**Post date:** [May 9, 2022, 8:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fregarding-the-eval-batch-size-for-large-models%2F17729%2F1 "2022-05-09T20:13:04Z")

</div>

I am training T5-base with 2040 effective batch size (train\_batch\_size\_per\_gpu=8, grad.accumartion\_steps=51, n\_gpus=8) on 8 v100 GPUs. The training works fine w/o any memory error. But during the evaluation step (with eval\_batch\_size\_per\_gpu=8), I get `MemoryError` though its the same size as train batch size. Its weird that I am getting the memory error just during evaluation.

Also, I tried reducing the eval\_batch\_size\_per\_gpu=4 (halved), even then, I get memory error. Can someone help me with whats going on?