# Evaluation loss depends on batch size

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluation-loss-depends-on-batch-size%2F112046%3E%5C
**Category:** Beginners\
**Created:** [October 14, 2024, 8:30pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluation-loss-depends-on-batch-size%2F112046 "2024-10-14T20:30:28Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 14, 2024, 11:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluation-loss-depends-on-batch-size%2F112046%2F2 "2024-10-14T23:46:30Z")

</div>

Apparently a known but neglected issue for several years…

> [@Batch size during training vs batch size during evaluation](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbatch-size-during-training-vs-batch-size-during-evaluation%2F20827)%3A
>
> I am confused about the difference between batch size during training versus batch size during evaluation. I am trying to measure how batch size influences the inference time(speed of prediction) of different NLP models after they have been trained using the Huggingface Trainer API. The code I used is below def print\_summary(result): print(f"Time: {result.metrics['test\_runtime']:.2f}") print(f"Samples/second: {result.metrics['test\_samples\_per\_second']:.2f}") print(f"Test Loss: {res…

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluation-loss-depends-on-batch-size%2F112046)._