# Smart Batching - speech up Bert finetune

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsmart-batching-speech-up-bert-finetune%2F4406%3E%5C
**Category:** Intermediate\
**Created:** [March 15, 2021, 9:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsmart-batching-speech-up-bert-finetune%2F4406 "2021-03-15T09:31:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BinhMinhs10](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/binhminhs10/32/60_2.png) [@BinhMinhs10](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBinhMinhs10)%5C
**Post date:** [March 15, 2021, 9:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsmart-batching-speech-up-bert-finetune%2F4406%2F1 "2021-03-15T09:31:17Z")

</div>

code: [Smart Batching Tutorial - Speed Up BERT Training · Chris McCormick](https://mccormickml.com/2020/07/29/smart-batching-tutorial/)  
I have tried fine-tune on 1 epoch and the time will decrease & accuracy increase but if it increases by 3-4 epoch then it looks like the model has learned less focus on pad token so smart batch will not accuracy increase.