# BERT model trained on small corpus (English)?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-model-trained-on-small-corpus-english%2F6079%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 8, 2021, 10:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-model-trained-on-small-corpus-english%2F6079 "2021-05-08T22:06:07Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![macabdul9](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/macabdul9/32/3302_2.png) [@macabdul9](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmacabdul9)%5C
**Post date:** [May 8, 2021, 10:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-model-trained-on-small-corpus-english%2F6079%2F1 "2021-05-08T22:06:07Z")

</div>

Is there any transformer-based model pre-trained on very small data ( less than BERT)?