# Tips for PreTraining BERT from scratch

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftips-for-pretraining-bert-from-scratch%2F1175%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 18, 2020, 3:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftips-for-pretraining-bert-from-scratch%2F1175 "2020-09-18T03:37:22Z")\
**Posts on this page:** 1\
**Showing post:** 17

<div class="post-metadata">

**Author:** ![prajjwal1](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/prajjwal1/32/2246_2.png) [@prajjwal1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprajjwal1)%5C
**Post date:** [September 24, 2020, 2:31pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftips-for-pretraining-bert-from-scratch%2F1175%2F17 "2020-09-24T14:31:29Z")

</div>

I’ve opened a new [issue](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpre-train-bert-without-nsp%2F1245) which is about pre-training. Training on GLUE part is resolved. Thanks for asking.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftips-for-pretraining-bert-from-scratch%2F1175)._