# Pretrain gpt2 example

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-gpt2-example%2F6743%3E%5C
**Category:** Beginners\
**Created:** [June 11, 2021, 9:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-gpt2-example%2F6743 "2021-06-11T21:49:39Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![scienceapptest](https://avatars.discourse-cdn.com/v4/letter/s/bcef8e/32.png) [@scienceapptest](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fscienceapptest)%5C
**Post date:** [June 11, 2021, 9:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-gpt2-example%2F6743%2F1 "2021-06-11T21:49:40Z")

</div>

Are there any docs or notebooks that show how to train GPT2 from scratch with a custom text corpus?