# Retrain T5 using unsupervised learning with MLM

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fretrain-t5-using-unsupervised-learning-with-mlm%2F40527%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 21, 2023, 10:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fretrain-t5-using-unsupervised-learning-with-mlm%2F40527 "2023-05-21T10:42:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ianuragbhatt](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ianuragbhatt/32/16236_2.png) [@ianuragbhatt](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fianuragbhatt)%5C
**Post date:** [May 21, 2023, 10:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fretrain-t5-using-unsupervised-learning-with-mlm%2F40527%2F1 "2023-05-21T10:42:17Z")

</div>

Hi,  
I have built my own corpus. Now, I want to retrain T5 on that corpus. I don’t want to do fine-tune that model on a particular task. I just want to add my corpus data above the T5-base model. Can you all have any suggestions and advise.  
Thanks