# Fine tune Masked Language Model on custom dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-masked-language-model-on-custom-dataset%2F747%3E%5C
**Category:** Beginners\
**Created:** [August 18, 2020, 1:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-masked-language-model-on-custom-dataset%2F747 "2020-08-18T13:10:35Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 20, 2020, 3:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-masked-language-model-on-custom-dataset%2F747%2F2 "2020-08-20T15:25:27Z")

</div>

Hi @smalltoken, what is the issue with [https://huggingface.co/blog/how-to-train](https://huggingface.co/blog/how-to-train) ?  
This colab should help you. It walks you through,

1. How to to train tokenizer from scratch
2. Create `RobertaModel` using the config
3. use the `DataCollatorForLanguageModeling`, which handle the masking
4. and train using `Trainer`.

---

_[View the full topic](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-masked-language-model-on-custom-dataset%2F747)._