# Masked language modeling loss

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasked-language-modeling-loss%2F698%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 12, 2020, 4:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasked-language-modeling-loss%2F698 "2020-08-12T16:10:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![abdallah197](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abdallah197/32/3973_2.png) [@abdallah197](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fabdallah197)%5C
**Post date:** [August 12, 2020, 4:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasked-language-modeling-loss%2F698%2F1 "2020-08-12T16:10:50Z")

</div>

Can anyone provide a link to a visual equation walkthrough for the MLM loss, or better how would it be implemented in torch.

---

<div class="post-metadata">

**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 13, 2020, 4:52am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmasked-language-modeling-loss%2F698%2F2 "2020-08-13T04:52:59Z")

</div>

This [thread](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbertformaskedlm-s-loss-and-scores-how-the-loss-is-computed%2F607) might help you