# Accuracy of MLM model
**URL:**
**Author:**  [@Anne](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnne)%5C
**Post date:** [June 2, 2021, 2:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F1 "2021-06-02T02:38:32Z")
How to calculate the accuracy of the testing dataset when we build MLM model using scrach?
---
**Author:**  [@sanjaysingh23](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsanjaysingh23)%5C
**Post date:** [July 7, 2021, 8:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F2 "2021-07-07T20:16:21Z")
have you got the answers?
---
**Author:**  [@Anne](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnne)%5C
**Post date:** [July 9, 2021, 1:43am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F3 "2021-07-09T01:43:41Z")
Hi
Yes, I found a way but not sure whether it’s the best way.
What I’m doing is for each sentence I’m masking a random word in the sentence and ask the model to predict it. If the actual word is in the predicted list I’m increasing the tp(positive) value or else the tn(negative) value. finally I’m calculating the accuracy (tp/tp+tn).
and I’m calculating the perplexity value using the following code
> **[Google Colaboratory](https://colab.research.google.com/github/huggingface/notebooks/blob/master/examples/language_modeling.ipynb)**
Hope my answer is clear to you…
---
**Author:**  [@sanjaysingh23](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsanjaysingh23)%5C
**Post date:** [July 9, 2021, 9:25am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F4 "2021-07-09T09:25:57Z")
Hey, thank you for your response,
i am also calculating perplexity using the same code and i am using perplexity as a metric rather than accuracy.
This is because i don’t thing accuracy would be a better choice(as a metric) for mlm because there can be may words which can be used for a given mask. (feel free the correct) , instead i am calculating perplexity in the compute metric function and printing it along with training\_loss and validation loss along in the logs.
Thanks again!!
---
**Author:**  [@sanjaysingh23](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsanjaysingh23)%5C
**Post date:** [July 12, 2021, 4:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F5 "2021-07-12T16:05:38Z")
Also how much minimum data is required to fine tune roberta any idea?
---
**Author:**  [@Anne](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnne)%5C
**Post date:** [July 13, 2021, 1:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Faccuracy-of-mlm-model%2F6516%2F6 "2021-07-13T01:07:53Z")
sorry. I don’t have an idea about it…
do you know a way to build a n-gram model for word prediction and calculate the perplexity value of it?