# \[HELP\] How to include emojis in masked language modelling?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-how-to-include-emojis-in-masked-language-modelling%2F6660%3E%5C
**Category:** Beginners\
**Created:** [June 8, 2021, 4:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-how-to-include-emojis-in-masked-language-modelling%2F6660 "2021-06-08T16:20:28Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon58275033](https://avatars.discourse-cdn.com/v4/letter/a/5f8ce5/32.png) [@anon58275033](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon58275033)%5C
**Post date:** [June 8, 2021, 4:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhelp-how-to-include-emojis-in-masked-language-modelling%2F6660%2F1 "2021-06-08T16:20:28Z")

</div>

Hello,

I am new to Hugging Face and masked language modelling (MLM), and I was wondering how to include emojis when doing such a task.

I have followed this tutorial: [notebooks/language\_modeling.ipynb at master · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/master/examples/language_modeling.ipynb)

I have a dataset with tweets, with each tweet containing an emoji at the end - here is a sample of my data:

| ID | Tweet |
| --- | --- |
| 1 | Looking good today 🕶 |
| 2 | Weather is so hot, lol ☀ |
| 3 | I hate you!!! 🤬 |

At the moment, I have fully trained my masked language model using my dataset, but when I predict something, it does **NOT** output or predict the emojis. It just predicts words.

This is my desired input from using my dataset for MLM:

```auto
"You look great [MASK]"

```

This is my desired output from using my dataset for MLM:

```auto
[{'score': 0.26041436195373535,
  'sequence': 'You look great 😎"',
  'token': 72,
  'token_str': '."'},
 {'score': 0.1813151091337204,
  'sequence': 'you look great 💯"',
  'token': 2901,
  'token_str': '!"'},
 {'score': 0.14516998827457428,
  'sequence': 'you look great 👌',
  'token': 328,
  'token_str': '!'},]

```

However, this is what I am actually getting from my output:

```auto
[{'score': 0.26041436195373535,
  'sequence': 'You look great?"',
  'token': 72,
  'token_str': '."'},
 {'score': 0.1813151091337204,
  'sequence': 'You look great."',
  'token': 2901,
  'token_str': '!"'},
 {'score': 0.14516998827457428,
  'sequence': 'You look great!',
  'token': 328,
  'token_str': '!'},]

```

I know it is possible to do this, but how do I do it? I am close, but not very.

Likewise, I have my model fully trained on my dataset, but it just does not seem to output emojis, even though I have included them in the training.

Does something need to be included to accept emoji? If so, what?

Thanks - I would really appreciate the help!