| language: | |
| - et | |
| license: cc-by-sa-4.0 | |
| # Usage | |
| Load in transformers library with: | |
| ``` | |
| from transformers import AutoTokenizer, AutoModelForMaskedLM | |
| tokenizer = AutoTokenizer.from_pretrained("EMBEDDIA/est-roberta") | |
| model = AutoModelForMaskedLM.from_pretrained("EMBEDDIA/est-roberta") | |
| ``` | |
| # Est-RoBERTa | |
| Est-RoBERTa model is a monolingual Estonian BERT-like model. It is closely related to French Camembert model https://camembert-model.fr/. The Estonian corpora used for training the model have 2.51 billion tokens in total. The subword vocabulary contains 40,000 tokens. | |
| Est-RoBERTa was trained for 40 epochs. | |