# RoBERT model for Sinhala Language

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frobert-model-for-sinhala-language%2F6353%3E%5C
**Category:** Beginners\
**Created:** [May 25, 2021, 3:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frobert-model-for-sinhala-language%2F6353 "2021-05-25T03:35:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anne](https://avatars.discourse-cdn.com/v4/letter/a/ccd318/32.png) [@Anne](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAnne)%5C
**Post date:** [May 25, 2021, 3:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frobert-model-for-sinhala-language%2F6353%2F1 "2021-05-25T03:35:00Z")

</div>

Hi  
I am tring to build RoBERT model for Sinhala language

My final Training data set is as follows

no of words = 64 129 561

no of sentences = 5 134 347

Size of the file = 938,019 KB

I already created the Bert Tokenizer using my Training dataset. (Size of the tokenizer is 1644 KB + 1299 KB)

Now Im trying to Train the model using google colab

since the dataset is large I divided the dataset into 10 subsets.

but it seems that for 1epoch the training time is really large around 270h ( But the remaining time seems to decrease quickly, within 20mins it dropped down from 290h to 250h).

Sometimes the program gets crashed after 10 or 20 mins

I created the model by referring the following link. (I am using exact same code, this code execute in about 3h for 1 epoch)

Is this happens because the created tokenizer is big ?

Is there a better way to do this?

Reference code

> **[Google Colaboratory](https://colab.research.google.com/github/huggingface/blog/blob/master/notebooks/01_how_to_train.ipynb)**