# Create a SentenceTransformer in Dhivehi using ELECTRA
**URL:**
**Author:**  [@ashraq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fashraq)%5C
**Post date:** [November 17, 2021, 4:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F1 "2021-11-17T04:31:49Z")
**Description**
Dhivehi is a low resource language. Since the available data is less, ELECTRA seems to be a good option as it requires less computing power and training data as compared to others.
**Model**
electra-small pretrained in dhivehi available [here](https://huggingface.co/ashraq/dv-electra-small)
### Discord channel
To chat and organise with other people interested in this project, head over to our [Discord](http://xwo7x.mjt.lu/lnk/CAAAAkdkZwAAAAAAAAAAALG-MTEAAYCr4H8AAAAAAAyFyQBhjpnimM4rcE5DT5ibCUtRT96tWwAMh8Y/15/U6BYNMNDksyyGG4jgtzPKQ/aHR0cHM6Ly9odWdnaW5nZmFjZS5jby9qb2luL2Rpc2NvcmQ) and:
- Follow the instructions on the `#join-course` channel
- Join the `#sentence-transformers-dhivehi` channel
Just make sure you comment here to indicate that you’ll be contributing to this project 🙂
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [November 17, 2021, 4:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F2 "2021-11-17T16:21:17Z")
He @ashraq thanks for proposing this interesting project! One question: what do you mean by create a “sentence transformer”? Are you talking about adding a pooling layer to the `electra-small` model and then training that on a Dhivevi corpus?
Do you also happen to have access / know of a Dhivevi corpus to train on?
---
**Author:**  [@ashraq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fashraq)%5C
**Post date:** [November 17, 2021, 5:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F3 "2021-11-17T17:26:36Z")
Yes. Adding a pooling layer and fine tuning sentence transformer for dhivehi.
Yes there is a corpus available. In fact i have been preparing data for this task since last week before I came to know about the event. So i think this will be a wonderful opportunity.
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [November 17, 2021, 9:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F4 "2021-11-17T21:09:38Z")
This sounds like a great project indeed! I’ve created a Discord channel (see topic description) in case you and others want to use it
---
**Author:**  [@ashraq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fashraq)%5C
**Post date:** [November 17, 2021, 9:43pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F5 "2021-11-17T21:43:28Z")
btw are there any limitations on the instance we can choose on aws sage maker during this event?
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [November 17, 2021, 10:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcreate-a-sentencetransformer-in-dhivehi-using-electra%2F11938%2F6 "2021-11-17T22:01:06Z")
As far as I know you can choose a p3 instance if it’s available or a T4 if not 🙂