# PreTrain Electra/T5 for Korean from scratch
**URL:**
**Author:**  [@JCooky](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJCooky)%5C
**Post date:** [June 24, 2021, 8:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-electra-t5-for-korean-from-scratch%2F7230%2F1 "2021-06-24T08:35:23Z")
# PreTrain Electra/T5 for Korean from scratch
This project will be train the pretrained-model (Electra, T5, …) from Korean corpus called as a 모두의말뭉치.
## 2. Language
The model will be trained in Korean.
## 3. Model
- Electra
- T5
## 4. Datasets
[모두의말뭉치](https://corpus.korean.go.kr)
## 5. Training scripts
We can make use of [example](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling) to train the model.\>
Addtionaly, we will be make the train codes origin from the transformers example
## 7. Desired project outcome
A pretrained weights. After, fine-tune both text classification and text summarization
---
**Author:**  [@chyjis](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fchyjis)%5C
**Post date:** [July 1, 2021, 3:54pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-electra-t5-for-korean-from-scratch%2F7230%2F2 "2021-07-01T15:54:48Z")
I’m interested in pretrained korean model, too! I wanna join in. ol
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [July 1, 2021, 6:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-electra-t5-for-korean-from-scratch%2F7230%2F3 "2021-07-01T18:22:25Z")
Awesome, finalizing you guys!
---
**Author:**  [@anon2936587](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon2936587)%5C
**Post date:** [July 2, 2021, 7:45am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpretrain-electra-t5-for-korean-from-scratch%2F7230%2F4 "2021-07-02T07:45:49Z")
Good! I’d like to join this. 🙂