# ELECTRA for Causal LM

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Felectra-for-causal-lm%2F5364%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 8, 2021, 5:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Felectra-for-causal-lm%2F5364 "2021-04-08T05:32:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![yusukemori](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/yusukemori/32/23302_2.png) [@yusukemori](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fyusukemori)%5C
**Post date:** [April 8, 2021, 5:32am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Felectra-for-causal-lm%2F5364%2F1 "2021-04-08T05:32:38Z")

</div>

Hi,

Can we use `ElectraForMaskedLM` for causal language modeling?

It seems `BertForMaskedLM` was split into and `BertLMHeadModel` and `BertForMaskedLM`, one model for causal LM, one for masked LM.  
[https://github.com/huggingface/transformers/pull/4874](https://github.com/huggingface/transformers/pull/4874)

Is ELECTRA’s Masked LM class can be applied for causal LM?  
Or is there a separate class?  
Or is it not possible to use ELECTRA as Causal LM in the first place?

Thank you in advance.