# EncoderDecoderModel with Longformer and Bert

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fencoderdecodermodel-with-longformer-and-bert%2F3601%3E%5C
**Category:** 🤗Transformers\
**Created:** [February 11, 2021, 2:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fencoderdecodermodel-with-longformer-and-bert%2F3601 "2021-02-11T14:33:06Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![cahya](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cahya/32/48725_2.png) [@cahya](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcahya)%5C
**Post date:** [February 11, 2021, 2:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fencoderdecodermodel-with-longformer-and-bert%2F3601%2F1 "2021-02-11T14:33:06Z")

</div>

Hi,  
Is it possible to create EncoderDecoderModel using Longformer and Bert? I tried Longformer and Roberta and it works (the training runs), but if I use Longformer and Bert, I get this error message `RuntimeError: CUDA error: device-side assert triggered` when I train it.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [February 11, 2021, 7:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fencoderdecodermodel-with-longformer-and-bert%2F3601%2F2 "2021-02-11T19:48:19Z")

</div>

Normally Longformer and BERT should work in an encoder-decoder setting.

If you have a CUDA error, it’s advised to test the code on CPU and see if you’re getting an error that is more interpretable.