# Original transformers model implementation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-transformers-model-implementation%2F6991%3E%5C
**Category:** Beginners\
**Created:** [June 19, 2021, 12:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-transformers-model-implementation%2F6991 "2021-06-19T00:46:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MickyMike](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mickymike/32/15863_2.png) [@MickyMike](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMickyMike)%5C
**Post date:** [June 19, 2021, 12:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-transformers-model-implementation%2F6991%2F1 "2021-06-19T00:46:55Z")

</div>

Hi there,  
In Huggingface transformers lib, is there implementation of the original transformers model mentioned in the paper, “Attention is all you need”.  
I just browsed the documentation site and saw the implementation like BERT, T5, etc., but I didn’t see the original model.

Thanks for the clarification.  
Kind Regards

---

<div class="post-metadata">

**Author:** ![Vasily](https://avatars.discourse-cdn.com/v4/letter/v/94ad74/32.png) [@Vasily](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FVasily)%5C
**Post date:** [May 31, 2022, 1:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-transformers-model-implementation%2F6991%2F2 "2022-05-31T13:26:10Z")

</div>

Hey @MickyMike did you find a solution?

---

<div class="post-metadata">

**Author:** ![MickyMike](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mickymike/32/15863_2.png) [@MickyMike](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMickyMike)%5C
**Post date:** [June 1, 2022, 5:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foriginal-transformers-model-implementation%2F6991%2F3 "2022-06-01T05:11:49Z")

</div>

Yes @Vasily  
I used the vanilla transformer implemented in PyTorch.

[https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html](https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html)