# Save custom transformer as PreTrainedModel

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-custom-transformer-as-pretrainedmodel%2F9768%3E%5C
**Category:** Intermediate\
**Created:** [September 7, 2021, 8:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-custom-transformer-as-pretrainedmodel%2F9768 "2021-09-07T08:36:28Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![cgutknecht](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/cgutknecht/32/2663_2.png) [@cgutknecht](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcgutknecht)%5C
**Post date:** [September 7, 2021, 8:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-custom-transformer-as-pretrainedmodel%2F9768%2F1 "2021-09-07T08:36:29Z")

</div>

I have a custom BERT-like model (with modified attention) that I pretrained with PyTorch. Data preparation was done with a Huggingface tokenizer. Now I want to integrate this PyTorch-model in the Huggingface environment so it can be used in pipelines and for finetuning as a PreTrainedModel. How do I generate the necessary config files?

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [September 7, 2021, 9:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsave-custom-transformer-as-pretrainedmodel%2F9768%2F2 "2021-09-07T09:34:27Z")

</div>

There might be a better way, but I would:

- subclass PretrainedModel with your own class
- load your trained model weights into this new class
- run yourmodel.save\_pretrained() to save the model weights for this class
- now you can do YourCustomModel.from\_pretrained() as it now can use those HF methods