# Adding custom layer to GPT-2

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-custom-layer-to-gpt-2%2F23677%3E%5C
**Category:** Models\
**Created:** [September 27, 2022, 1:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-custom-layer-to-gpt-2%2F23677 "2022-09-27T13:32:05Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vibhu04](https://avatars.discourse-cdn.com/v4/letter/v/c2a13f/32.png) [@Vibhu04](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FVibhu04)%5C
**Post date:** [September 27, 2022, 1:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-custom-layer-to-gpt-2%2F23677%2F1 "2022-09-27T13:32:05Z")

</div>

Hi everyone,  
I am looking for a way to modify GPT-2’s architecture slightly by inserting a custom feedforward layer inside a GPT-2 decoder block right after the masked self-attention sublayer. Is there a way to achieve this using Hugging Face’s GPT-2 model? I’m new to Hugging Face, any suggestions would be appreciated.  
Thank you!