# Adding new layer to T5encoder

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-new-layer-to-t5encoder%2F55164%3E%5C
**Category:** 🤗Transformers\
**Created:** [September 15, 2023, 6:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-new-layer-to-t5encoder%2F55164 "2023-09-15T18:21:59Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![mhdgholizadeh](https://avatars.discourse-cdn.com/v4/letter/m/5daacb/32.png) [@mhdgholizadeh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmhdgholizadeh)%5C
**Post date:** [September 15, 2023, 6:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fadding-new-layer-to-t5encoder%2F55164%2F1 "2023-09-15T18:22:00Z")

</div>

Hi,  
I need to add a new layer at the end of T5encoder for sequence classification. You can find my code bellow

class CustomT5(nn.Module):  
def **init** (self, checkpoint = “t5-small”, size\_last\_hidden\_state = 0, num\_labels = 1):  
super(CustomT5, self). **init** ()  
self.T5 = T5EncoderModel.from\_pretrained(checkpoint) # checkpoint: t5-small

```
    ## new layer
    self.linear = nn.Linear(size_last_hidden_state , num_labels) # T5EncoderModel(t5_small) last_hidden_state size = (1,15,512) 
    self.sigmoid = nn.Sigmoid()

def forward(self,size_last_hidden_state, input_ids = None):
    output = self.T5(input_ids = input_ids)
    last_hidden_state = self.linear(output.last_hidden_state.view(size_last_hidden_state))
    label = self.sigmoid(last_hidden_state)
    
    return label

```

The problem is, I need to know the size of last\_hidden\_state of t5 encoder, that depends on sequence length. so, the code doesn’t work properly. could you help about that?