# T5forConditionalGeneration + classification
**URL:**
**Author:**  [@manzar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmanzar)%5C
**Post date:** [December 10, 2020, 11:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5forconditionalgeneration-classification%2F2786%2F1 "2020-12-10T23:03:16Z")
I would like to do sequence classification over the encoder in parallel with conditional generation using an auxiliary loss. However, I am confused about which hidden state I should take for the classification.
Supposing that the hidden state of the last layer has the following dimensions: [batch size, seq length, hidden size] should I take the last one [:, -1, :] ?
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [December 11, 2020, 12:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5forconditionalgeneration-classification%2F2786%2F2 "2020-12-11T12:44:58Z")
It depends on the model. BERT uses the first one (where the CLS token is), some models use a pooling of all hidden states, other the one for the last logits (which is not necessarily `-1` since you could have padding). I’d look at what is done in `T5ForSequenceClassification` and copy the code.
---
**Author:**  [@manzar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmanzar)%5C
**Post date:** [December 11, 2020, 3:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5forconditionalgeneration-classification%2F2786%2F3 "2020-12-11T15:20:14Z")
You are absolutely right. That’s what I tried to do at first. However, the T5 model has no `T5ForSequenceClassification` class or something similar. I think that the most suitable is to use the last logits taking into account padding. Do you have in mind any function that can be helpful?
---
**Author:**  [@manzar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmanzar)%5C
**Post date:** [December 13, 2020, 2:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ft5forconditionalgeneration-classification%2F2786%2F4 "2020-12-13T14:48:27Z")
I would also like to ask if there is any way to tie the weights of the encoder and the decoder.