# Distilbert Seq2clas

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%3E%5C
**Category:** 🤗Transformers\
**Created:** [June 25, 2021, 1:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323 "2021-06-25T01:46:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![natank](https://avatars.discourse-cdn.com/v4/letter/n/46a35a/32.png) [@natank](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnatank)%5C
**Post date:** [June 25, 2021, 1:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%2F1 "2021-06-25T01:46:37Z")

</div>

Hello

I have two questions:  
1 We can view the. pooled layer by using output\_hidden\_states=true and follow the logic, but is there a generic way to do so?

2 When we are doing inference, can we output the hidden\_states from trainer.predict() as we do by model(input/output\_hidden\_states=True)?

Thanks

---

<div class="post-metadata">

**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpratikbhatia0011)%5C
**Post date:** [July 18, 2021, 10:59am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%2F2 "2021-07-18T10:59:29Z")

</div>

I have a similar doubt as regards point 2.

I am working on Question Answering with Distilbert.  
The predict function in the Trainer does not work if output\_hidden\_states = True. It works fine if the same argument is set to False.  
Is this a bug? If not, then how is one to use a model for prediction if one has set the argument output\_hidden\_states = True while initializing the model ?

---

<div class="post-metadata">

**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpratikbhatia0011)%5C
**Post date:** [July 18, 2021, 12:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%2F3 "2021-07-18T12:01:09Z")

</div>

Found the solution. Posting it here just in case someone else too gets stuck with my particular problem.

I just had to pass ignore\_keys = [‘attentions’] in the predict function and everything works fine.

---

<div class="post-metadata">

**Author:** ![pratikbhatia0011](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@pratikbhatia0011](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpratikbhatia0011)%5C
**Post date:** [July 18, 2021, 12:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%2F4 "2021-07-18T12:22:45Z")

</div>

@natank To answer your second question, I don’t think so because as seen in the predict function source code, it only returns predictions, label\_ids and metrics.

 ![trainer_predict](https://us1.discourse-cdn.com/hellohellohello/original/2X/9/9548774cb4fa6980305311e4894aa48f00876563.png)

---

<div class="post-metadata">

**Author:** ![natank](https://avatars.discourse-cdn.com/v4/letter/n/46a35a/32.png) [@natank](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnatank)%5C
**Post date:** [July 19, 2021, 11:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdistilbert-seq2clas%2F7323%2F5 "2021-07-19T11:53:32Z")

</div>

Thanks for you replies. Regarding one it indeed resolved me a problem. Regarding your answer on q2. It seems that Huggingsface offer two working modes, research : use model , and model taining - use predict. Am I right?
