# Returned Tensors and Hidden State
**URL:**
**Author:**  [@azhx](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fazhx)%5C
**Post date:** [August 27, 2020, 4:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freturned-tensors-and-hidden-state%2F883%2F1 "2020-08-27T04:21:23Z")
Hi, just quickly getting started with GPT2.
from [https://huggingface.co/gpt2](https://huggingface.co/gpt2) :
```auto
from transformers import GPT2Tokenizer, GPT2Model
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2Model.from_pretrained('gpt2')
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)
```
is said to yield the features of the text.
Upon inspecting the output, it is an irregularly shaped tuple with nested tensors. Looking at the source code for GPT2Model, this is supposed to represent the hidden state. I can guess what some of these dimensions represent, for example the `768` dimension is obviously the word embedding, but in general I can’t find any documentation about interpreting the information in `output`
I also tried adding:
`output = model(**encoded_input, output_attentions = True)`
but I do not know how to interpret the dimensions of this either.
I am told to “See `attentions` under returned tensors for more detail.” in the docstring at [https://huggingface.co/transformers/\_modules/transformers/modeling\_gpt2.html#GPT2Model](https://huggingface.co/transformers/_modules/transformers/modeling_gpt2.html#GPT2Model)
But I cannot find what this is referring to. Can someone help me interpret the dimensions of these nested tuples?
---
**Author:**  [@prajjwal1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprajjwal1)%5C
**Post date:** [August 27, 2020, 6:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freturned-tensors-and-hidden-state%2F883%2F2 "2020-08-27T06:37:24Z")
Please refer to [GPT2 docs](https://huggingface.co/transformers/model_doc/gpt2.html). It will give a detailed description of what `GPT2Model` is supposed to return.
It returns (in order of output):
- `last_hidden_state ` : `(batch_size, sequence_length, hidden_size)`
- `past`: `(2, batch_size, num_heads, sequence_length, embed_size_per_head)`
- `hidden_states `
- `attentions`
---
**Author:**  [@azhx](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fazhx)%5C
**Post date:** [August 28, 2020, 1:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freturned-tensors-and-hidden-state%2F883%2F3 "2020-08-28T01:11:31Z")
Can’t believe I missed this
---
**Author:**  [@valhalla](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvalhalla)%5C
**Post date:** [August 28, 2020, 3:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freturned-tensors-and-hidden-state%2F883%2F4 "2020-08-28T15:48:40Z")
Hey @azhx,
if you are on `master` then you can also use the `ModelOutput` object, which is a `dict` like object which lets you access the output as `out.attentions`, `out.hidden_states` etc. For `GPT2Model` model, it returns `BaseModelOutputWithPast`. You can find the docs [here](https://huggingface.co/transformers/master/main_classes/output.html#model-outputs)
---
**Author:**  [@azhx](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fazhx)%5C
**Post date:** [September 5, 2020, 6:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Freturned-tensors-and-hidden-state%2F883%2F5 "2020-09-05T06:18:35Z")
Yes, I’m on master now and using this, thanks!