# VivitModel last hidden states dimension Problem

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvivitmodel-last-hidden-states-dimension-problem%2F96852%3E%5C
**Category:** 🤗Transformers\
**Created:** [July 11, 2024, 4:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvivitmodel-last-hidden-states-dimension-problem%2F96852 "2024-07-11T16:02:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BeefWellington](https://avatars.discourse-cdn.com/v4/letter/b/bc79bd/32.png) [@BeefWellington](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBeefWellington)%5C
**Post date:** [July 11, 2024, 4:02pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvivitmodel-last-hidden-states-dimension-problem%2F96852%2F1 "2024-07-11T16:02:48Z")

</div>

In the example document of VivitModel [Video Vision Transformer (ViViT) (huggingface.co)](https://huggingface.co/docs/transformers/model_doc/vivit), why the last\_hidden\_states.shape is 3137 but not 3136? What is the additional 1 dimension?

Based on the default parameters, image\_size=224, num\_frames=32, tubelet\_size=[2,16,16], then number of tubelets is (224/16)x(224/16)x(32/2) = 14x14x16 = 3136.  
🤯

```auto
# forward pass
outputs = model(**inputs)
last_hidden_states = outputs.last_hidden_state
list(last_hidden_states.shape)
[1, 3137, 768]

```