# VisionEncoderDecoder X-Attn Question
**URL:**
**Author:**  [@prithivida](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprithivida)%5C
**Post date:** [June 17, 2022, 2:51am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvisionencoderdecoder-x-attn-question%2F19276%2F1 "2022-06-17T02:51:12Z")
@nielsr - I am trying to understand the X-attn code in VisionEncoderDecoder architecture. I can see the comment in the code `" Cross-attention layers are automatically added to the decoder and should be fine-tuned on a downstream generative task, like image captioning."`
What is the best place to see the actual implementation of the X-attn between V Encoder and L Decoder?
Please advice.
Thanks
Prithivi
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [June 18, 2022, 8:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvisionencoderdecoder-x-attn-question%2F19276%2F2 "2022-06-18T08:07:33Z")
You can see [here](https://github.com/huggingface/transformers/blob/6589e510fa4e6c442059de2fab84752535de9b23/src/transformers/models/bert/modeling_bert.py#L294) that the `BertSelfAttention` class can be used both for self-attention and cross-attention, depending on whether or not `encoder_hidden_states` are provided.
---
**Author:**  [@prithivida](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprithivida)%5C
**Post date:** [June 20, 2022, 7:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvisionencoderdecoder-x-attn-question%2F19276%2F3 "2022-06-20T07:30:31Z")
Thanks @nielsr. Just to close out the loading and fine-tuning models with `VisionEcoderDecoder` only finetunes the X-Attn layers - is my assumption right? if not is there an explicit way to freeze the pretrained vision and language models?
Please advice
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [June 20, 2022, 8:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvisionencoderdecoder-x-attn-question%2F19276%2F4 "2022-06-20T08:09:41Z")
No, all parameters get updated. It’s just that the weights of the cross-attention layers are the only ones that start from scratch, all other weights are already pre-trained, and those get fine-tuned.
---
**Author:**  [@prithivida](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fprithivida)%5C
**Post date:** [June 20, 2022, 9:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvisionencoderdecoder-x-attn-question%2F19276%2F5 "2022-06-20T09:15:16Z")
Ok thank you