# Freeze Deberta Layers

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreeze-deberta-layers%2F49621%3E%5C
**Category:** 🤗Transformers\
**Created:** [August 5, 2023, 10:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreeze-deberta-layers%2F49621 "2023-08-05T22:03:00Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![alejopaullier](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alejopaullier/32/27750_2.png) [@alejopaullier](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falejopaullier)%5C
**Post date:** [August 5, 2023, 10:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffreeze-deberta-layers%2F49621%2F1 "2023-08-05T22:03:00Z")

</div>

I am trying to freeze `deberta-v3-small` layers. I first froze 4 blocks with:

```python
NUM_FROZEN_LAYERS = 83 # <--- This index correspond to last layer of block 4

for i,(name, param) in enumerate(list(model.named_parameters())\
                                 [0:NUM_FROZEN_LAYERS]):
    param.requires_grad = False

```

This works fine. However, I later wanted to freeze 6 blocks (up to layer number 115) and the following error was raised:

```auto
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

```

I played around and found out that after layer number 103 this error starts appearing. This layer is `model.encoder.layer.6.attention.self.value_proj.weight`.

This looks very random. Does anyone know why is this happening and if there is another way to freeze certain blocks? Maybe I am not doing it the right way.