# Move model with device\_map="balanced" to CPU

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmove-model-with-device-map-balanced-to-cpu%2F43932%3E%5C
**Category:** 🤗Transformers\
**Created:** [June 20, 2023, 2:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmove-model-with-device-map-balanced-to-cpu%2F43932 "2023-06-20T14:00:15Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhinavkulkarni](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/abhinavkulkarni/32/32750_2.png) [@abhinavkulkarni](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fabhinavkulkarni)%5C
**Post date:** [June 20, 2023, 2:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmove-model-with-device-map-balanced-to-cpu%2F43932%2F1 "2023-06-20T14:00:15Z")

</div>

Hi,

I have a large model that I am unable to fit into GPU, so I am loading it as follows:

```auto
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, AutoConfig

kwargs = {"device_map": "balanced", "torch_dtype": torch.float16}
max_memory = {0: "10GiB", "cpu": "99GiB"}

model_name = 'facebook/opt-6.7b'
config = AutoConfig.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, config=config, max_memory=max_memory, **kwargs)
tokenizer = AutoTokenizer.from_pretrained(model_name, fast=True)

```

Now, when I try to move the model back to CPU to free up GPU memory for other processing, I get an error:

```auto
model = model.to('cpu')
torch.cuda.empty_cache()

```

I get:

`NotImplementedError: Cannot copy out of meta tensor; no data!`

How can I free up GPU memory?

When I check `model.hf_device_map`, I see that some layers are on GPU while others are on CPU.

Thanks!

---

<div class="post-metadata">

**Author:** ![jhilgart22](https://avatars.discourse-cdn.com/v4/letter/j/f08c70/32.png) [@jhilgart22](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjhilgart22)%5C
**Post date:** [February 5, 2024, 8:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmove-model-with-device-map-balanced-to-cpu%2F43932%2F2 "2024-02-05T20:38:19Z")

</div>

I’m also running into this with device\_map=‘auto’