# Colab error (memory crashes)

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429%3E%5C
**Category:** Beginners\
**Created:** [October 8, 2020, 6:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429 "2020-10-08T18:47:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 8, 2020, 6:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429%2F1 "2020-10-08T18:47:43Z")

</div>

I have this trainer code on a sample of only 10,000 records, still the GPU runs out, I am using Google Colab pro, before that it didnt happen with me, something wrong in my code, please see

```
from transformers import DistilBertForSequenceClassification, Trainer, TrainingArguments

training_args = TrainingArguments(

    output_dir='/content/drive/My Drive/results/distillbert', # output directory

    overwrite_output_dir= True,

    do_predict= True, 

    num_train_epochs=3, # total number of training epochs

    per_device_train_batch_size=4, # batch size per device during training

    per_device_eval_batch_size=2, # batch size for evaluation

    warmup_steps=1000, # number of warmup steps for learning rate scheduler

    save_steps=1000,

    save_total_limit=10,

    load_best_model_at_end= True,

    weight_decay=0.01, # strength of weight decay

    logging_dir='./logs', # directory for storing logs

    logging_steps=0,

)

model = DistilBertForSequenceClassification.from_pretrained("distilbert-base-cased")

trainer = Trainer(

    model=model, # the instantiated 🤗 Transformers model to be trained

    args=training_args, # training arguments, defined above

    train_dataset=train_dataset, # training dataset

    eval_dataset=val_dataset # evaluation dataset

)
```

---

<div class="post-metadata">

**Author:** ![rgwatwormhill](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@rgwatwormhill](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frgwatwormhill)%5C
**Post date:** [October 8, 2020, 8:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429%2F2 "2020-10-08T20:28:30Z")

</div>

How big is each record? How big is it after tokenization?  
Are you using a data-loader? What batchsize is it using?

What happens if you you try to train using only 10 records?

---

<div class="post-metadata">

**Author:** ![shainaraza](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/shainaraza/32/2854_2.png) [@shainaraza](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fshainaraza)%5C
**Post date:** [October 8, 2020, 8:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429%2F3 "2020-10-08T20:51:22Z")

</div>

thanks, it is resolved, after I made following changes  
per\_device\_train\_batch\_size=4, # batch size per device during training  
per\_device\_eval\_batch\_size=2,

not sure if it is good approach?  
thanks

---

<div class="post-metadata">

**Author:** ![alikeram](https://avatars.discourse-cdn.com/v4/letter/a/d78d45/32.png) [@alikeram](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falikeram)%5C
**Post date:** [April 22, 2021, 11:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcolab-error-memory-crashes%2F1429%2F4 "2021-04-22T11:37:08Z")

</div>

I handled the problem by getting the vectors of the dataset at different batches not at once.