# Fine-Tune GPT-2 Spanish From Example Notebook OOM

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt-2-spanish-from-example-notebook-oom%2F2870%3E%5C
**Category:** Beginners\
**Created:** [December 17, 2020, 10:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt-2-spanish-from-example-notebook-oom%2F2870 "2020-12-17T22:32:11Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![enrico](https://avatars.discourse-cdn.com/v4/letter/e/e495f1/32.png) [@enrico](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fenrico)%5C
**Post date:** [December 17, 2020, 10:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tune-gpt-2-spanish-from-example-notebook-oom%2F2870%2F1 "2020-12-17T22:32:12Z")

</div>

Hi!

I was hoping to adopt the approach from one of the official notebooks described here: [https://www.philschmid.de/fine-tune-a-non-english-gpt-2-model-with-huggingface](https://www.philschmid.de/fine-tune-a-non-english-gpt-2-model-with-huggingface)

I wanted to switch the german model for a spanish one I found on the hub (ensamblador/gpt2-es-48heads) but I’m running out of memory on the colab notebook at this point:

```
from transformers import TextDataset,DataCollatorForLanguageModeling

def load_dataset(train_path,test_path,tokenizer):
    train_dataset = TextDataset(
          tokenizer=tokenizer,
          file_path=train_path,
          block_size=128)
     
    test_dataset = TextDataset(
          tokenizer=tokenizer,
          file_path=test_path,
          block_size=128)   
    
    data_collator = DataCollatorForLanguageModeling(
        tokenizer=tokenizer, mlm=False,
    )
    return train_dataset,test_dataset,data_collator

train_dataset,test_dataset,data_collator = load_dataset(train_path,test_path,tokenizer)

```

My training set is about 200 mb. The obvious solution to me would be to increase RAM via another GPU in a Google Ai Notebook.

Is there a way to avoid OOM on the free colab notebooks with one GPU?