# GPT2Tokenizer not working in Kaggle Notebook

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt2tokenizer-not-working-in-kaggle-notebook%2F41508%3E%5C
**Category:** 🤗Tokenizers\
**Created:** [May 30, 2023, 2:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt2tokenizer-not-working-in-kaggle-notebook%2F41508 "2023-05-30T14:58:06Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![admarcosai](https://avatars.discourse-cdn.com/v4/letter/a/7ab992/32.png) [@admarcosai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fadmarcosai)%5C
**Post date:** [May 30, 2023, 2:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpt2tokenizer-not-working-in-kaggle-notebook%2F41508%2F1 "2023-05-30T14:58:06Z")

</div>

Hello I have been trying to tokenize the WMT14 en-de. I first started in a free google colab and the following code worked there; I then switched to Kaggle notebook since it is a better environment but it doesn’t work there for some reason. The error it throws is below:

```auto
from transformers import GPT2Tokenizer
bpe_tokenizer = GPT2Tokenizer.from_pretrained('gpt2')

def tokenization(examples):
    source, target = [], []
    for example in examples:
        trgt = bpe_tokenizer(example['de'])
        src = bpe_tokenizer(example['en'])
        target.append(trgt)
        source.append(src)
        
    return {'de': target,
            'en': source}

train_dataset = dataset_de_en['train'].map(lambda examples: tokenization(examples['translation']), batched=True)
test_dataset = dataset_de_en['test'].map(lambda examples: tokenization(examples['translation']), batched=True)
val_dataset = dataset_de_en['val'].map(lambda examples: tokenization(examples['translation']), batched=True)

```

```auto
ArrowInvalid: Could not convert {'input_ids': [54, 798, 263, 559, 22184, 993, 1326, 4587, 311, 4224, 2150, 82, 525, 72, 1098], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]} with type BatchEncoding: did not recognize Python value type when inferring an Arrow data type

```

Is there an explanation to this?