# Dataset for training BlenderBot

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-for-training-blenderbot%2F5853%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 29, 2021, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-for-training-blenderbot%2F5853 "2021-04-29T12:56:12Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alokin](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alokin/32/1532_2.png) [@Alokin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAlokin)%5C
**Post date:** [April 29, 2021, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-for-training-blenderbot%2F5853%2F1 "2021-04-29T12:56:12Z")

</div>

I am trying to build a chatbot using BlenderbotForConditionalGeneration. I am using the pretrained model, however, I have to fine-tune it. My question is how should the training data look like and is there any tutorials how should I preprocess it in order to fine-tune the model?

Thank you! 🤗

---

<div class="post-metadata">

**Author:** ![b7leung](https://avatars.discourse-cdn.com/v4/letter/b/ee7513/32.png) [@b7leung](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fb7leung)%5C
**Post date:** [May 1, 2021, 7:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-for-training-blenderbot%2F5853%2F2 "2021-05-01T07:36:45Z")

</div>

I also want to fine-tune blenderbot with custom data. I noticed the [blended\_skill\_talk dataset page](https://huggingface.co/datasets/blended_skill_talk) says that [blenderbot weights](https://huggingface.co/facebook/blenderbot-400M-distill) use that to train on. So, it might make sense to structure our own custom training data like the blended\_skill\_talk dataset. But, I’m not sure, since when I tried to train on the blended\_skill\_talk dataset, like this:

```
from transformers import BlenderbotForConditionalGeneration, Trainer, TrainingArguments
from datasets import load_dataset

mname = 'facebook/blenderbot-400M-distill'
model = BlenderbotForConditionalGeneration.from_pretrained(mname).to('cuda')
train_dataset = load_dataset("blended_skill_talk", split="train")
val_dataset = load_dataset("blended_skill_talk", split="validation")
training_args = TrainingArguments("tmp_trainer")
trainer = Trainer(model=model, args=training_args, train_dataset=train_dataset, 
eval_dataset=val_dataset)
trainer.train()

```

I get an error (ValueError: too many dimensions ‘str’). I think it has to do with the collator, but I’m not sure how to fix this.