# SFTTrainer and wikitext dataset

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-and-wikitext-dataset%2F77679%3E%5C
**Category:** Beginners\
**Created:** [March 16, 2024, 2:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-and-wikitext-dataset%2F77679 "2024-03-16T14:27:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Khaledai](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@Khaledai](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FKhaledai)%5C
**Post date:** [March 16, 2024, 2:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsfttrainer-and-wikitext-dataset%2F77679%2F1 "2024-03-16T14:27:44Z")

</div>

I downloaded the wikitext dataset:

```auto
from datasets import load_dataset

dataset = load_dataset("wikitext",'wikitext-103-raw-v1')

```

After downloading it I used SFTtrainer:

```auto
from trl import SFTTrainer

max_seq_length = 512

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    peft_config=peft_config,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    tokenizer=tokenizer,
    args=training_arguments,
)

```

After using `trainer.train()` it gave the error "IndexError: Invalid key: 1593817 is out of bounds for size 0  
"