# Error: Fine-tune GPT2 model for question answer task **URL:** **Author:** ![tsan1](https://avatars.discourse-cdn.com/v4/letter/t/53a042/32.png) [@tsan1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftsan1)%5C **Post date:** [June 15, 2023, 6:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-fine-tune-gpt2-model-for-question-answer-task%2F43415%2F1 "2023-06-15T18:45:01Z") Hi All, I have a working script for fine-tuning t5 model for question answer task. I want to adapt the script for fine-tuning gpt2 model for the same question answer task on the same dataset. My dataset looks like: ‘context’ = … ‘question’ = … ‘answer’ = … I am getting some errors with this script. **Please can you help me resolve this error.** The error is : /usr/local/lib/python3.10/dist-packages/torch/nn/functional.py in embedding(input, weight, padding\_idx, max\_norm, norm\_type, scale\_grad\_by\_freq, sparse) 2208 # remove once script supports set\_grad\_enabled 2209 _no\_grad\_embedding\_renorm_(weight, input, max\_norm, norm\_type) → 2210 return torch.embedding(weight, input, padding\_idx, scale\_grad\_by\_freq, sparse) 2211 2212 RuntimeError: CUDA error: device-side assert triggered Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions. **The adapted script for fine-tuning gpt2 is :** import sys import torch import pandas as pd import json import os os.environ[‘CUDA\_LAUNCH\_BLOCKING’] = ‘1’ from transformers import GPT2LMHeadModel, GPT2Tokenizer, TrainingArguments, Trainer from transformers import DataCollatorWithPadding from datasets import Dataset # global var ignore\_pad\_token\_for\_loss = True padding = “max\_length” def preprocess\_data(question, context, answer, max\_length=512): inputs = tokenizer( question, context, max\_length=max\_length, truncation=“only\_second”, padding=“max\_length”, return\_attention\_mask=True, add\_special\_tokens=True, return\_tensors=“pt” ) input\_ids = inputs[‘input\_ids’].squeeze() attention\_mask = inputs[‘attention\_mask’].squeeze() ``` # Encode the answer # Tokenize targets with text_target=... labels = tokenizer(text_target=answer, max_length=max_length, padding=padding, truncation=True) # If we are padding here, replace all tokenizer.pad_token_id in the labels by -100 when we want to ignore # padding in the loss. if padding == "max_length" and ignore_pad_token_for_loss: labels["input_ids"] = [(l if l != tokenizer.pad_token_id else -100) for l in labels["input_ids"]] inputs["labels"] = labels["input_ids"] return { 'input_ids': input_ids, 'attention_mask': attention_mask, 'labels': inputs["labels"] } ``` model = GPT2LMHeadModel.from\_pretrained(“gpt2”) # Set device to GPU if available device = torch.device(“cuda” if torch.cuda.is\_available() else “cpu”) print(device) # Load custom training dataset as pandas DataFrame with open(“augmented\_data.json”, “r”) as f: data = json.load(f) data = pd.DataFrame(data) data = data.drop(columns=[“id”]) # remove the id column # Convert pandas DataFrame to Hugging Face Dataset format train\_dataset = Dataset.from\_pandas(data) print(train\_dataset.shape) # Load custom validation dataset with open(“validation\_set.json”, “r”) as f: data = json.load(f) data = pd.DataFrame(data) data = data.drop(columns=[“id”]) # remove the id column # Convert pandas DataFrame to Hugging Face Dataset format val\_dataset = Dataset.from\_pandas(data) print(val\_dataset.shape) # define tokenizer tokenizer = GPT2Tokenizer.from\_pretrained(“gpt2”) tokenizer.add\_special\_tokens({‘pad\_token’: ‘[PAD]’}) # Preprocess dataset preprocessed\_train\_dataset = preprocessed\_val\_dataset = for example in train\_dataset: preprocessed\_example = preprocess\_data(example[“question”], example[“context”], example[“answer”]) preprocessed\_train\_dataset.append(preprocessed\_example) for example in val\_dataset: preprocessed\_example = preprocess\_data(example[“question”], example[“context”], example[“answer”]) preprocessed\_val\_dataset.append(preprocessed\_example) # Convert preprocessed dataset to Hugging Face Dataset format tokenized\_train\_dataset = Dataset.from\_dict( {key: [example[key] for example in preprocessed\_train\_dataset] for key in preprocessed\_train\_dataset[0].keys()}) print(tokenized\_train\_dataset) tokenized\_val\_dataset = Dataset.from\_dict( {key: [example[key] for example in preprocessed\_val\_dataset] for key in preprocessed\_val\_dataset[0].keys()}) print(tokenized\_val\_dataset) “”“Define training arguments”“” training\_args = TrainingArguments( output\_dir=“qa\_model”, evaluation\_strategy=“epoch”, learning\_rate=15e-6, per\_device\_train\_batch\_size=4, per\_device\_eval\_batch\_size=4, num\_train\_epochs=100, weight\_decay=0.01, push\_to\_hub=False, save\_strategy=‘epoch’, ) “”“Define data collator”“” data\_collator = DataCollatorWithPadding(tokenizer=tokenizer) “”“Instantiate the Trainer class”“” trainer = Trainer( model=model, args=training\_args, train\_dataset=tokenized\_train\_dataset, eval\_dataset=tokenized\_val\_dataset, data\_collator=data\_collator, ) trainer.train() trainer.save\_model(“final\_result”) print(" Training is Over ") ---
**Author:** ![tsan1](https://avatars.discourse-cdn.com/v4/letter/t/53a042/32.png) [@tsan1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftsan1)%5C **Post date:** [June 16, 2023, 8:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ferror-fine-tune-gpt2-model-for-question-answer-task%2F43415%2F2 "2023-06-16T20:20:59Z")
Please could anyone help me with this. Thank you