# Fine tuning gpt2 for question answering
**URL:**
**Author:**  [@dimasml](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdimasml)%5C
**Post date:** [February 17, 2023, 9:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-question-answering%2F31895%2F1 "2023-02-17T09:58:01Z")
Hello, I have a code, I do not know what else I need to configure so that I can get answers to my questions. the answers are wrong. please help me to make the model answer the questions I have trained. the end goal is to run this as a tech support service so that the trained model takes care of the easy stuff.
# !pip3 install transformers
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel, AdamW
# Define the questions and answers
qa\_pairs = [
[‘Q: How to delete the account A: IOS: Go to settings on the app, click on Edit, Click on Logout, Delete the Account Android: Go to settings, Click on the three dots, Click on Logout, Delete the Account.’],
[‘Q: How to deactivate my account A: IOS: Go to settings on the app, click on Edit, Click on Logout, Delete the Account Android: Go to settings, Click on the three dots, Click on Logout, Delete the Account.’],
[‘Q: How i can remove my nmbr A: IOS: Go to settings on the app, click on Edit, Click on Logout, Delete the Account Android: Go to settings, Click on the three dots, Click on Logout, Delete the Account.’],
[‘Q: How to delete an account A: IOS: Go to settings on the app, click on Edit, Click on Logout, Delete the Account Android: Go to settings, Click on the three dots, Click on Logout, Delete the Account.’]
]
# Load the pre-trained GPT-2 model and tokenizer
model = GPT2LMHeadModel.from\_pretrained(‘gpt2’)
tokenizer = GPT2Tokenizer.from\_pretrained(‘gpt2’)
text = “”
for question in qa\_pairs:
text += f"{question} {tokenizer.eos\_token}"
# Encode the text using the tokenizer
input\_ids = tokenizer.encode(text, add\_special\_tokens=True, return\_tensors=‘pt’)
# Define the optimizer and learning rate scheduler
optimizer = AdamW(model.parameters(), lr=5e-5)
scheduler = torch.optim.lr\_scheduler.StepLR(optimizer, step\_size=1, gamma=0.9)
# Fine-tune the model
device = torch.device(“cuda” if torch.cuda.is\_available() else “cpu”)
model.to(device)
model.train()
for epoch in range(15):
optimizer.zero\_grad()
loss = model(input\_ids.to(device), labels=input\_ids.to(device))[0]
loss.backward()
optimizer.step()
scheduler.step()
```
if epoch % 100 == 0:
print(f"Epoch {epoch}, Loss {loss.item()}")
```
# Generate responses to new questions
model.eval()
def generate\_response(question):
# Encode the question using the tokenizer
input\_ids = tokenizer.encode(question + " \<|question|\>", add\_special\_tokens=False, return\_tensors=‘pt’).to(device)
```
# Generate the answer using the model
sample_output = model.generate(input_ids, do_sample=True, max_length=100, top_k=20, top_p=1.0)
# Decode the generated answer using the tokenizer
answer = tokenizer.decode(sample_output[0], skip_special_tokens=True)
# Split the generated answer into individual sentences
sentences = answer.split('. ')
# Look for the sentence that contains the answer to the question
for sentence in sentences:
if question in sentence:
return sentence
# If no sentence contains the answer, return the full generated answer
return answer
```
# Example usage
question = “How to delete an account”
response = generate\_response(question)
print(f"{question}\n {response}")
Answer: How to delete an account
How to delete an account \<|question|\>
This tool allows you to:
Remove an account
Delete an account by entering an invalid password
Delete an account by entering an invalid password
Unblock an account
Unblock an account by entering an invalid password
Unblock an account by entering an invalid password
Unblock an account by entering an invalid password
Unblock an account by entering an invalid password or by entering an invalid password
---
**Author:**  [@lukassan](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flukassan)%5C
**Post date:** [April 7, 2023, 10:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-question-answering%2F31895%2F2 "2023-04-07T10:05:09Z")
Hi, I don’t have an answer but I am about to tackle a similar problem. Do you have any advice, how did you make it work?
---
**Author:**  [@Geo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FGeo)%5C
**Post date:** [September 5, 2023, 7:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-question-answering%2F31895%2F3 "2023-09-05T07:28:34Z")
My task is the same , how did you solved your problem. I have tried everything
---
**Author:**  [@Arij](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FArij)%5C
**Post date:** [March 29, 2024, 9:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-gpt2-for-question-answering%2F31895%2F4 "2024-03-29T09:12:30Z")
The format of gpt input should be the same during the inference and train
you have for training the format Q: question text A: IOS: answer text
for generation it should be like Q: question text A: IOS: