# Multi-task instruction fine-tuning

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-task-instruction-fine-tuning%2F50310%3E%5C
**Category:** Amazon SageMaker\
**Created:** [August 10, 2023, 5:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-task-instruction-fine-tuning%2F50310 "2023-08-10T17:13:42Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorgeutd](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/jorgeutd/32/39292_2.png) [@Jorgeutd](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJorgeutd)%5C
**Post date:** [August 10, 2023, 5:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-task-instruction-fine-tuning%2F50310%2F1 "2023-08-10T17:13:42Z")

</div>

Hello everyone,

I have a question regarding supervise instruction fine-tuning LLM models. Most of the examples and documentation I’ve come across focus on single-task instruction fine-tuning. I am wondering if it’s possible to perform multi-task instruction fine-tuning, for instance, using two or three different tasks concurrently with some of these models, such as Llama2, Falcon, etc.?

Any insights or guidance would be greatly appreciated. Thank you!

---

<div class="post-metadata">

**Author:** ![CKeibel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ckeibel/32/55416_2.png) [@CKeibel](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCKeibel)%5C
**Post date:** [February 2, 2024, 1:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmulti-task-instruction-fine-tuning%2F50310%2F2 "2024-02-02T13:17:01Z")

</div>

Hey!

It’s been a while since you asked your question but I thought I’d answer it in case someone comes across this post later.

It’s actually a good idea to fine tune a model with several different tasks, so the model doesn’t lose the skills it had before.  
For example, if you train a model exclusively on language translation, it may lose the ability to summarize texts. (catastrophic forgetting)  
You can easily fine-tune your model by creating a data set that contains different tasks.

For additional information, I can highly recommend the book “Generative AI on AWS” by Chris Fregly or the Coursera course based on it, also by Chris Fregly.