# Continual training on LLM like Whisper

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinual-training-on-llm-like-whisper%2F36314%3E%5C
**Category:** 🤗Transformers\
**Created:** [April 12, 2023, 7:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinual-training-on-llm-like-whisper%2F36314 "2023-04-12T07:42:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![bjubert](https://avatars.discourse-cdn.com/v4/letter/b/a587f6/32.png) [@bjubert](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbjubert)%5C
**Post date:** [April 12, 2023, 7:42am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcontinual-training-on-llm-like-whisper%2F36314%2F1 "2023-04-12T07:42:38Z")

</div>

Hello,

I fine-tuned the medium model of Whisper with my own data set (a set of transcribed audio files for a specific case). The results are good but may be better due to the small size of my own data set. I will be able to obtain new transcribed audio recordings in the future. So my questions are, is it possible to make continuous learning with Whisper? Is there a possiblity of catastrophic forgetting and how to avoid it ?

thanks in advance.