# Fine-tuning Whisper for Audio Classification
**URL:**
**Author:**  [@Zahra99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZahra99)%5C
**Post date:** [June 27, 2023, 1:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F1 "2023-06-27T13:33:32Z")
Hi, I hope this message finds you well. I am reaching out to seek clarification regarding fine-tuning the Whisper model for audio classification.
I have been exploring the Hugging Face resources and documentation, but unfortunately, I couldn’t find any specific blogs or example notebooks specifically addressing the fine-tuning of Whisper for audio classification tasks. Most of the available resources appear to focus on Automatic Speech Recognition (ASR) tasks instead. Nonetheless, I experiment and implemented some code to fine-tune Whisper for audio classification. After a considerable amount of time and effort, I managed to make progress without encountering any errors. However, due to the lack of dedicated resources, I would appreciate confirmation that the approach I followed is indeed correct.
What I did for fine-tuning Whisper for audio classification was similar to the process used for fine-tuning other transformer models like Wav2Vec and Hubert, with the only difference being the function for the feature extractor. In my implementation, I defined a function as follows:
```auto
def prepare_dataset(batch):
# load and resample audio data from 48 to 16kHz
audio = batch["audio"]
# compute log-Mel input features from input audio array
batch["input_features"] = feature_extractor(audio["array"], sampling_rate=audio["sampling_rate"]).input_features[0]
# there is no need for encoding label
batch["label"] = batch["label"]
return batch
```
And after that I applied this function to the entire dataset in the following manner:
```auto
encoded_dataset = dataset.map(prepare_dataset, remove_columns="audio", num_proc=4)
```
My question is, can I confirm whether this approach is correct and valid?
And if not, I kindly request your guidance or insights on the appropriate methodology for fine-tuning Whisper for audio classification.
I appreciate any help 🙏
---
**Author:**  [@Zahra99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZahra99)%5C
**Post date:** [July 4, 2023, 6:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F2 "2023-07-04T06:23:56Z")
@sanchit-gandhi
Could you please help me with this issue?
---
**Author:**  [@avar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Favar)%5C
**Post date:** [July 9, 2023, 4:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F3 "2023-07-09T04:11:52Z")
Hi,
Could you give more details? Or the process of whole fine-tuning? Thanks
---
**Author:**  [@mirix](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmirix)%5C
**Post date:** [October 13, 2023, 1:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F4 "2023-10-13T13:27:50Z")
I was considering attempting a similar approach.
How do the results compare to those obtained with other models such as wav2vec2?
---
**Author:**  [@mdege](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmdege)%5C
**Post date:** [October 9, 2024, 12:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F5 "2024-10-09T12:50:34Z")
I’m a newbie to the finetuning process, and would also like to try and finetune whisper on classification tasks. Do you have a github with a more detailed code, or more information ?
Can you use your personal datasets ?
Thank you !
---
**Author:**  [@Zahra99](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZahra99)%5C
**Post date:** [October 9, 2024, 8:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F6 "2024-10-09T20:25:20Z")
Hi @ [mdege](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmdege)%2C
You can find the code in my repository here: [GitHub link](https://github.com/ZahraDehghani99/Audio-Emotion-Recognition-using-Whisper/blob/main/whisper_finetuning.ipynb)
Regarding the dataset, I used the IEMOCAP dataset for fine-tuning, but you can definitely try it with your own dataset.
---
**Author:**  [@thelou1s](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fthelou1s)%5C
**Post date:** [November 8, 2024, 3:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffine-tuning-whisper-for-audio-classification%2F44735%2F7 "2024-11-08T03:03:48Z")
what’s the most accurate model for audio classification? from user side