# Wav2Vec2 for Audio Emotion Classification
**URL:**
**Author:**  [@Winstead](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWinstead)%5C
**Post date:** [March 11, 2021, 3:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F1 "2021-03-11T15:00:10Z")
We are having a thesis project on _Podcast Trailer Generation - Hotspot Detection_ for Podcast Dataset at Spotify.
The Spotify Podcast Dataset contains both transcript and audio data for many podcast episodes, and currently we are looking to use Wav2Vec2 embeddings as input to train an emotion classification model for the audio data. The audio data is currently only in English (with accompanied transcript).
It would be much appreciated if you could help out with fine-tuning Wav2Vec2 on some standard emotion-annotated audio datasets (e.g. [RAVDESS](https://www.kaggle.com/uwrfkaggler/ravdess-emotional-speech-audio), [SAVEE](https://www.kaggle.com/ejlok1/surrey-audiovisual-expressed-emotion-savee)). We will then use the fine-tuned embeddings as input for emotion classification, after which we will have human evaluation on the classified results.
---
**Author:**  [@birgermoell](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbirgermoell)%5C
**Post date:** [March 11, 2021, 3:14pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F2 "2021-03-11T15:14:50Z")
That sounds great. I’m also working with fine-tuning Wav2Vec2. I can help you out if you have any questions. @patrickvonplaten is also a great person to ask.
---
**Author:**  [@patrickvonplaten](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpatrickvonplaten)%5C
**Post date:** [March 15, 2021, 6:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F3 "2021-03-15T06:04:14Z")
Hey @Winstead,
Thanks for your post here! I think it would be a good idea to use Wav2Vec2 for emotion classification. I won’t find time to fine-tune the model myself any time soon, but it should be rather straightforward to do so. Things that need to be done before being able to fine-tune Wav2Vec2 on emotion classification.
1.
- Add a `Wav2Vec2ForSpeechClassification` model that would be very similar to how `BertForSequenceClassification` is implemented.
2.
- It would be probably much easier to train the model if the two datasets you linked above would be added to `datasets`. It should be rather straight-forward to do this yourself or else you can put up a `dataset request` issue in the library and maybe someone in the open-source community is interested in tackling the issue. See those issues for example: [Issues · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues?q=label%3Aspeech+is%3Aclosed)
3. Having added those things it should be rather straight-forward to train the model. You can look for “transformers” sentiment analysis online to get a feeling for how it should be done. See this article for example: [https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/](https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/)
---
**Author:**  [@othrif](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fothrif)%5C
**Post date:** [April 26, 2021, 12:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F4 "2021-04-26T12:48:42Z")
@Winstead, have you made any progress on this?
I am also working on a similar project and will be happy to collaborate/assist.
---
**Author:**  [@Winstead](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWinstead)%5C
**Post date:** [April 26, 2021, 3:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F5 "2021-04-26T15:07:01Z")
Hi @othrif !
I’ve written the basic code and trained on some emotion-annotated speech dataset, but the accuracy has not been good so far. As I’m also working on several other approaches in the meantime and haven’t spent a lot of time on fine-tuning wav2vec2, I believe there is a lot of room for improvement.
And yes, I would be glad to collaborate/discuss on this. How would you prefer to communicate?
---
**Author:**  [@m3hrdadfi](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fm3hrdadfi)%5C
**Post date:** [May 25, 2021, 8:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F6 "2021-05-25T08:28:40Z")
@Winstead, It would probably solve your problem.
> [@Using Wav2Vec in speech classification/regression problems](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fusing-wav2vec-in-speech-classification-regression-problems%2F6361)%3A
>
> Hi Wav2Vec enthusiasts, I created a script for using Wav2Vec 2.0 in speech classification/regression problems. I tested the model on Persian and Greek and got significant results even way better than what they proposed in their papers. I’m going to share this script/model with you to take advantage of it in your research, and let me know your results and feedback. Repo: [GitHub - m3hrdadfi/soxan: Wav2Vec2 for speech recognition and classification](https://github.com/m3hrdadfi/soxan) Notebook: [https://colab.research.google.com/gi…](https://colab.research.google.com/github/m3hrdadfi/soxan/blob/main/notebooks/Emotion_recognition_in_Greek_speech_using_Wav2Vec2.ipynb)
---
**Author:**  [@Winstead](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWinstead)%5C
**Post date:** [May 26, 2021, 3:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwav2vec2-for-audio-emotion-classification%2F4312%2F7 "2021-05-26T15:50:35Z")
@m3hrdadfi Awesome! Thank you so much!