# Pre-training loss Wav2Vec2 Conformer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpre-training-loss-wav2vec2-conformer%2F33721%3E%5C
**Category:** Models\
**Created:** [March 13, 2023, 9:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpre-training-loss-wav2vec2-conformer%2F33721 "2023-03-13T21:09:14Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![morenolq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/morenolq/32/55542_2.png) [@morenolq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmorenolq)%5C
**Post date:** [March 13, 2023, 9:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpre-training-loss-wav2vec2-conformer%2F33721%2F1 "2023-03-13T21:09:15Z")

</div>

Hi there,

I am trying to pre-train the Wav2Vec2-Conformer model available on the Hugging Face Transformers library. During the training process, I am computing the loss using both masked indexes and sampled negative indexes (100 distractors). However, I have noticed that the model’s contrastive loss is around 2000, while the diversity loss is around 200. I am currently testing this training on Mozilla CV 8.0.

I am writing to ask if these losses are acceptable, or if there is something that I might be missing or doing wrong. I would appreciate any feedback or suggestions that you might have regarding this issue.

Thank you for your time and help.