# Mistral Unrecognized configuration class

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmistral-unrecognized-configuration-class%2F84694%3E%5C
**Category:** 🤗AutoTrain\
**Created:** [May 1, 2024, 3:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmistral-unrecognized-configuration-class%2F84694 "2024-05-01T15:10:57Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nelis5174473](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nelis5174473/32/26457_2.png) [@Nelis5174473](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNelis5174473)%5C
**Post date:** [May 1, 2024, 3:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmistral-unrecognized-configuration-class%2F84694%2F1 "2024-05-01T15:10:57Z")

</div>

I’m using autotrain to finetune an already fine-tuned Question Answering model ‘BramVanroy/GEITje-7B-ULTRA’, which is based on Mistral 7B and further pretrained on Dutch data.

 ![Screenshot 2024-05-01 at 17.09.48](https://us1.discourse-cdn.com/hellohellohello/original/3X/9/8/98f5438e5b4402bc68a2075662df36241a872fd4.png)

I have some context specific question-answer pairs I’d like to further finetune the model with. However, when using a simple autotrain space with the following settings (see picture), I get the following error:

_aautotrain.trainers.common:wrapper:121 - Unrecognized configuration class \<class ‘transformers.models.mistral.configuration\_mistral.MistralConfig’\> for this kind of AutoModel: AutoModelForSeq2SeqLM.  
Model type should be one of BartConfig, BigBirdPegasusConfig, BlenderbotConfig, BlenderbotSmallConfig, EncoderDecoderConfig, FSMTConfig, GPTSanJapaneseConfig, LEDConfig, LongT5Config, M2M100Config, MarianConfig, MBartConfig, MT5Config, MvpConfig, NllbMoeConfig, PegasusConfig, PegasusXConfig, PLBartConfig, ProphetNetConfig, SeamlessM4TConfig, SeamlessM4Tv2Config, SwitchTransformersConfig, T5Config, UMT5Config, XLMProphetNetConfig…_

What could I be doing wrong?

---

<div class="post-metadata">

**Author:** ![Nelis5174473](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nelis5174473/32/26457_2.png) [@Nelis5174473](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNelis5174473)%5C
**Post date:** [May 2, 2024, 9:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmistral-unrecognized-configuration-class%2F84694%2F2 "2024-05-02T09:48:53Z")

</div>

I figured out the solution: the model `BramVanroy/GEITje-7B-ULTRA` uses the Mistral architecture. The error message showed that this architecture doesn’t work with ‘Sequence to Sequence’ tasks. So, I needed to choose **LLM SFT** (Supervised Fine-Tuning) instead. This approach is meant for refining language models using labeled data. In my case, I used question-answer pairs related to a specific domain. Here, each question is the input and its corresponding answer is the output. After switching to LLM SFT, the error disappeared and I could fine-tune the model successfully."

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [May 2, 2024, 9:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmistral-unrecognized-configuration-class%2F84694%2F3 "2024-05-02T21:49:44Z")

</div>

Closed on May 2, 2024, 9:49pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.