# Evaluating with quantized model using Trainer

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluating-with-quantized-model-using-trainer%2F82627%3E%5C
**Category:** Beginners\
**Created:** [April 19, 2024, 11:55am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluating-with-quantized-model-using-trainer%2F82627 "2024-04-19T11:55:39Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![daltonh](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@daltonh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdaltonh)%5C
**Post date:** [April 19, 2024, 11:55am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluating-with-quantized-model-using-trainer%2F82627%2F1 "2024-04-19T11:55:39Z")

</div>

I have quantized a model using BitsAndBytes, and I want to evaluate this model on some benchmark tasks. For evaluation on other models, I use trainer.evaluate() to obtain the metrics.  
This is however different for quantized models. I use the following code:

```auto
...
trainer = Trainer(
    model=model_quantized,
    ....
)
trainer.evaluate(test_data)
...

```

I obtain the following error:

```auto
ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: https://huggingface.co/docs/transformers/peft for more details

```

I understand that the error gets raised, as I am using the Trainer object on a quantized model, but I do not intend to train this model, and only perfom evaluation.  
Is there a way to avoid this error and run evaluation on a quantized model?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![daminho](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/daminho/32/29506_2.png) [@daminho](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdaminho)%5C
**Post date:** [August 10, 2024, 11:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fevaluating-with-quantized-model-using-trainer%2F82627%2F2 "2024-08-10T11:14:50Z")

</div>

> **[🤗 Evaluate](https://huggingface.co/docs/evaluate/en/index)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

You can try to use evaluate library from huggingface instead