# Optimum roberta base quantization model recall drop 10%
**URL:**
**Author:**  [@Ivan1999](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIvan1999)%5C
**Post date:** [December 1, 2023, 11:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F1 "2023-12-01T11:31:08Z")
We use roberta base model for a binary classification problem, and I tried quantization using torch, onnxruntime and Optimum, both torch and onnxruntime quantization model recall drop around 5%, but Optimum quantization model recall drop around 10%, for all these models, precision is same as the original model.
Code:
```auto
ort_model = ORTModelForSequenceClassification.from_pretrained(model_path, export=True)
quantizer = ORTQuantizer.from_pretrained(ort_model)
dqconfig = AutoQuantizationConfig.arm64(
is_static=False,
per_channel=True,
)
model_quantized_path = quantizer.quantize(
save_dir=onnx_model_path,
quantization_config=dqconfig,
)
```
---
**Author:**  [@Ivan1999](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIvan1999)%5C
**Post date:** [December 1, 2023, 11:34am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F2 "2023-12-01T11:34:02Z")
I tried different params in `AutoQuantizationConfig.arm64` function, but the result is similar.
By the way, how can I determine which nodes to exclude in quantization, is there a tool?
---
**Author:**  [@echarlaix](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fecharlaix)%5C
**Post date:** [December 1, 2023, 2:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F3 "2023-12-01T14:12:58Z")
Hi @Ivan1999,
To check out which nodes are sensitive to quantization, you can create a quantization preprocessor (as done [here](https://github.com/huggingface/optimum/blob/e2d3c8b6765e66c606b1cb59e3bb3e95453217e4/examples/onnxruntime/quantization/text-classification/run_glue.py#L467-L476)), to iteratively exclude operations which can result in significant drop in accuracy when quantized (such as GELU, LayerNorm, softmax)
---
**Author:**  [@Ivan1999](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIvan1999)%5C
**Post date:** [December 1, 2023, 3:58pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F5 "2023-12-01T15:58:06Z")
@echarlaix , I tried, not work,
```auto
# Create a quantization preprocessor to determine the nodes to exclude when applying static quantization
quantization_preprocessor = QuantizationPreprocessor()
# Exclude the nodes constituting LayerNorm
quantization_preprocessor.register_pass(ExcludeLayerNormNodes())
# Exclude the nodes constituting GELU
quantization_preprocessor.register_pass(ExcludeGeLUNodes())
# Exclude the residual connection Add nodes
quantization_preprocessor.register_pass(ExcludeNodeAfter("Add", "Add"))
# Exclude the Add nodes following the Gather operator
quantization_preprocessor.register_pass(ExcludeNodeAfter("Gather", "Add"))
# Exclude the Add nodes followed by the Softmax operator
quantization_preprocessor.register_pass(ExcludeNodeFollowedBy("Add", "Softmax"))
```
---
**Author:**  [@regisss](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fregisss)%5C
**Post date:** [January 12, 2024, 4:32pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F6 "2024-01-12T16:32:53Z")
Can you show the script you used to quantize the model with ONNNXRuntime (i.e. without Optimum) please?
---
**Author:**  [@Ivan1999](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FIvan1999)%5C
**Post date:** [January 15, 2024, 3:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Foptimum-roberta-base-quantization-model-recall-drop-10%2F64240%2F8 "2024-01-15T03:02:10Z")
It’s been a long time, after some configuration I made both onnx and optimum quantization recall drop around 6%.