# How to Quantization the m2m-100 418M model？？
**URL:**
**Author:**  [@doinv](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdoinv)%5C
**Post date:** [February 21, 2025, 5:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-quantization-the-m2m-100-418m-model%2F142259%2F1 "2025-02-21T17:35:11Z")
I want to quan the m2m-100 model to do the translation work, but i don’t know how to do
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [February 22, 2025, 4:40am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-quantization-the-m2m-100-418m-model%2F142259%2F2 "2025-02-22T04:40:13Z")
For example, like this.
```py
# "pip install bitsandbytes" in advance
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
from transformers import BitsAndBytesConfig # <= added
import torch # <= added
nf4_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16) # <= added
hi_text = "जीवन एक चॉकलेट बॉक्स की तरह है।"
chinese_text = "生活就像一盒巧克力。"
model = M2M100ForConditionalGeneration.from_pretrained("facebook/m2m100_418M", quantization_config=nf4_config) # <= modified
tokenizer = M2M100Tokenizer.from_pretrained("facebook/m2m100_418M")
# translate Hindi to French
tokenizer.src_lang = "hi"
encoded_hi = tokenizer(hi_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_hi, forced_bos_token_id=tokenizer.get_lang_id("fr"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
# => "La vie est comme une boîte de chocolat."
# translate Chinese to English
tokenizer.src_lang = "zh"
encoded_zh = tokenizer(chinese_text, return_tensors="pt")
generated_tokens = model.generate(**encoded_zh, forced_bos_token_id=tokenizer.get_lang_id("en"))
tokenizer.batch_decode(generated_tokens, skip_special_tokens=True)
# => "Life is like a box of chocolate."
```
> **[facebook/m2m100\_418M · Hugging Face](https://huggingface.co/facebook/m2m100_418M)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [February 25, 2025, 3:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-quantization-the-m2m-100-418m-model%2F142259%2F3 "2025-02-25T03:03:51Z")
Closed on February 25, 2025, 3:03am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.