# 4bit quantization on inference end point

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2F4bit-quantization-on-inference-end-point%2F69366%3E%5C
**Category:** Inference Endpoints on the Hub\
**Created:** [January 16, 2024, 1:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2F4bit-quantization-on-inference-end-point%2F69366 "2024-01-16T13:06:09Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dbshift](https://avatars.discourse-cdn.com/v4/letter/d/ba8739/32.png) [@dbshift](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdbshift)%5C
**Post date:** [January 16, 2024, 1:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2F4bit-quantization-on-inference-end-point%2F69366%2F1 "2024-01-16T13:06:09Z")

</div>

I need to know is there a way to quantize a model in 4bit and run it in inference end point. currently it is taking 8bit quantization as default. one more thing is can I do my custom quantization in handler.py file?