# Inference 8 bit or 4 bit bit models on cpu?
**URL:**
**Author:**  [@sd3ntato](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsd3ntato)%5C
**Post date:** [July 18, 2023, 5:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-8-bit-or-4-bit-bit-models-on-cpu%2F47237%2F1 "2023-07-18T17:57:10Z")
hello, is it possible to run inference of quantized 8 bit or 4 bit models on cpu?
---
**Author:**  [@YaTharThShaRma999](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FYaTharThShaRma999)%5C
**Post date:** [August 2, 2023, 9:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-8-bit-or-4-bit-bit-models-on-cpu%2F47237%2F2 "2023-08-02T21:06:41Z")
I don’t believe so since the bitsandbytes library is just a wrapper around cuda functions which are for gpu.
---
**Author:**  [@sd3ntato](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsd3ntato)%5C
**Post date:** [August 3, 2023, 8:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-8-bit-or-4-bit-bit-models-on-cpu%2F47237%2F3 "2023-08-03T08:14:03Z")
for those still searching, I found some sources about
- optimum intel for quantization on intel cpus
[🤗 Optimum Intel](https://huggingface.co/docs/optimum/intel/index)
- core ml has quantization tools for apple cpus
[Compressing Neural Network Weights](https://coremltools.readme.io/docs/quantization)