# Possible to use transformers with GGML-style quantization?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpossible-to-use-transformers-with-ggml-style-quantization%2F72648%3E%5C
**Category:** 🤗Transformers\
**Created:** [February 10, 2024, 2:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpossible-to-use-transformers-with-ggml-style-quantization%2F72648 "2024-02-10T14:50:13Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![neilf79](https://avatars.discourse-cdn.com/v4/letter/n/cc9497/32.png) [@neilf79](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fneilf79)%5C
**Post date:** [February 10, 2024, 2:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fpossible-to-use-transformers-with-ggml-style-quantization%2F72648%2F1 "2024-02-10T14:50:13Z")

</div>

Hi!

Is it possible to use the transformers library with GGML-style quantization? For example, GGML\_TYPE\_Q5\_K (as detailed [here](https://huggingface.co/TheBloke/CodeLlama-7B-GGUF#explanation-of-quantisation-methods)).

I’m aware that I can use bitsandbytes to do 8-bit and 4-bit quantization, but I’d prefer to use GGML-style quantization.

Thanks!