# BitsAndBytesConfig is not compitable in TPU env
**URL:**
**Author:**  [@VasudevaK](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FVasudevaK)%5C
**Post date:** [July 6, 2024, 7:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbitsandbytesconfig-is-not-compitable-in-tpu-env%2F95817%2F1 "2024-07-06T07:21:04Z")
`No GPU found error is raised`. How can I quantize llama in the TPU environment? Here is the code I used.
```auto
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=False,
bnb_4bit_quant_type="nf4")
base_model = LlamaForSequenceClassification.from_pretrained(
CFG.MODEL_NAME,
num_labels=CFG.NUM_LABELS,
torch_dtype=torch.bfloat16,
quantization_config=bnb_config,)
base_model.config.pretraining_tp = 1
base_model.config.pad_token_id = tokenizer.pad_token_id
```
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [July 6, 2024, 10:29am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbitsandbytesconfig-is-not-compitable-in-tpu-env%2F95817%2F2 "2024-07-06T10:29:12Z")
Hi,
That’s correct, BitsandBytes quantization only supports CUDA: [GitHub - TimDettmers/bitsandbytes: Accessible large language models via k-bit quantization for PyTorch.](https://github.com/TimDettmers/bitsandbytes).
You might need to take a look at other quantization methods: [Quantization](https://huggingface.co/docs/transformers/en/main_classes/quantization)
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [July 6, 2024, 10:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbitsandbytesconfig-is-not-compitable-in-tpu-env%2F95817%2F3 "2024-07-06T22:29:13Z")
Closed on July 6, 2024, 10:29pm UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.