# Float16 on CPU torch support

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffloat16-on-cpu-torch-support%2F69427%3E%5C
**Category:** Beginners\
**Created:** [January 16, 2024, 9:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffloat16-on-cpu-torch-support%2F69427 "2024-01-16T21:04:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rmiller3](https://avatars.discourse-cdn.com/v4/letter/r/c5a1d2/32.png) [@rmiller3](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frmiller3)%5C
**Post date:** [January 16, 2024, 9:04pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffloat16-on-cpu-torch-support%2F69427%2F1 "2024-01-16T21:04:56Z")

</div>

I’ve read about torch not supporting float16 on CPU when trying to test microsoft/phi2 model.  
Example threads:

- [microsoft/phi-2 · RuntimeError: "LayerNormKernelImpl" not implemented for 'Half'](https://huggingface.co/microsoft/phi-2/discussions/14)
- [Better error message when trying to run fp16 weights on CPU · Issue #96292 · pytorch/pytorch · GitHub](https://github.com/pytorch/pytorch/issues/96292)

How come the following code works on a GPT2 model where I’m seemingly setting it to run in that config? Is torch\_dtype somehow different than the topic in those comments?

```auto
        pipe = transformers.pipeline('text-generation', model=model, tokenizer=tokenizer, device="cpu", torch_dtype=torch.float16)

        res = pipe("Here is a recipe for vegan banana bread:\n",
                           max_new_tokens=max_new_tokens,
                           do_sample=False,
                           use_cache=True)

```