# Endpoint issue with GPTQ

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fendpoint-issue-with-gptq%2F70304%3E%5C
**Category:** Inference Endpoints on the Hub\
**Created:** [January 23, 2024, 6:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fendpoint-issue-with-gptq%2F70304 "2024-01-23T18:09:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![BlahBlah314](https://avatars.discourse-cdn.com/v4/letter/b/71c47a/32.png) [@BlahBlah314](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBlahBlah314)%5C
**Post date:** [January 23, 2024, 6:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fendpoint-issue-with-gptq%2F70304%2F1 "2024-01-23T18:09:12Z")

</div>

Hi there ! I need help for this one.

I converted a finetuned model to a GPTQ model. It works well. Now I want to deploy it with an inference endpoint, so I did, I created an endpoint with a custom handler. My handler works well, I tested it locally, the inference is really quick, so everything seems perfect. Problem is, the endpoint doesn’t work as well as my inference locally when I test the handler.  
When I want to test my endpoint in the Overview, it returns a 504 error. When I test it with the API, it generate the correct ouput, but it takes 1min30 + to generate it (although it took 8 sec when I tested the handler locally)

I don’t really understand what’s wrong. Since then, all of my endpoints worked very well, but here, with the GPTQ model, it doesn’t work properly.

Do you have a clue about what’s going on and how to solve it ?