# Requirements for Hosting LLM via Inference Endpoints
**URL:**
**Author:**  [@cbolles](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcbolles)%5C
**Post date:** [June 13, 2025, 2:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frequirements-for-hosting-llm-via-inference-endpoints%2F159160%2F1 "2025-06-13T14:18:51Z")
# Overview
I’m trying to deploy a fine-tuned LLM via Hugging Face Inference Endpoints. However following the instructions [here](https://huggingface.co/docs/inference-endpoints/guides/create_endpoint) when I select my model I get the message
```auto
Warning: deploying this model will probably fail because no
"handler.py"
file was found in the repository. Try selecting a different model or creating a custom handler.
```
# Question
What do I need to include in the model repo for Hugging Face to recognize the model as one it can serve itself?
# Details
The model is fine-tuned from Llama 3.1 8B and from what I can tell Hugging Face can detect and run other LLama3.1 varients without a custom handler. For example `arcee-ai/Llama-3.1-SuperNova-Lite` doesn’t give me the same error in the endpoint UI.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [June 13, 2025, 2:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frequirements-for-hosting-llm-via-inference-endpoints%2F159160%2F2 "2025-06-13T14:39:19Z")
Since Inference Endpoint is a pay-as-you-go service, I think it would be safer to consult **Expert Support** …
In any case, there is also the option of writing `handler.py`, but if you already have a model repository that can be used with Transformers, setting the `TRUST_REMOTE_CODE` environment variable should make it work.
> **[Model safety.](https://huggingface.co/docs/text-generation-inference/en/basic_tutorials/safety#tgi-20)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
> **[HuggingFaceM4/idefics2-8b · Dedicated Inference Endpoints for Idefics2-8b](https://huggingface.co/HuggingFaceM4/idefics2-8b/discussions/32)**
>
> Hey team,
> **[🤗 Serve any model with Inference Endpoints + Custom Handlers](https://huggingface.co/blog/alvarobartt/serve-any-model-inference-endpoints-custom)**
>
> A Blog post by Alvaro Bartolome on Hugging Face
> **[Expert Support – Hugging Face](https://huggingface.co/support)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
---
**Author:**  [@meganariley](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmeganariley)%5C
**Post date:** [June 13, 2025, 3:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Frequirements-for-hosting-llm-via-inference-endpoints%2F159160%2F3 "2025-06-13T15:44:59Z")
Hi @cbolles! A custom handler may be required for custom tasks, including custom pre- & post-processing. We have additional details on creating and adding a custom handler to your model to use with Inference Endpoints here: [Create custom Inference Handler](https://huggingface.co/docs/inference-endpoints/guides/custom_handler).
I wanted to mention our Inference Endpoints catalog of ready-to-deploy models that require no additional customization, and deployment is verified by Hugging Face: [Inference Catalog | Inference Endpoints by Hugging Face](https://ui.endpoints.huggingface.co/catalog).
Hope this helps and let us know if you have other questions!