# Inference Endpoint Failure - Error: "Shard 0 failed to start"

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-endpoint-failure-error-shard-0-failed-to-start%2F62205%3E%5C
**Category:** Inference Endpoints on the Hub\
**Created:** [November 14, 2023, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-endpoint-failure-error-shard-0-failed-to-start%2F62205 "2023-11-14T12:56:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![eboraks](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/eboraks/32/19060_2.png) [@eboraks](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Feboraks)%5C
**Post date:** [November 14, 2023, 12:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-endpoint-failure-error-shard-0-failed-to-start%2F62205%2F1 "2023-11-14T12:56:10Z")

</div>

I am trying the Inference Endpoint for TheBloke/Llama-2-70B-Chat-GPTQ and I am getting the following errors.

```auto
2023/11/14 07:47:31 ~ {"timestamp":"2023-11-14T12:47:31.273613Z","level":"ERROR","fields":{"message":"Shard 0 failed to start"},"target":"text_generation_launcher"}

2023/11/14 07:47:31 ~ {"timestamp":"2023-11-14T12:47:31.273635Z","level":"INFO","fields":{"message":"Shutting down shards"},"target":"text_generation_launcher"}

2023/11/14 07:47:31 ~ {"timestamp":"2023-11-14T12:47:31.278347Z","level":"ERROR","fields":{"message":"Shard complete standard error output:\n\n[W socket.cpp:601] [c10d] The client socket has failed to connect to [localhost]:29500 (errno: 99 - Cannot assign requested address).\nYou are using a model of type llama to instantiate a model of type . This is not supported for all configurations of models and can yield errors.\nTraceback (most recent call last):\n\n File

```

Any advice appreciated?

---

<div class="post-metadata">

**Author:** ![echeyde](https://avatars.discourse-cdn.com/v4/letter/e/ba9def/32.png) [@echeyde](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fecheyde)%5C
**Post date:** [November 21, 2024, 4:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-endpoint-failure-error-shard-0-failed-to-start%2F62205%2F2 "2024-11-21T16:13:00Z")

</div>

I’d be curious if you resolved and if so how, thanks.