# Inference service for large models, such as Vicuna 13b

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-service-for-large-models-such-as-vicuna-13b%2F38810%3E%5C
**Category:** Beginners\
**Created:** [May 5, 2023, 3:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-service-for-large-models-such-as-vicuna-13b%2F38810 "2023-05-05T15:46:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rlancemartin](https://avatars.discourse-cdn.com/v4/letter/r/8edcca/32.png) [@rlancemartin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frlancemartin)%5C
**Post date:** [May 5, 2023, 3:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Finference-service-for-large-models-such-as-vicuna-13b%2F38810%2F1 "2023-05-05T15:46:43Z")

</div>

Hi All!

I’d like API access to some of the new SOTA models, like Vicuna 13b.

I found [jeffwan/vicuna-13b](https://huggingface.co/jeffwan/vicuna-13b) and I see:

`Use this model with the Inference API`

I copy over the code:

```auto
import requests

API_URL = "/static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fjeffwan%2Fvicuna-13b"
headers = {"Authorization": "Bearer_xxx"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()

output = query({"inputs": "Can you please let us know more details about your ",})

```

I test with my API key, but see:

```auto
{'error': 'The model jeffwan/vicuna-13b is too large to be loaded automatically (26GB > 10GB). For commercial use please use PRO spaces (https://huggingface.co/spaces) or Inference Endpoints (https://huggingface.co/inference-endpoints).'}

```

I looked at deploying an inference endpoint (e.g., on T4 or A100).

It it is [far too expensive](https://ui.endpoints.huggingface.co/new?repository=jeffwan/vicuna-13b) for an individual developer: ~$5k / month.

Does HuggingFace host models such as `vicuna-13b`?

I was confused by the language `Use this model with the Inference API` but then `The model jeffwan/vicuna-13b is too large to be loaded automatically`, indicating that it does not work with the inference API.

Thank!