# ValueError: Unsupported model type mllama

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593%3E%5C
**Category:** Models\
**Created:** [October 23, 2024, 1:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593 "2024-10-23T13:18:51Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![giridhar0813](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@giridhar0813](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgiridhar0813)%5C
**Post date:** [October 23, 2024, 1:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593%2F1 "2024-10-23T13:18:51Z")

</div>

I am looking to deploy “meta-llama/Llama-3.2-11B-Vision-Instruct” to AWS SageMaker but getting below error while deploying.

```auto
Traceback (most recent call last):
  File "/opt/conda/bin/text-generation-server", line 8, in <module>
    sys.exit(app())
  File "/opt/conda/lib/python3.10/site-packages/text_generation_server/cli.py", line 118, in serve
    server.serve(
  File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 297, in serve
    asyncio.run(
  File "/opt/conda/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/opt/conda/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/opt/conda/lib/python3.10/site-packages/text_generation_server/server.py", line 231, in serve_inner
    model = get_model(
  File "/opt/conda/lib/python3.10/site-packages/text_generation_server/models/ __init__.py", line 1117, in get_model
    raise ValueError(f"Unsupported model type {model_type}")

```

### sagemaker config

instance\_type = “ml.g5.4xlarge”  
number\_of\_gpu = 1  
health\_check\_timeout = 1000

### Define Model and Endpoint configuration parameter

config = {  
‘HF\_MODEL\_ID’: “meta-llama/Llama-3.2-11B-Vision-Instruct”,  
‘SM\_NUM\_GPUS’: json.dumps(number\_of\_gpu), # Number of GPU used per replica  
‘MAX\_INPUT\_LENGTH’: json.dumps(6000), # Max length of input text  
‘MAX\_TOTAL\_TOKENS’: json.dumps(8192), # Max length of the generation (including input text)  
}

---

<div class="post-metadata">

**Author:** ![giridhar0813](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@giridhar0813](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgiridhar0813)%5C
**Post date:** [October 23, 2024, 2:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593%2F2 "2024-10-23T14:13:13Z")

</div>

I am using container - [763104351884.dkr.ecr.eu-central-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.3.0-gpu-py310-cu121-ubuntu22.04](http://763104351884.dkr.ecr.eu-central-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.3.0-gpu-py310-cu121-ubuntu22.04).

Should I be using different container ?

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [October 23, 2024, 3:12pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593%2F3 "2024-10-23T15:12:17Z")

</div>

It looks like 2.3.0 is still compatible with Llama 3.2. Do other models work?

> <https://github.com/huggingface/text-generation-inference/blob/v2.3.0/server/text_generation_server/models/ __init__.py>

---

<div class="post-metadata">

**Author:** ![giridhar0813](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@giridhar0813](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fgiridhar0813)%5C
**Post date:** [October 23, 2024, 4:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fvalueerror-unsupported-model-type-mllama%2F113593%2F4 "2024-10-23T16:34:15Z")

</div>

Above issue is resolved after using tgi container - 2.4