# How to deploy Sagemaker Multi-model Endpoints on GPU?

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-deploy-sagemaker-multi-model-endpoints-on-gpu%2F65833%3E%5C
**Category:** Amazon SageMaker\
**Created:** [December 14, 2023, 1:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-deploy-sagemaker-multi-model-endpoints-on-gpu%2F65833 "2023-12-14T13:23:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![CarlosAndrea](https://avatars.discourse-cdn.com/v4/letter/c/71e660/32.png) [@CarlosAndrea](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FCarlosAndrea)%5C
**Post date:** [December 14, 2023, 1:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhow-to-deploy-sagemaker-multi-model-endpoints-on-gpu%2F65833%2F1 "2023-12-14T13:23:53Z")

</div>

Hello,

I’m trying to deploy Multi-model Endpoints by following : [AWS-SageMaker-Examples/03\_MultiModelEndpointWithHuggingFace/huggingface-sagemaker-multi-model-endpoint.ipynb at main · vinayak-shanawad/AWS-SageMaker-Examples · GitHub](https://github.com/vinayak-shanawad/AWS-SageMaker-Examples/blob/main/03_MultiModelEndpointWithHuggingFace/huggingface-sagemaker-multi-model-endpoint.ipynb)

I can deploy each model separately either on CPU or GPU.  
I can deploy both model on a multi-model Endpoints on CPU, but as soon as I try to deploy them on GPU I get the following error :

ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Your Ecr Image [763104351884.dkr.ecr.eu-central-1.amazonaws.com/huggingface-pytorch-inference:1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04](http://763104351884.dkr.ecr.eu-central-1.amazonaws.com/huggingface-pytorch-inference:1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04) does not contain required com.amazonaws.sagemaker.capabilities.multi-models=true Docker label(s)

Any help would be appreciated !