# HF Model Deployment Trust Remote Code

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-model-deployment-trust-remote-code%2F83412%3E%5C
**Category:** Amazon SageMaker\
**Created:** [April 24, 2024, 12:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-model-deployment-trust-remote-code%2F83412 "2024-04-24T00:11:09Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![cooper-white](https://avatars.discourse-cdn.com/v4/letter/c/96bed5/32.png) [@cooper-white](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcooper-white)%5C
**Post date:** [April 24, 2024, 12:11am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-model-deployment-trust-remote-code%2F83412%2F1 "2024-04-24T00:11:09Z")

</div>

Hi,

I am deploying gte-large-en-v1.5 l to sagemaker via the `sagemaker.huggingface.HuggingFaceModel.deploy` method.

When requesting inference I get the following error:

```auto
An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
  "code": 400,
  "type": "InternalServerException",
  "message": "Loading /.sagemaker/mms/models/Alibaba-NLP__gte-large-en-v1.5 requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code\u003dTrue` to remove this error."
}

```

This is my Hub env var dictionary for deployment:

```auto
# Hub Model configuration. <https://huggingface.co/models> 
hub = {
    'HF_MODEL_ID':'Alibaba-NLP/gte-large-en-v1.5', # model_id from hf.co/models
    'HF_TASK':'feature-extraction', 
    'MMS_MAX_REQUEST_SIZE': json.dumps(200000000),
    'MMS_MAX_RESPONSE_SIZE': json.dumps(200000000),
    'TRUST_REMOTE_CODE': json.dumps(True) # https://github.com/huggingface/text-generation-inference/issues/493
}

```

I have also tried using `HF_TRUST_REMOTE_CODE`, it has the same result. How can I set this bool to true in my Sagemaker Endpoint Env?

Thanks.

My total deplay code is here:

```auto
# Hub Model configuration. <https://huggingface.co/models> 
hub = {
    'HF_MODEL_ID':'Alibaba-NLP/gte-large-en-v1.5', # model_id from hf.co/models
    'HF_TASK':'feature-extraction', # NLP task you want to use for predictions
    'MMS_MAX_REQUEST_SIZE': json.dumps(200000000),
    'MMS_MAX_RESPONSE_SIZE': json.dumps(200000000),
    'TRUST_REMOTE_CODE': json.dumps(True) # https://github.com/huggingface/text-generation-inference/issues/493
    # 'MAX_INPUT_LENGTH': json.dumps(3000) # https://github.com/huggingface/text-embeddings-inference/issues/141
  # 'SM_NUM_GPUS': '1',
}

from sagemaker.huggingface import HuggingFaceModel

huggingface_model = HuggingFaceModel(
    env=hub, # configuration for loading model from Hub
    role=role, # iam role with permissions to create an Endpoint
    py_version='py310',
    transformers_version="4.37.0", # transformers version used
    pytorch_version="2.1.0", # pytorch version used
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
    initial_instance_count=1,
    instance_type="ml.g4dn.xlarge",
    endpoint_name = ENDPOINT_NAME,
)

```

---

<div class="post-metadata">

**Author:** ![aliiqbal0123](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@aliiqbal0123](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faliiqbal0123)%5C
**Post date:** [July 30, 2024, 6:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhf-model-deployment-trust-remote-code%2F83412%2F2 "2024-07-30T06:26:50Z")

</div>

Hey, did you get any solution? I am also facing the same situation