# Sentence similarity models on Sagemaker
**URL:**
**Author:**  [@pavel-nesterov](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpavel-nesterov)%5C
**Post date:** [September 16, 2021, 1:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F1 "2021-09-16T13:13:55Z")
Hello,
I am wondering if the “sentence-similarity” pipelines can be used on Sagemaker from Hub with the same ease as popular piplines like “question-answering”?
I was trying to start an endpoint with sentence-similarity, but it gave me this error:
```auto
2021-09-16 13:13:49,252 [INFO] W-sentence-transformers__pa-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: "Unknown task sentence-similarity, available tasks are ['feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification', 'translation_XX_to_YY']" : 400
```
This is how it was deployed inside Sagemaker Studio:
```python
from sagemaker.huggingface import HuggingFaceModel
import sagemaker
role = sagemaker.get_execution_role()
# Hub Model configuration. https://huggingface.co/models
hub = {
'HF_MODEL_ID':'sentence-transformers/paraphrase-xlm-r-multilingual-v1',
'HF_TASK':'sentence-similarity' # NLP task you want to use for predictions
}
# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
env=hub,
role=role, # iam role with permissions to create an Endpoint
transformers_version="4.6", # transformers version used
pytorch_version="1.7", # pytorch version used
py_version="py36", # python version of the DLC
)
# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
initial_instance_count=1,
instance_type="ml.t2.medium"
)
```
---
**Author:**  [@philschmid](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphilschmid)%5C
**Post date:** [September 16, 2021, 1:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F2 "2021-09-16T13:53:38Z")
Hello @pavel-nesterov,
Currently, the SageMaker Inference Toolkit only supports the NLP-based pipelines from the `transformers` library for zero-code deployments.
But you can easily create your own `inference.py` which contains the code you would need for that.
Here is Documentation on that: [Deploy models to Amazon SageMaker](https://huggingface.co/docs/sagemaker/inference#user-defined-codemodules)
Here the code for the `sentence-simliarity` pipeline: [huggingface\_hub/sentence\_similarity.py at main · huggingface/huggingface\_hub · GitHub](https://github.com/huggingface/huggingface_hub/blob/main/api-inference-community/docker_images/sentence_transformers/app/pipelines/sentence_similarity.py)
Thank you for the request I will add it as potential feature to the roadmap.
---
**Author:**  [@pavel-nesterov](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpavel-nesterov)%5C
**Post date:** [September 16, 2021, 2:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F3 "2021-09-16T14:26:15Z")
Thank you, Philipp.
I will try it as “next easiest step” in the search of the solution.
The approach I am trying to explore now is “Are there any of the 186 Sentence Similarity models can be deployed on sagemaker with no-code approach”. I checked all of them and [I found some](https://huggingface.co/AIDA-UPM/mstsb-paraphrase-multilingual-mpnet-base-v2)! (scerenshot is below)
The only thing is that it doesn’t have “sentence similarity” pipeline, but I think I would try “token-classification” one. I am writing it because someone later may come to the same idea and I may save some time for that explorer.
---
**Author:**  [@pavel-nesterov](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpavel-nesterov)%5C
**Post date:** [September 17, 2021, 10:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F4 "2021-09-17T10:44:40Z")
My 1st iteration [failed](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosted-inference-api-errror-on-aida-upm-mstsb-paraphrase-multilingual-mpnet-base-v2%2F10075)%2C but I hope it will help to improve the Model Hub. Therefore I am getting deeper into your suggestion, Philipp @philschmid
I will post here how it goes (it may help someone like me, who tries to find answers in this forum).
---
**Author:**  [@pavel-nesterov](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpavel-nesterov)%5C
**Post date:** [September 18, 2021, 7:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F5 "2021-09-18T07:56:16Z")
Hi Philipp! @philschmid
I read the documentation twice and concluded, that I need to get deeper understanding of how the transformers work. The best way I found (as of know) - is to go through the [HF Course](https://huggingface.co/course/) .
Even though I did many training jobs on Sagemaker with vanilla Pytorch, I feel like I need to
1. get a deeper understanding of “pipelines”, “tokenisers” and “post-processing”.
2. In my project I will also need to fine-tune a pre-trained model (because I am builing a product for not a widely-spoken language - slovenian).
3. The link you shared leads to “Advanced features” of the HF-Sagemeker" documentation (read twice).
Considering the three points above, I am timeboxing a week to go through the HF course to see if it will me to approach overriding default methods of HF inference toolkit.
Not sure if there is a better way to approach the problem of making sentence-similarity endpoint on sagemaker. My candidate was to train bag-of-words + cosine similarity from scratch in Pytorch (and then deploy the endpoint), but I feel I will be able to benefit from learning the transformers’ basics.
Thank you for helping me to find the direction for the solution!
---
**Author:**  [@philschmid](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphilschmid)%5C
**Post date:** [September 20, 2021, 6:50am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F6 "2021-09-20T06:50:51Z")
Sounds good!
I found two solvenian model on the hub, which you can give a try to fine-tune and compare with your `bag-of-words` method:
- [EMBEDDIA/sloberta · Hugging Face](https://huggingface.co/EMBEDDIA/sloberta)
- [EMBEDDIA/crosloengual-bert · Hugging Face](https://huggingface.co/EMBEDDIA/crosloengual-bert)
---
**Author:**  [@316usman](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2F316usman)%5C
**Post date:** [January 12, 2024, 12:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fsentence-similarity-models-on-sagemaker%2F10045%2F7 "2024-01-12T12:01:26Z")
@philschmid I tried the custom method using inference.py, but I am faing the following error
## "You need to define one of the following ['audio-classification', 'automatic-speech-recognition', 'feature-extraction', 'text-classification', 'token-classification', 'question-answering', 'table-question-answering', 'fill-mask', 'summarization', 'translation', 'text2text-generation', 'text-generation', 'zero-shot-classification', 'conversational', 'image-classification', 'image-segmentation', 'object-detection'] as env 'HF\_TASK’
Will setting an environment variable solve this error?