# Loading inference.py separately from model.tar.gz
**URL:**
**Author:**  [@liammc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fliammc)%5C
**Post date:** [March 22, 2023, 9:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-inference-py-separately-from-model-tar-gz%2F34373%2F1 "2023-03-22T09:23:47Z")
Hi,
I’m deploying some inference endpoints on Sagemaker using the HuggingFace Inference Toolkit, and am overriding some of the default methods (model\_fn and predict\_fn) as described here:
> **[Deploy models to Amazon SageMaker](https://huggingface.co/docs/sagemaker/inference#user-defined-code-and-modules)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.
This works, but the problem is that when I want to test changes to my `inference.py`, it is very slow and cumbersome to create and upload a new `model.tar.gz` file that includes the new `inference.py` script.
Is it possible to provide the `inference.py` script separately from the compressed model archive?
---
**Author:**  [@philschmid](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphilschmid)%5C
**Post date:** [March 22, 2023, 10:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-inference-py-separately-from-model-tar-gz%2F34373%2F2 "2023-03-22T10:47:26Z")
No, there is sadly not way except that you can test your inference.py locally or load the model in the model\_fn
---
**Author:**  [@augustindal](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Faugustindal)%5C
**Post date:** [March 22, 2023, 11:04am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-inference-py-separately-from-model-tar-gz%2F34373%2F3 "2023-03-22T11:04:20Z")
It is possible to give only the inference.py script if you have your model.tar.gz in s3. Not sure if it works with hub.
When you create the HuggingFaceModel() object, give it source dir (local folder where inference.py script is), entry point (inference.py) and model\_data (s3 url).
Then next time you do HuggingFaceModel.deploy() it will use the inference script from your local folder and the model from s3.
---
**Author:**  [@philschmid](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fphilschmid)%5C
**Post date:** [March 22, 2023, 12:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-inference-py-separately-from-model-tar-gz%2F34373%2F4 "2023-03-22T12:39:13Z")
> [@augustindal](#):
>
> It is possible to give only the inference.py script if you have your model.tar.gz in s3. Not sure if it works with hub.
Thats true but behind the scenes a new `model.tar.gz` is built and used.
---
**Author:**  [@Devj2013](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FDevj2013)%5C
**Post date:** [June 5, 2023, 8:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Floading-inference-py-separately-from-model-tar-gz%2F34373%2F5 "2023-06-05T08:47:51Z")
> [@liammc](#):
>
> it possible to provide the `inference.py` script separately from the compressed model archive?
Where should the inference.py file be inside the model tar ball? Should it be inside the subdirectory “code” or should it be immediately inside the tar ball as a file?