# Unable to load nvidia canary and parakeet models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-nvidia-canary-and-parakeet-models%2F85203%3E%5C
**Category:** Inference Endpoints on the Hub\
**Created:** [May 5, 2024, 1:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-nvidia-canary-and-parakeet-models%2F85203 "2024-05-05T01:58:47Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![bamitsmanas](https://avatars.discourse-cdn.com/v4/letter/b/6f9a4e/32.png) [@bamitsmanas](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbamitsmanas)%5C
**Post date:** [May 5, 2024, 1:58am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Funable-to-load-nvidia-canary-and-parakeet-models%2F85203%2F1 "2024-05-05T01:58:47Z")

</div>

I’ve been trying for the last 24 hours and the models neither load on the normal huggingface website nor when I try to deploy them as a specific inference endpoint. Could there be an issue with the model config itself?

Here are the logs from when I tried to deploy them on the dedicated inference api:

2024/05/04 22:02:02 ~ /usr/local/lib/python3.10/dist-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils.\_pytree.\_register\_pytree\_node is deprecated. Please use torch.utils.\_pytree.register\_pytree\_node instead.  
2024/05/04 22:02:02 ~ torch.utils.\_pytree.\_register\_pytree\_node(  
2024/05/04 22:02:02 ~ 2024-05-04 16:32:02,510 | INFO | Initializing model from directory:/repository  
2024/05/04 22:02:02 ~ 2024-05-04 16:32:02,510 | INFO | No custom pipeline found at /repository/handler.py  
2024/05/04 22:02:02 ~ 2024-05-04 16:32:02,510 | INFO | Using device CPU  
2024/05/04 22:02:02 ~ Traceback (most recent call last):  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/starlette/routing.py”, line 732, in lifespan  
2024/05/04 22:02:02 ~ async with self.lifespan\_context(app) as maybe\_state:  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/starlette/routing.py”, line 608, in **aenter**  
2024/05/04 22:02:02 ~ await self.\_router.startup()  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/starlette/routing.py”, line 709, in startup  
2024/05/04 22:02:02 ~ await handler()  
2024/05/04 22:02:02 ~ File “/app/webservice\_starlette.py”, line 60, in some\_startup\_task  
2024/05/04 22:02:02 ~ inference\_handler = get\_inference\_handler\_either\_custom\_or\_default\_handler(HF\_MODEL\_DIR, task=HF\_TASK)  
2024/05/04 22:02:02 ~ File “/app/huggingface\_inference\_toolkit/handler.py”, line 54, in get\_inference\_handler\_either\_custom\_or\_default\_handler  
2024/05/04 22:02:02 ~ return HuggingFaceHandler(model\_dir=model\_dir, task=task)  
2024/05/04 22:02:02 ~ File “/app/huggingface\_inference\_toolkit/handler.py”, line 18, in **init**  
2024/05/04 22:02:02 ~ self.pipeline = get\_pipeline(  
2024/05/04 22:02:02 ~ File “/app/huggingface\_inference\_toolkit/utils.py”, line 276, in get\_pipeline  
2024/05/04 22:02:02 ~ hf\_pipeline = pipeline(  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/transformers/pipelines/ **init**.py”, line 815, in pipeline  
2024/05/04 22:02:02 ~ config = AutoConfig.from\_pretrained(  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration\_auto.py”, line 1111, in from\_pretrained  
2024/05/04 22:02:02 ~ config\_dict, unused\_kwargs = PretrainedConfig.get\_config\_dict(pretrained\_model\_name\_or\_path, \*\*kwargs)  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/transformers/configuration\_utils.py”, line 633, in get\_config\_dict  
2024/05/04 22:02:02 ~ config\_dict, kwargs = cls.\_get\_config\_dict(pretrained\_model\_name\_or\_path, \*\*kwargs)  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/transformers/configuration\_utils.py”, line 688, in \_get\_config\_dict  
2024/05/04 22:02:02 ~ resolved\_config\_file = cached\_file(  
2024/05/04 22:02:02 ~ File “/usr/local/lib/python3.10/dist-packages/transformers/utils/hub.py”, line 369, in cached\_file  
2024/05/04 22:02:02 ~ raise EnvironmentError(  
2024/05/04 22:02:02 ~ OSError: /repository does not appear to have a file named config.json. Checkout ‘[https://huggingface.co//repository/None](https://huggingface.co//repository/None)’ for available files.

- 2024-05-04T16:32:02.512+00:00  
2024/05/04 22:02:02 ~ Application startup failed. Exiting.