Loading Mistral Models in vLLM

#2
by BuiDoan - opened

In vllm, I see that:
load_format: "mistral" will load weights from consolidated safetensors files used by Mistral models.
Therefore, I only need to download the consolidated.safetensors file, right?

cyankiwi org

Yes, that is correct Bùi Đoàn.

Sign up or log in to comment