# Tensor parallelism for customized model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallelism-for-customized-model%2F104867%3E%5C
**Category:** 🤗Accelerate\
**Created:** [September 2, 2024, 12:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallelism-for-customized-model%2F104867 "2024-09-02T00:48:18Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![zzxslp](https://avatars.discourse-cdn.com/v4/letter/z/e47774/32.png) [@zzxslp](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fzzxslp)%5C
**Post date:** [September 2, 2024, 12:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallelism-for-customized-model%2F104867%2F1 "2024-09-02T00:48:18Z")

</div>

Hi! If I want to do **model inference with a customized large model** (can’t fit into a single GPU), how do I enable tensor parallel to shard the model across multiple GPUs?

For my specific case, I have a multimodal-LLM with a ViT, projector, and an LLM. I am not sure what is the best library to start with. Should I use deepspeed, accelerate or something else?