# Model parallel with deepspeed integration

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-parallel-with-deepspeed-integration%2F9980%3E%5C
**Category:** Beginners\
**Created:** [September 14, 2021, 12:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-parallel-with-deepspeed-integration%2F9980 "2021-09-14T12:39:23Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiami](https://avatars.discourse-cdn.com/v4/letter/x/dc4da7/32.png) [@xiami](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fxiami)%5C
**Post date:** [September 14, 2021, 12:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmodel-parallel-with-deepspeed-integration%2F9980%2F1 "2021-09-14T12:39:23Z")

</div>

If I use the deepspeed integration with multi-GPU, am I doing model parallel or just data parallel?  
I know the deepspeed library may be able of model parallel but I am still learning it. Is there a way to do model parallel in transformers?

And is there a doc or example for the parallelize method in T5(or any other model)?

Thank you for your time!