# Tensor parallel in Pytorch 2.3

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallel-in-pytorch-2-3%2F84909%3E%5C
**Category:** 🤗Transformers\
**Created:** [May 2, 2024, 11:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallel-in-pytorch-2-3%2F84909 "2024-05-02T23:17:19Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![proj-persona](https://avatars.discourse-cdn.com/v4/letter/p/f6c823/32.png) [@proj-persona](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fproj-persona)%5C
**Post date:** [May 2, 2024, 11:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftensor-parallel-in-pytorch-2-3%2F84909%2F1 "2024-05-02T23:17:19Z")

</div>

Tensor parallelism has been supported by Pytorch 2.3. Just wonder when Transformers could also officially support this feature, which would be very helpful to handle very large language models.