# Training using FSDP, qLoRa on multinode

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-using-fsdp-qlora-on-multinode%2F137923%3E%5C
**Category:** 🤗Accelerate\
**Created:** [January 29, 2025, 3:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-using-fsdp-qlora-on-multinode%2F137923 "2025-01-29T15:39:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yustas](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/yustas/32/40036_2.png) [@Yustas](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FYustas)%5C
**Post date:** [January 29, 2025, 3:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftraining-using-fsdp-qlora-on-multinode%2F137923%2F1 "2025-01-29T15:39:43Z")

</div>

How to train a model like Llama3 using FSDP, qLoRa on two (or more) nodes if each node has one (or more) GPU?  
Can anyone provide a link to a similar example?  
I am grateful in advance for any help!