# Estimate training compute for 150B LLM

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Festimate-training-compute-for-150b-llm%2F45079%3E%5C
**Category:** DeepSpeed\
**Created:** [June 30, 2023, 3:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Festimate-training-compute-for-150b-llm%2F45079 "2023-06-30T03:31:38Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ngoquanghuy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ngoquanghuy/32/17235_2.png) [@ngoquanghuy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fngoquanghuy)%5C
**Post date:** [June 30, 2023, 3:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Festimate-training-compute-for-150b-llm%2F45079%2F1 "2023-06-30T03:31:38Z")

</div>

Hi everyone,

I’m trying to estimate GPU memory usage needed to train an 150B language model.  
This seems complex because of various factors.  
What should I notice? Is there any formula for this?