# Finetuning cost estimation

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-cost-estimation%2F57016%3E%5C
**Category:** Languages at Hugging Face\
**Created:** [October 1, 2023, 5:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-cost-estimation%2F57016 "2023-10-01T17:39:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![alielfilali01](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alielfilali01/32/59086_2.png) [@alielfilali01](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falielfilali01)%5C
**Post date:** [October 1, 2023, 5:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-cost-estimation%2F57016%2F1 "2023-10-01T17:39:32Z")

</div>

Hi everyone, i’am starting a research project for arabic and the idea is to finetune multiple SOTA models then compare them , the thing is we are short in terms of budget so i would like to know if there is a way to estimate the cost giving the model size and the dataset size (assuming it will be done on an A100 80GB)

i.e : is there a magical formula to estimate the cost ?

---

<div class="post-metadata">

**Author:** ![MattiLinnanvuori](https://avatars.discourse-cdn.com/v4/letter/m/ec9cab/32.png) [@MattiLinnanvuori](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMattiLinnanvuori)%5C
**Post date:** [October 2, 2023, 6:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-cost-estimation%2F57016%2F2 "2023-10-02T18:08:01Z")

</div>

Hugging Face has a model memory usage calculator that I guess can be used in calculating the number of A100 80Gs needed. [https://huggingface.co/spaces/hf-accelerate/model-memory-usage](https://huggingface.co/spaces/hf-accelerate/model-memory-usage)

---

<div class="post-metadata">

**Author:** ![alielfilali01](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/alielfilali01/32/59086_2.png) [@alielfilali01](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falielfilali01)%5C
**Post date:** [October 2, 2023, 8:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffinetuning-cost-estimation%2F57016%2F3 "2023-10-02T20:42:17Z")

</div>

Not very useful in my case ! I’m more interested to know that given a model size let’s 560M params (8% of 7B since we are using LoRA) and a dataset size of let’s say 2B tokens, how much would it take to fine-tune that model for 1 epoch on a single A100 80gb ? I don’t even know if it’s possible to do such calculation but the HF AutoTrain ([https://ui.autotrain.huggingface.co](https://ui.autotrain.huggingface.co)) gives a cost estimation before launching the training so i assumed it is possible after all 🙃

@merve can you guide to who from the team can help with that ?