# Hosting a HF Space for Ultra-Large Language Models

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosting-a-hf-space-for-ultra-large-language-models%2F11661%3E%5C
**Category:** Spaces\
**Created:** [November 13, 2021, 2:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosting-a-hf-space-for-ultra-large-language-models%2F11661 "2021-11-13T14:17:11Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![kmeng01](https://avatars.discourse-cdn.com/v4/letter/k/e99b99/32.png) [@kmeng01](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fkmeng01)%5C
**Post date:** [November 13, 2021, 2:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosting-a-hf-space-for-ultra-large-language-models%2F11661%2F1 "2021-11-13T14:17:11Z")

</div>

Hi all!

My collaborators and I would like to host a research project demo on spaces. The challenge is that we operate on ultra-large language models such as GPT-2 XL (requires ~6GB VRAM) and GPT-J-6B (~24GB VRAM). Our code itself does not use much VRAM outside of the loading the model (it basically makes some user-specified changes to model and lets users generate text using the new model).

It seems like we can fit GPT-2 XL into our 16 GB allowance for T4s, but what about GPT-J-6B? How is this even hosted at [EleutherAI/gpt-j-6B · Hugging Face](https://huggingface.co/EleutherAI/gpt-j-6B)?

Thanks for your insight 🙂

---

<div class="post-metadata">

**Author:** ![julien-c](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/julien-c/32/41937_2.png) [@julien-c](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjulien-c)%5C
**Post date:** [November 15, 2021, 9:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosting-a-hf-space-for-ultra-large-language-models%2F11661%2F2 "2021-11-15T09:39:44Z")

</div>

Discussed over email but let’s also try to paste relevant discussion items here in the future so that interesting discussions can be publicly available 🙏

---

<div class="post-metadata">

**Author:** ![NoToken](https://avatars.discourse-cdn.com/v4/letter/n/97f17d/32.png) [@NoToken](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FNoToken)%5C
**Post date:** [December 15, 2021, 8:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fhosting-a-hf-space-for-ultra-large-language-models%2F11661%2F3 "2021-12-15T08:28:07Z")

</div>

I would for example very much love to hear any tips related to loading GPT-J in different contexts (Colab GPU, TPU VM v3-8, Pytorch, Flax et cetera) and training / finetuning possibilities. Is there any open discussion with the latest updates from HuggingFace somewhere? There are topics and issues spread all over, some outdated, some contradicting.