# Llama 2 support for AutoModelForQuestionAnswering
**URL:**
**Author:**  [@nicholasKluge](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FnicholasKluge)%5C
**Post date:** [December 24, 2023, 5:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F1 "2023-12-24T17:01:13Z")
Hello everyone!
I was wondering if there is any way to use the Llama 2 type models with the `AutoModelForQuestionAnswering`? Currently, as far as I am aware, Llama models cannot be use in a `AutoModelForQuestionAnswering` pipeline. Is there any prediction for their integration, or no?
If not, any one recommends a work around?
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 26, 2023, 8:28am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F2 "2023-12-26T08:28:55Z")
Hi,
There’s no `Llama2ForQuestionAnswering` model yet in the Transformers library. The reason for that is because Llama-2 is a decoder-only Transformer, mainly useful for generative tasks. `xxxForQuestionAnswering` models are trained to extract the appropriate answer from a given piece of context, hence one typically uses encoder-only Transformers like BERT, RoBERTa, etc. for these. They are also a lot smaller than models like Llama-2.
See this guide for more info: [notebooks/examples/question\_answering.ipynb at main · huggingface/notebooks · GitHub](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb).
---
**Author:**  [@nicholasKluge](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FnicholasKluge)%5C
**Post date:** [December 26, 2023, 5:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F3 "2023-12-26T17:33:51Z")
Thanks for the answer, but I don’t think that is the case (only encoder transformers are supported for this pipeline, like BERT-style models).
According to the documentation, all these models can be used by the question-and-answering pipeline:
[ALBERT](https://huggingface.co/docs/transformers/model_doc/albert), [BART](https://huggingface.co/docs/transformers/model_doc/bart), [BERT](https://huggingface.co/docs/transformers/model_doc/bert), [BigBird](https://huggingface.co/docs/transformers/model_doc/big_bird), [BigBird-Pegasus](https://huggingface.co/docs/transformers/model_doc/bigbird_pegasus), [BLOOM](https://huggingface.co/docs/transformers/model_doc/bloom), [CamemBERT](https://huggingface.co/docs/transformers/model_doc/camembert), [CANINE](https://huggingface.co/docs/transformers/model_doc/canine), [ConvBERT](https://huggingface.co/docs/transformers/model_doc/convbert), [Data2VecText](https://huggingface.co/docs/transformers/model_doc/data2vec-text), [DeBERTa](https://huggingface.co/docs/transformers/model_doc/deberta), [DeBERTa-v2](https://huggingface.co/docs/transformers/model_doc/deberta-v2), [DistilBERT](https://huggingface.co/docs/transformers/model_doc/distilbert), [ELECTRA](https://huggingface.co/docs/transformers/model_doc/electra), [ERNIE](https://huggingface.co/docs/transformers/model_doc/ernie), [ErnieM](https://huggingface.co/docs/transformers/model_doc/ernie_m), [Falcon](https://huggingface.co/docs/transformers/model_doc/falcon), [FlauBERT](https://huggingface.co/docs/transformers/model_doc/flaubert), [FNet](https://huggingface.co/docs/transformers/model_doc/fnet), [Funnel Transformer](https://huggingface.co/docs/transformers/model_doc/funnel), [OpenAI GPT-2](https://huggingface.co/docs/transformers/model_doc/gpt2), [GPT Neo](https://huggingface.co/docs/transformers/model_doc/gpt_neo), [GPT NeoX](https://huggingface.co/docs/transformers/model_doc/gpt_neox), [GPT-J](https://huggingface.co/docs/transformers/model_doc/gptj), [I-BERT](https://huggingface.co/docs/transformers/model_doc/ibert), [LayoutLMv2](https://huggingface.co/docs/transformers/model_doc/layoutlmv2), [LayoutLMv3](https://huggingface.co/docs/transformers/model_doc/layoutlmv3), [LED](https://huggingface.co/docs/transformers/model_doc/led), [LiLT](https://huggingface.co/docs/transformers/model_doc/lilt), [Longformer](https://huggingface.co/docs/transformers/model_doc/longformer), [LUKE](https://huggingface.co/docs/transformers/model_doc/luke), [LXMERT](https://huggingface.co/docs/transformers/model_doc/lxmert), [MarkupLM](https://huggingface.co/docs/transformers/model_doc/markuplm), [mBART](https://huggingface.co/docs/transformers/model_doc/mbart), [MEGA](https://huggingface.co/docs/transformers/model_doc/mega), [Megatron-BERT](https://huggingface.co/docs/transformers/model_doc/megatron-bert), [MobileBERT](https://huggingface.co/docs/transformers/model_doc/mobilebert), [MPNet](https://huggingface.co/docs/transformers/model_doc/mpnet), [MPT](https://huggingface.co/docs/transformers/model_doc/mpt), [MRA](https://huggingface.co/docs/transformers/model_doc/mra), [MT5](https://huggingface.co/docs/transformers/model_doc/mt5), [MVP](https://huggingface.co/docs/transformers/model_doc/mvp), [Nezha](https://huggingface.co/docs/transformers/model_doc/nezha), [Nyströmformer](https://huggingface.co/docs/transformers/model_doc/nystromformer), [OPT](https://huggingface.co/docs/transformers/model_doc/opt), [QDQBert](https://huggingface.co/docs/transformers/model_doc/qdqbert), [Reformer](https://huggingface.co/docs/transformers/model_doc/reformer), [RemBERT](https://huggingface.co/docs/transformers/model_doc/rembert), [RoBERTa](https://huggingface.co/docs/transformers/model_doc/roberta), [RoBERTa-PreLayerNorm](https://huggingface.co/docs/transformers/model_doc/roberta-prelayernorm), [RoCBert](https://huggingface.co/docs/transformers/model_doc/roc_bert), [RoFormer](https://huggingface.co/docs/transformers/model_doc/roformer), [Splinter](https://huggingface.co/docs/transformers/model_doc/splinter), [SqueezeBERT](https://huggingface.co/docs/transformers/model_doc/squeezebert), [T5](https://huggingface.co/docs/transformers/model_doc/t5), [UMT5](https://huggingface.co/docs/transformers/model_doc/umt5), [XLM](https://huggingface.co/docs/transformers/model_doc/xlm), [XLM-RoBERTa](https://huggingface.co/docs/transformers/model_doc/xlm-roberta), [XLM-RoBERTa-XL](https://huggingface.co/docs/transformers/model_doc/xlm-roberta-xl), [XLNet](https://huggingface.co/docs/transformers/model_doc/xlnet), [X-MOD](https://huggingface.co/docs/transformers/model_doc/xmod), [YOSO](https://huggingface.co/docs/transformers/model_doc/yoso)
As you can see, many are deconder-only transformers (GPT-2, GPT-J, etc).
Hence, my question is, **is there any prediction for Llama 2 models to be inclueded in this list?**
---
**Author:**  [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [December 26, 2023, 6:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F4 "2023-12-26T18:23:10Z")
Yes it’s true that you can also do that using decoder-only Transformers (I just don’t recommend it).
For LLaMa, could you open an issue on the Transformers library? Then I’ll assign it as “good first issue” since it’s a nice opportunity for a first contribution by someone. The implementation can be copied from `GptjForQuestionAnswering`.
---
**Author:**  [@nicholasKluge](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FnicholasKluge)%5C
**Post date:** [December 27, 2023, 5:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F5 "2023-12-27T17:53:10Z")
Thank you for the suggestion. I will open this feature request.
The motivation behind it is that the `AutoModelForQuestionAnswering`, regardless of whether encoder-only transformers are superior in closed Q&A settings, facilitates the evaluation of generative transformers in benchmarks like Squad.
---
**Author:**  [@nicholasKluge](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FnicholasKluge)%5C
**Post date:** [December 27, 2023, 6:59pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fllama-2-support-for-automodelforquestionanswering%2F66906%2F6 "2023-12-27T18:59:47Z")
A feature request is [open](https://github.com/huggingface/transformers/issues/28265).