# Multiple texts as inputs to Transformers models
**URL:**
**Author:**  [@Zack](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZack)%5C
**Post date:** [November 25, 2020, 11:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F1 "2020-11-25T23:49:54Z")
I would like to use multiple texts as inputs to a model, let’s say I have a dataset with 10 columns each column is a text (sentence or two), how can I fit all these inputs to the model and do a classification for example ?
I can see it’s possible to just concatenate all texts in one, but seems that for me, I need a very large data to be apple to achieve good accuracy.
Maybe using multiple models (BERT) in parallel, taking last hidden state, concatenate them and classify ? But the problem is that there’s so many values order of 30 texts.
Any idea how to tackle this ?
---
**Author:**  [@astariul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fastariul)%5C
**Post date:** [November 26, 2020, 7:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F2 "2020-11-26T07:15:01Z")
You should take the same approach as Extractive text summarization :
Concatenate all your sentences, separated with a special token (`CLS` for example), then use the `CLS` token representation to do classification.
From the [Presumm paper](https://arxiv.org/abs/1908.08345)
---
**Author:**  [@Zack](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FZack)%5C
**Post date:** [November 26, 2020, 11:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F3 "2020-11-26T11:47:05Z")
Hi @astariul, thank you for your reply.
I understand what you suggested, the problem is that I don’t have only texts as inputs I have also some floats values, is converting this values to text would be sufficient ?
---
**Author:**  [@astariul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fastariul)%5C
**Post date:** [November 26, 2020, 11:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F4 "2020-11-26T23:33:50Z")
I see…
I never encountered this case myself, but maybe you can directly input the float values in the last classifier ?
Since it’s not text, there is no need for BERT to encode it (?)
---
**Author:**  [@scroobiustrip](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fscroobiustrip)%5C
**Post date:** [December 7, 2020, 10:18pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F5 "2020-12-07T22:18:18Z")
Hi @Zack
I don’t know whether you’ve tried / considered the multimodal toolkit ([blog post](https://medium.com/georgian-impact-blog/how-to-incorporate-tabular-data-with-huggingface-transformers-b70ac45fcfb4), [github](https://colab.research.google.com/github/georgianpartners/Multimodal-Toolkit/blob/master/notebooks/text_w_tabular_classification.ipynb))- takes in tabular data (text, numbers, categorical data) and can use them as inputs to develop models.
Haven’t tried it myself, but looks quite promising.
---
**Author:**  [@Inder1207](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FInder1207)%5C
**Post date:** [July 11, 2023, 3:10pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F6 "2023-07-11T15:10:49Z")
Hello @astariul ,
Thank you for your answer. I am also trying to do something similar but I just had a question though. So, even after concatenating the data will the different input data be provided with different weights and biases in this case? or the whole input text itself will be provided with weights and biases?
---
**Author:**  [@astariul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fastariul)%5C
**Post date:** [July 12, 2023, 12:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F7 "2023-07-12T00:09:42Z")
You can do both.
In the case of BERT for summarization, they just use sentence-specific representation :
`[CLS] sen1 [SEP] [CLS] sen2 [SEP] [CLS] sen3 [SEP]`
Then use each CLS token as a representation of each sentence.
But if you want a general representation for the whole text, you can just train your model with an additional token at the beginning :
`[CLS2] [CLS] sen1 [SEP] [CLS] sen2 [SEP] [CLS] sen3 [SEP]`
Then use `CLS` for sentence representation, `CLS2` for token representation.
---
**Author:**  [@Inder1207](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FInder1207)%5C
**Post date:** [July 25, 2023, 12:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F8 "2023-07-25T12:41:20Z")
Hello @astariul , Thank you for your reply again. I was just curious to know if I can build a model using huggingface transformers which can give me multiple outputs?
---
**Author:**  [@astariul](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fastariul)%5C
**Post date:** [July 25, 2023, 1:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F9 "2023-07-25T13:52:11Z")
Yes you can.
Regular models have a single output (usually a LM head or a classification head) on top of the Transformer stack, but you can just add several different heads on top of the Transformer stack to get multiple outputs.
---
**Author:**  [@samme80](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsamme80)%5C
**Post date:** [September 13, 2024, 11:50am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fmultiple-texts-as-inputs-to-transformers-models%2F2222%2F10 "2024-09-13T11:50:32Z")
@astariul , Hi I am also working on similar project. My use case has only text with multiple inputs and multiple outputs. If you know resources around it please help. Currently using hugging face’s trainer giving me better accuracy