# From Pandas Dataframe to Huggingface Dataset
**URL:**
**Author:**  [@marlon89](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmarlon89)%5C
**Post date:** [August 18, 2021, 8:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F1 "2021-08-18T08:56:58Z")
Hello everyone,
I am doing a tutorial on how to finetune pretrained Sentiment Analysis Classifier and all the finetuning part is based on a HuggingFace Dataset. Is there a way to transform a pandas Dataframe to a HuggingFace Dataset? Would help me alot with my data preprocessing…
---
**Author:**  [@ehalit](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fehalit)%5C
**Post date:** [August 18, 2021, 10:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F2 "2021-08-18T10:35:56Z")
You can have a look at here: [link](https://huggingface.co/docs/datasets/loading_datasets.html#from-a-pandas-dataframe)
---
**Author:**  [@marlon89](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmarlon89)%5C
**Post date:** [August 18, 2021, 12:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F3 "2021-08-18T12:29:19Z")
Thanks for your help! Now it works 🙂
---
**Author:**  [@akomma](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fakomma)%5C
**Post date:** [February 23, 2022, 6:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F4 "2022-02-23T06:57:40Z")
is there a way to load this into the `train` split and another dataframe in memory into the `validation` split
None of the following options seem to do the trick:
```auto
dataset = Dataset.from_pandas(df)
dataset = Dataset.from_pandas(df, split='train')
dataset = Dataset.from_pandas(df, split=NamedSplit('train'))
dataset = Dataset.from_pandas(df, split=datasets.Split.TRAIN)
print(dataset)
```
The best I could come up that worked was (not sure if there is a easier/right way):
```auto
import pandas as pd
import datasets
from datasets import Dataset, DatasetDict
tdf = pd.DataFrame({"a": [1, 2, 3], "b": ['hello', 'ola', 'thammi']})
vdf = pd.DataFrame({"a": [4, 5, 6], "b": ['four', 'five', 'six']})
tds = Dataset.from_pandas(tdf)
vds = Dataset.from_pandas(vdf)
ds = DatasetDict()
ds['train'] = tds
ds['validation'] = vds
print(ds)
```
---
**Author:**  [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [February 23, 2022, 1:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F5 "2022-02-23T13:11:08Z")
Hi @akomma ! Yes, your second approach is the correct one.
---
**Author:**  [@irfan767](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Firfan767)%5C
**Post date:** [December 2, 2023, 9:51am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F6 "2023-12-02T09:51:30Z")
Page is not there. Could you please add a page link to that again or page name ?
---
**Author:**  [@ehalit](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fehalit)%5C
**Post date:** [December 3, 2023, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F7 "2023-12-03T09:01:17Z")
Look for the from\_pandas method at [link](https://huggingface.co/docs/datasets/v2.15.0/en/package_reference/main_classes#datasets.Dataset.from_pandas)
---
**Author:**  [@vpkprasanna](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvpkprasanna)%5C
**Post date:** [June 18, 2024, 2:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F8 "2024-06-18T14:39:02Z")
this [link](https://huggingface.co/docs/datasets/en/loading#load) should work
---
**Author:**  [@Tarun-1999M](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FTarun-1999M)%5C
**Post date:** [July 3, 2024, 4:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F9 "2024-07-03T16:21:58Z")
Thank you it is helpful
---
**Author:**  [@Innovator2K](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FInnovator2K)%5C
**Post date:** [December 20, 2024, 3:49pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ffrom-pandas-dataframe-to-huggingface-dataset%2F9322%2F10 "2024-12-20T15:49:57Z")
Hi @mariosasko! What did you mean by the second approach?
```auto
dataset = Dataset.from_pandas(df, split='train')
```
still doesn’t work in 2024. Do I still have to create it with something like this:
```auto
dataset = DatasetDict({"train": tds, "val": vds})
```
?