# Dataset object has no attribute \`to\_tf\_dataset\` **URL:** **Author:** ![rajkumar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rajkumar/32/3072_2.png) [@rajkumar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frajkumar)%5C **Post date:** [November 20, 2021, 11:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F1 "2021-11-20T11:06:35Z") I am following HuggingFace Course. I am at Fine-tuning a model. Link: [Fine-tuning a pretrained model - Hugging Face Course](https://huggingface.co/course/chapter3/2?fw=tf) I use tokenize\_function and `map` as mentioned in the course to process data. `# define a tokenize function` `def Tokenize_function(example):` ` return tokenizer(example['sentence'], truncation=True)` `# tokenize entire data` `tokenized_data = raw_data.map(Tokenize_function, batched=True)` I get Dataset object at this point. When I try converting this to a TF dataset object as mentioned in the course, it throws the following error. `# convert to TF dataset` `train_data = tokenized_data["train"].to_tf_dataset( ` ` columns = ['attention_mask', 'input_ids', 'token_type_ids'],` ` label_cols = ['label'],` ` shuffle = True,` ` collate_fn = data_collator,` ` batch_size = 8` `)` Output: `---------------------------------------------------------------------------` `AttributeError Traceback (most recent call last)` `/tmp/ipykernel_42/103099799.py in ` ` 1 # convert to TF dataset` `----> 2 train_data = tokenized_data["train"].to_tf_dataset( \` ` 3 columns = ['attention_mask', 'input_ids', 'token_type_ids'], \` ` 4 label_cols = ['label'], \` ` 5 shuffle = True, \` `AttributeError: 'Dataset' object has no attribute 'to_tf_dataset'` When I look for `dir(tokenized_data["train"])`, there is no method or attribute in the name of `to_tf_dataset`. Why do I get this error? And how to clear this? Please help me. ---
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C **Post date:** [November 20, 2021, 9:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F2 "2021-11-20T21:01:47Z")
Hey @rajkumar I believe the `to_tf_dataset()` method was only added in a recent version of `datasets`. Could you try upgrading to the latest version and check if the problem persists? ---
**Author:** ![rajkumar](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/rajkumar/32/3072_2.png) [@rajkumar](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frajkumar)%5C **Post date:** [November 21, 2021, 6:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F3 "2021-11-21T06:53:06Z")
Hi @lewtun. You are absolutely correct. I upgraded transformers and datasets to the latest versions and the issues are resolved. ```auto # upgrade transformers and datasets to latest versions !pip install --upgrade transformers !pip install --upgrade datasets ``` Thanks a lot for your timely reply. ---
**Author:** ![BabaYao](https://avatars.discourse-cdn.com/v4/letter/b/71e660/32.png) [@BabaYao](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBabaYao)%5C **Post date:** [February 25, 2022, 11:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F4 "2022-02-25T23:45:18Z")
Hello am getting the same error, though using the newer version of transformers and datasets: UnexpectedStatusException: Error for Training job sample-huggingface-training-2022-02-25-22-41-34: Failed. Reason: AlgorithmError: ExecuteUserScriptError: ExitCode 1 ErrorMessage "AttributeError: ‘Dataset’ object has no attribute ‘to\_tf\_dataset’ " These are the versions am using: sagemaker: 2.77.0 transformers: 4.11.0 tensorflow: 2.7.1 dataset version: 1.18.3 How do I resolve this? Thanks ---
**Author:** ![Ayomidejoe](https://avatars.discourse-cdn.com/v4/letter/a/eb9ed0/32.png) [@Ayomidejoe](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAyomidejoe)%5C **Post date:** [September 30, 2022, 5:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F5 "2022-09-30T05:26:56Z")
I am having this problem too (‘DatasetDict’ object has no attribute ‘to\_tf\_dataset’) I have tried the following with no solution: # upgrade transformers and datasets to latest versions !pip install --upgrade transformers !pip install --upgrade datasets I will appreciate any help on how to solve this challenge ---
**Author:** ![soundaryasundari](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/soundaryasundari/32/12934_2.png) [@soundaryasundari](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsoundaryasundari)%5C **Post date:** [December 6, 2022, 8:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F6 "2022-12-06T20:07:42Z")
I found a solution! 😊 Wow! Looks like many are having similar problems because TensorFlow _seems_ to have difficulty working or integrating with Pandas dataframes to convert to **tf.data.Dataset** format using **to\_tf\_dataset()**, in general. Therefore I followed this solution from Stack Overflow and it worked! I just had to repeat the same steps that I had been doing with Pandas dataframes and tokenize. TensorFlow needs to get this solved instead of having to use: ```auto from datasets import Dataset tf_dataset = Dataset.from_pandas(dataframe) ``` Hope this helps~ Happy Automating! 🤗 Link: [nlp - Attribute error: DatasetDict' object has no attribute 'to\_tf\_dataset' - Stack Overflow](https://stackoverflow.com/questions/73937384/attribute-error-datasetdict-object-has-no-attribute-to-tf-dataset) ---
**Author:** ![MUmairAB](https://avatars.discourse-cdn.com/v4/letter/m/54ee81/32.png) [@MUmairAB](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMUmairAB)%5C **Post date:** [July 8, 2023, 1:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-object-has-no-attribute-to-tf-dataset%2F12099%2F8 "2023-07-08T13:41:52Z")
I also faced the similar problem and though @soundaryasundari’s solution helped a little bit, it is not complete. So, I thought I should share mine. First, we need to convert the DatasetDict Object to Pandas DataFrame. For that, **dataset.set\_format()** and **dataset.with\_format()** should work. But unfortunately, they don’t. We need to manually convert them as follows: ```auto #Convert the dataset to DataFrame pd_data = pd.DataFrame(dataset["train"]) #Now, convert it back to the DatasetDict object from datasets import Dataset ds_data = Dataset.from_pandas(pd_data) ``` Now, it can be converted to tf.data object.