# Perform 1 Pretrain epoch on Pretrained model

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-1-pretrain-epoch-on-pretrained-model%2F20043%3E%5C
**Category:** Beginners\
**Created:** [July 5, 2022, 11:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-1-pretrain-epoch-on-pretrained-model%2F20043 "2022-07-05T23:41:46Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![rufimelo](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rufimelo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frufimelo)%5C
**Post date:** [July 5, 2022, 11:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fperform-1-pretrain-epoch-on-pretrained-model%2F20043%2F1 "2022-07-05T23:41:46Z")

</div>

Hello,

I’ve come across the "How to train a new language model from scratch using Transformers and Tokenizers " blog post and wanted to perform 1 epoch of pretraining on a already pretrained model.

My goal was to use it later on to perform semantic search.

My code (adapted) was:

```auto
from transformers import AutoTokenizer 
from transformers import AutoModelForPreTraining 
from transformers import LineByLineTextDataset
from transformers import DataCollatorForLanguageModeling

model = AutoModelForPreTraining .from_pretrained('neuralmind/bert-large-portuguese-cased')
tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-large-portuguese-cased', do_lower_case=False)

dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="dataForModel.txt",
    block_size=128,
)

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=True, mlm_probability=0.15
)

from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./newModel",
    overwrite_output_dir=True,
    num_train_epochs=1,
    per_gpu_train_batch_size=64,
    save_steps=10_000,
    save_total_limit=2,
    prediction_loss_only=True,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
)

trainer.train()

```

I receive the error:

```auto
TypeError: forward() got an unexpected keyword argument 'labels'

```

Any help understanding what am I doing wrong?  
Thanks in advance  
Rui