# Train BERT with sentence embeddings

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-bert-with-sentence-embeddings%2F27785%3E%5C
**Category:** Research\
**Created:** [December 14, 2022, 1:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-bert-with-sentence-embeddings%2F27785 "2022-12-14T01:39:56Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![weslleylima](https://avatars.discourse-cdn.com/v4/letter/w/cab0a1/32.png) [@weslleylima](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fweslleylima)%5C
**Post date:** [December 14, 2022, 1:39am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Ftrain-bert-with-sentence-embeddings%2F27785%2F1 "2022-12-14T01:39:56Z")

</div>

Hi,

I’m trying to use calculated sentence embeddings by average pooling of chunks of a long sentence as input to train a model based on the AutoModelForSequenceClassification class. I used the “inputs\_embeds” parameter to pass the embeddings to the model, but something strange is happening. Metrics do not change over time. These are the values that practically remain in the 30 epochs:

{‘eval\_loss’: 0.48057085275650024,  
‘eval\_f1’: 0.3008849557522124,  
‘eval\_roc\_auc’: 0.5,  
‘eval\_accuracy’: 0.0,  
‘eval\_precision’: 0.17708333333333334,  
‘eval\_recall’: 1.0,  
‘eval\_hammingLoss’: 0.8229166666666666,  
‘eval\_runtime’: 0.7474,  
‘eval\_samples\_per\_second’: 149,856,  
‘eval\_steps\_per\_second’: 149,856,  
‘epoch’: 30.0}

Does anyone have any tips on how to train BERT using embeddings as input?