# BERT for text classification

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-for-text-classification%2F78198%3E%5C
**Category:** 🤗Transformers\
**Created:** [March 20, 2024, 5:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-for-text-classification%2F78198 "2024-03-20T05:24:24Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Qeerah](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/qeerah/32/25026_2.png) [@Qeerah](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FQeerah)%5C
**Post date:** [March 20, 2024, 5:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-for-text-classification%2F78198%2F1 "2024-03-20T05:24:24Z")

</div>

Hi. I am trying to use BERT with CNNBiLSTM for text classification but seems to be having an incompatibility issue with the transformer and TensorFlow. I used the latest version of TensorFlow version: 2.15.0 and Transformers version: 4.38.2. But once I run this:  
from transformers import BertTokenizer, TFBertForSequenceClassification

model = TFBertForSequenceClassification.from\_pretrained(‘bert-base-uncased’)  
tokenizer = BertTokenizer.from\_pretrained(‘bert-base-uncased’)

def bert\_model(model\_name=‘bert-base-uncased’, max\_length=50):  
bert\_encoder = TFBertModel.from\_pretrained(model\_name)  
input\_ids = Input(shape=(max\_length,), dtype=tf.int32, name=“input\_ids”) # Correct variable name used here  
attention\_mask = Input(shape=(max\_length,), dtype=tf.int32, name=“attention\_mask”)  
bert\_output = bert\_encoder(input\_ids, attention\_mask=attention\_mask)[0]  
x = SpatialDropout1D(0.2)(bert\_output)  
x = Conv1D(64, 3, activation=‘relu’)(x)  
x = Bidirectional(LSTM(64, dropout=0.2))(x)  
x = Dense(256, activation=‘relu’)(x)  
x = Dropout(0.4)(x)  
x = Dense(128, activation=‘relu’)(x)  
outputs = Dense(1, activation=‘sigmoid’)(x)  
model = Model(inputs=[input\_ids, attention\_mask], outputs=outputs)  
return model

model = bert\_model()  
adam\_optimizer = tf.keras.optimizers.Adam(learning\_rate=1e-5)  
model.compile(loss=‘binary\_crossentropy’, optimizer=adam\_optimizer, metrics=[‘accuracy’])

model.summary()

history = model.fit(  
train\_dataset,  
epochs=3,  
verbose=1  
)

I get this error  
TypeError: Exception encountered when calling layer ‘embeddings’ (type TFBertEmbeddings).

Could not build a TypeSpec for name: “tf.debugging.assert\_less\_24/assert\_less/Assert/Assert”  
op: “Assert”…

Please help out. Thanks