# Bert strugling with Padded sentence **URL:** **Author:** ![Jeydev](https://avatars.discourse-cdn.com/v4/letter/j/dc4da7/32.png) [@Jeydev](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJeydev)%5C **Post date:** [August 24, 2021, 12:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbert-strugling-with-padded-sentence%2F9446%2F1 "2021-08-24T12:42:25Z") I am Playing around with Bert Pretrained Models (bert-large-uncased-whole-word-masking) I used Huggingface to try it I first Used this Piece of Code ```auto m = TFBertLMHeadModel.from_pretrained("bert-large-cased-whole-word-masking") logits = m(tokenizer("hello world [MASK] like it",return_tensors="tf")["input_ids"]).logits ``` I then used Argmax to get max probabilities after applying softmax, Things works fine Until now. When I used padding with max\_length = 100 The model started making false prediction and not working well and all predicted tokens were the same i.e 119-Token ID Code I used for Argmax ```auto tf.argmax(tf.keras.activations.softmax(m(tokenizer("hello world [MASK] like it",return_tensors="tf",max_length=,padding="max_length")["input_ids"]).logits)[0],axis=-1) ``` Output Before using padding ```auto ``` Output After using padding with max\_length of 100 ```auto ``` I wonder if this problem prevail even training a new model as It is mandatory to set Input shape for training new model I Padded and tokenized the data but, now I want to know if this problem continues with it too.