# LayoutLMv3 processor error

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%3E%5C
**Category:** Intermediate\
**Created:** [September 27, 2024, 8:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976 "2024-09-27T08:21:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AmitALU](https://avatars.discourse-cdn.com/v4/letter/a/47e85d/32.png) [@AmitALU](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAmitALU)%5C
**Post date:** [September 27, 2024, 8:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%2F1 "2024-09-27T08:21:00Z")

</div>

I am currently using the finetuned LayoutLMv3 on the FUNSD dataset.

When I was using the model for new images, I noticed a problem using the processor.

encoding = processor(resized\_image, words, boxes=boxes, return\_offsets\_mapping=True, return\_tensors=“pt”)

The elements of boxes can only be less than 1000. so I have resized the boxed and also the image to have a max dim of 1000.

Is that the correct way of doing things?

Then, I have encountered an error during

with torch.no\_grad():  
outputs = model(\*\*encoding)

“IndexError: index out of range in self”

Can anybody explain to me why this error?  
encoding = processor(resized\_image, words, boxes=boxes, return\_offsets\_mapping=True, return\_tensors=“pt”)

And lastlyb are all the images resized to (224,224)?  
Here are the encodings.  
for k,v in encoding.items():  
print(k,v.shape)

Outputs:  
input\_ids torch.Size([1, 795])  
attention\_mask torch.Size([1, 795])  
offset\_mapping torch.Size([1, 795, 2])  
bbox torch.Size([1, 795, 4])  
pixel\_values torch.Size([1, 3, 224, 224])

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 27, 2024, 8:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%2F2 "2024-09-27T08:48:46Z")

</div>

Apparently there is a precedent. It seems that the dataset and the model are incompatible. You will probably need to normalize the dataset manually.

> <https://github.com/microsoft/unilm/issues/771>
>
> I'm using \`transformers\` to finetune \` microsoft/layoutlmv2-base-uncased\` on my …custom dataset that is similar to FUNSD. After a few iterations of training I get this error :
> \`\`\`
> Traceback (most recent call last):
> File "layoutlmV2/train.py", line 137, in \<module\>
> trainer.train()
> File "..../lib/python3.8/site-packages/transformers/trainer.py", line 1409, in train
> return inner\_training\_loop(
> File "..../lib/python3.8/site-packages/transformers/trainer.py", line 1651, in \_inner\_training\_loop
> tr\_loss\_step = self.training\_step(model, inputs)
> File "..../lib/python3.8/site-packages/transformers/trainer.py", line 2345, in training\_step
> loss = self.compute\_loss(model, inputs)
> File "..../lib/python3.8/site-packages/transformers/trainer.py", line 2377, in compute\_loss
> outputs = model(\*\*inputs)
> File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling\_layoutlmv2.py", line 1228, in forward
> outputs = self.layoutlmv2(
> File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling\_layoutlmv2.py", line 902, in forward
> text\_layout\_emb = self.\_calc\_text\_embeddings(
> File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling\_layoutlmv2.py", line 753, in \_calc\_text\_embeddings
> spatial\_position\_embeddings = self.embeddings.\_calc\_spatial\_position\_embeddings(bbox)
> File "..../lib/python3.8/site-packages/transformers/models/layoutlmv2/modeling\_layoutlmv2.py", line 93, in \_calc\_spatial\_position\_embeddings
> h\_position\_embeddings = self.h\_position\_embeddings(bbox\[:, :, 3\] - bbox\[:, :, 1\])
> File "..../lib/python3.8/site-packages/torch/nn/modules/module.py", line 1131, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "..../lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 158, in forward
> return F.embedding(
> File "..../lib/python3.8/site-packages/torch/nn/functional.py", line 2203, in embedding
> return torch.embedding(weight, input, padding\_idx, scale\_grad\_by\_freq, sparse)
> IndexError: index out of range in self
> \`\`\`
> After further inspection (vocab size, bboxes, dimensions, classes...) I noticed that there's negative numbers inside the input tensor causing the error. These negative numbers are returned by \_calc\_spatial\_position\_embeddings(self, bbox) in modeling\_layoutlmv2.py line 92 :
> 
> \`h\_position\_embeddings = self.h\_position\_embeddings(bbox\[:, :, 3\] - bbox\[:, :, 1\]) \`
> 
> What does this line exactly do ? Assuming that the negative values are the cause of the issue, what could I do to prevent the input values from becoming negative ? Thanks in advance !

> <https://github.com/microsoft/unilm/issues/821>
>
> \*\*Describe the bug\*\*
> Model I am using (UniLM, MiniLM, LayoutLM ...): 
> LayoutLM…v3
> 
> The problem arises when using:
> \* my own modified scripts: (give details below)
> 
> 
> I'm using LayoutLMv3 from huggingface transformers - https://huggingface.co/docs/transformers/model\_doc/layoutlmv3#transformers.LayoutLMv3Model - and I am getting an "IndexError: index out of range in self" on some inputs. It works fine for some inputs, but fails for others. I found a similar issue here -https://github.com/microsoft/unilm/issues/771 - but there, the problem was with bboxes. However, I am using the full processor with OCR that creates the bounding boxes for me, so my only actual input are PIL images.
> I initialize the processor with 
> \`processor = AutoProcessor.from\_pretrained("microsoft/layoutlmv3-base", apply\_ocr=True)\`
> and the call it with
> \`processor(img1, return\_tensors="np", padding='max\_length')\` (img1 is PIL image)
> The full stack trace is
> \`\`\`
> Traceback (most recent call last):
> File "/home/h/projects/layoutLM/main\_layoutLM.py", line 58, in \<module\>
> main()
> File "/home/h/projects/layoutLM/main\_layoutLM.py", line 37, in main
> output1, output2 = net(img0, img1)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "/home/h/projects/layoutLM/layoutlm\_classifier\_contrastive.py", line 36, in forward
> output1 = self.forward\_once(input1)
> File "/home/h/projects/layoutLM/layoutlm\_classifier\_contrastive.py", line 29, in forward\_once
> output = self.layout\_LM(\*\*x)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/transformers/models/layoutlmv3/modeling\_layoutlmv3.py", line 833, in forward
> embedding\_output = self.embeddings(
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/transformers/models/layoutlmv3/modeling\_layoutlmv3.py", line 261, in forward
> position\_embeddings = self.position\_embeddings(position\_ids)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1130, in \_call\_impl
> return forward\_call(\*input, \*\*kwargs)
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/modules/sparse.py", line 158, in forward
> return F.embedding(
> File "/home/h/projects/layoutLM/venv/lib/python3.9/site-packages/torch/nn/functional.py", line 2199, in embedding
> return torch.embedding(weight, input, padding\_idx, scale\_grad\_by\_freq, sparse)
> IndexError: index out of range in self
> 
> Process finished with exit code 1
> 
> \`\`\`
> where \`output = self.layout\_LM(\*\*x)\` is the actual calling of LayoutLM, which is created by
> \`self.layout\_LM = AutoModel.from\_pretrained("microsoft/layoutlmv3-base")\`
> I have no idea what could be wrong, as I said, it works fine for some inputs and I don't see anything special about cases where it fails 
> 
> 
> 
> \- Platform: Ubuntu 20.04
> \- Python version: 3.9
> \- PyTorch version (GPU?): 1.12.0+cpu

> <https://stackoverflow.com/questions/62081155/pytorch-indexerror-index-out-of-range-in-self-how-to-solve>

---

<div class="post-metadata">

**Author:** ![AmitALU](https://avatars.discourse-cdn.com/v4/letter/a/47e85d/32.png) [@AmitALU](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAmitALU)%5C
**Post date:** [September 27, 2024, 9:12am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%2F3 "2024-09-27T09:12:35Z")

</div>

> [@John6666](#):
>
> recedent. It seems that the dataset and the model are incompatible. You will probably need to normalize the dataset manually.

For me the normalization was not the problem.  
As mentioned in the above posts one of the recurrring problems was that the bounding boxes were too small.

That was not the problem for me:

# code to check if each bbox is at least (1,1)

for bbox in bounding\_boxes:  
assert bbox[2] - bbox[0] \> 1  
assert bbox[3] - bbox[1] \> 1

The problem was that the embedding layer in model wass not accepting the input ids in the data sample. This generally happens when the length of data sample is more than 512. one has to set the truncate parameter to True. So that the length never more than 512. Mine was 700.

encoding = processor(original\_image, words, boxes=boxes, return\_offsets\_mapping=True, max\_length=512, padding=“max\_length”, truncation=True, return\_tensors=“pt”)

But still have not figured out why resize to 224,224.

Thanks John6666

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 27, 2024, 10:21am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%2F4 "2024-09-27T10:21:18Z")

</div>

I’m glad you were able to resolve some of this.

> But still have not figured out why resize to 224,224.

Other models such as the SigLip, for example, also resize to that size, though not exactly the same size, so perhaps the current model is designed for that level of resolution.  
However, there are a lot of things I don’t understand, such as why it is necessary to reduce the resolution that much, and which one gives better results if it is stretching, padding, or cropping.  
Well, I’m not mainly dealing with VLM or LLM, so I’m going to assume that as long as there are no problems with the operation, it’s fine.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [September 30, 2024, 5:16am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Flayoutlmv3-processor-error%2F108976%2F5 "2024-09-30T05:16:05Z")

</div>

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.