bordias/layoutlm

layoutlm's Issues

how to use image embeddings in layoutLM?

Hi,
I am confused about the image embeddings in the paper

1. Does the pre-training stage only use position embeddings and text embeddings, but not image embeddings? Since in Figure 2, image embeddings are only fed into the downstream block. If so, why the pre-training task called Masked Visual-Language Model? Since there is no vision-related input to the model during training.
2. I cannot find codes related to the image embeddings in this repo, where do you leverage it in the downstream task?

IndexError: index out of range in self

model_path = 'bert-large-uncased'
num_labels = len(labels)
config_class, model_class, tokenizer_class = LayoutlmConfig, LayoutlmForTokenClassification, BertTokenizerFast
config = config_class.from_pretrained(model_path, num_labels=num_labels+1)
tokenizer = tokenizer_class.from_pretrained(model_path, do_lower_case=True)
model = model_class.from_pretrained(model_path, from_tf=bool(".ckpt" in model_path), config=config)
model = model.to(device)

max_seq_length = 150
pad_token_label_id = CrossEntropyLoss().ignore_index
train_dataset = CordDataset(train, tokenizer, labels, pad_token_label_id)
validation_dataset = CordDataset(val, tokenizer, labels, pad_token_label_id)
model_type = 'layoutlm'

Traceback:

IndexError                                Traceback (most recent call last)
<ipython-input-31-34ebe37ab2df> in <module>()
     30           batch[2].to(device) if model_type in ["bert", "layoutlm"] else None)
     31       # try:
---> 32       outputs = model(**inputs)
     33       # except Exception as e:
     34       #   print("===>", e, step, batch[0].shape, batch[1].shape, batch[2].shape, batch[3].shape, batch[4].shape)

8 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   1850         # remove once script supports set_grad_enabled
   1851         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1852     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
   1853 
   1854 

IndexError: index out of range in self

Can you help what's going wrong (I guess it's due to config only) and how to solve it?

I am using LayoutLM for NER Task.

Requesting for help me to run the code

Hi, I hope you are doing well, The purpose of writing this email is to face issues, while running this code, please help me to run this code, I really need your help.

My email: [email protected]
Whatsapp: [phone]

Thank you
Best Regards,
Ahsan

This error originates from a subprocess, and is likely not a problem with pip. Preparing metadata (setup.py) ... error error: metadata-generation-failed

hey i cant install using this command

! cd unilm/layoutlm
! pip install unilm/layoutlm

i am getting an error

Processing ./unilm/layoutlm
  Preparing metadata (setup.py) ... done
Collecting transformers==2.9.0 (from layoutlm==0.0)
  Using cached transformers-2.9.0-py3-none-any.whl (635 kB)
Collecting tensorboardX==2.0 (from layoutlm==0.0)
  Using cached tensorboardX-2.0-py2.py3-none-any.whl (195 kB)
Collecting lxml==4.5.1 (from layoutlm==0.0)
  Using cached lxml-4.5.1.tar.gz (4.5 MB)
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Please help me to solve this error asap