ellenzhuwang / implicitood Goto Github PK

3.0 3.0 0.0 179 KB

An end-to-end vision and language model incorporating explicit knowledge graphs and OOD-detection.

Home Page: https://arxiv.org/pdf/2302.05608.pdf

Python 100.00%

deep-learning knowledge-graph mscoco-dataset multimodal-learning ood-detection transformer vision-and-language visual-question-answering

implicitood's People

Contributors

Stargazers

Watchers

implicitood's Issues

IndexError: list index out of range

Hi author, I found that there is no data_root/name.arrow file when I executed to this step pa.ipc.RecordBatchFileReader( pa.memory_map(f"{data_dir}/{name}.arrow", "r") ).read_all(), may I know the way the arrow file is generated?

Is there something wrong in GMM?

https://github.com/ellenzhuwang/VK_OOD/blob/f7af0f40f1b75e668b0591dfe979c214fae5d189/vkood/modules/vk_module.py#L295
Hello, I would like to ask if there is any problem with the GMM model.
Should it be gmm.fit() or a new function defined as gmm_fit()? This text_embeds will cause dimension error.
In addition, I would like to ask how to use KG to obtain triplets and perform OOD detection.

Full Pretraining Code

Hi, thanks for your valuable work!
I notice that the current repo only contains fine-tuning code on VQA, so could you plan to release the full pretraining code and models of your work? Can't wait for it :)

Recommend Projects