theblackcat102/language-models-are-knowledge-graphs-pytorch

Language models are open knowledge graphs ( non official implementation )

Hi there 👋

65% research, 35% software dev

Staff ML Engineer as day job

Currently working on these side project topics

• Open Source LLM Assistant : OpenAssistant Conversations -- Democratizing Large Language Model Alignment

• Data mining

  • url domain-region mapping
  • reranking search engine using neural nets

• local assistant ( LLM model ) for handling my daily info aggregation and search

My huggingface profile

language-models-are-knowledge-graphs-pytorch's Issues

using GPT3.5/4

Hi Nikhil, this is great work, very valuable experimenting.

Given the break thro of GPT models, is it possible to re-do your experiments using GPT3.5/4?
Curious about the results...

Thank you so much!

Issue with using REL

Hey,

I get the following error while running the extract.py file.

Traceback (most recent call last):
File "extract.py", line 88, in
mapped_triplet = Map(head, relations, tail)
File "/Users/I535445/Desktop/Me/new/mapper.py", line 12, in Map
head_p_e_m = emb.wiki(str(head), 'wiki')
File "/Users/I535445/Desktop/Me/new/env1/lib/python3.8/site-packages/REL/db/generic.py", line 45, in wiki
g = self.lookup_wik(mention, table_name, column_name)
File "/Users/I535445/Desktop/Me/new/env1/lib/python3.8/site-packages/REL/db/base.py", line 182, in lookup_wik
e = c.execute(
sqlite3.OperationalError: no such table: wiki

Can someone please help ?

Hello, I am a beginner in Python

Hello, I am a beginner in Python. I am studying hard on the code that I really need. There are several python files (utils.py, constant.py, extract.py, mapper.py, stats,py). What order should they be executed? I would really appreciate it if you let me know

Issues with installation and running extract

The install links in requirements.txt for spacy models dont work. I had to manually install them.

Further, after installing the models, I keep getting the following error :
Traceback (most recent call last):
File "language-models-are-knowledge-graphs-pytorch-main/extract.py", line 3, in
from mapper import Map, deduplication
File "language-models-are-knowledge-graphs-pytorch-main/mapper.py", line 3, in
from REL.db.generic import GenericLookup
ModuleNotFoundError: No module named 'REL'

I tried installing rel which further throws the following error :
pip install rel

Looking in indexes: https://pypi.org/simple
Collecting rel
Using cached rel-0.3.1.tar.gz (9.0 kB)
ERROR: Command errored out with exit status 1:
command: language-models-are-knowledge-graphs-pytorch-main/env/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py'"'"'; file='"'"'/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-pip-egg-info-_ir2rrqc
cwd: /private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/
Complete output (7 lines):
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py", line 1, in
from rel import version
File "/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/rel/init.py", line 2, in
from rel import override, supported_methods, initialize, read, write, timeout, signal, event, dispatch, loop, abort, init, sys, EV_PERSIST, EV_READ, EV_SIGNAL, EV_TIMEOUT, EV_WRITE
ImportError: cannot import name 'override'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Any help is appreciated.

sqlite3.OperationalError: unable to open database file

Hi,
python extract.py examples/bob_dylan.txt bert-large-cased-bob_dynlan.jsonl --language_model bert-large-cased --use_cuda true
wandb: WARNING W&B installed but not logged in. Run wandb login or set the WANDB_API_KEY env variable.
Traceback (most recent call last):
File "extract.py", line 3, in
from mapper import Map, deduplication
File "language-models-are-knowledge-graphs-pytorch/mapper.py", line 5, in
emb = GenericLookup("entity_word_embedding", save_dir=sqlite_path, table_name="embeddings")
File "anaconda3/lib/python3.6/site-packages/REL/db/generic.py", line 36, in init
self.db = self.initialize_db(path_db, table_name, columns)
File "anaconda3/lib/python3.6/site-packages/REL/db/base.py", line 41, in initialize_db
db = sqlite3.connect(fname, isolation_level=None)
sqlite3.OperationalError: unable to open database file

thanks

Using GPT as shown in the paper

I am interested in using GPT as the main model as BERT is not performing too well.

Has it been implemented? Otherwise I am ready to implement it myself. Please let me know

Consider adding a license?

Never mind, I clearly need glasses - no idea how I didn't notice there's already a license here. Sorry for the bother!

function of black list relations (and issue with lemmatisation)

Hi black cat,

Thanks for the public implementation!

I was wondering what the function of the blacklist relation is?
Seems like it removed the option that noun_chunk can be in the path, but I see no mention of this in the original paper and I also see that the underlying assumption (that noun chunk can't be relations) might be false. Was hoping you could share your perspective on this? Hopefully I am just missing a good point :)

A sidenote on the lemmatization used in filtering is also that the lemmatization is done on a word by word basis, which is flawed as lemmatization is contextually dependent for instance the lemma of meeting can either be meeting (a meeting) or meet (to meet).

Best and thanks again for the public implementation,
Kenneth

Requirements Missing REL

ModuleNotFoundError: No module named 'REL'

What are the parameters of the python line

What does bert-large-cased-bob_dynlan.jsonl do in the example? Can we use it for different data ?

extract.py not running even with all requirements installed

When running
python extract.py examples/bob_dylan.txt bert-large-cased-bob_dynlan.jsonl --language_model bert-large-cased --use_cuda true

on google collab the following error appears:

/usr/local/lib/python3.7/dist-packages/spacy/util.py:275: UserWarning: [W031] Model 'en_core_web_sm' (2.2.5) requires spaCy v2.2 and is incompatible with the current spaCy version (2.3.2). This may lead to unexpected results or runtime errors. To resolve this, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate
warnings.warn(warn_msg)
2021-04-22 19:14:11.225481: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
Traceback (most recent call last):
File "extract.py", line 3, in
from mapper import Map, deduplication
File "/content/mapper.py", line 5, in
emb = GenericLookup("entity_word_embedding", save_dir=sqlite_path, table_name="embeddings")
File "/usr/local/lib/python3.7/dist-packages/REL/db/generic.py", line 36, in init
self.db = self.initialize_db(path_db, table_name, columns)
File "/usr/local/lib/python3.7/dist-packages/REL/db/base.py", line 41, in initialize_db
db = sqlite3.connect(fname, isolation_level=None)
sqlite3.OperationalError: unable to open database file

The issuues about BFS

Your BFS function always return an array of length 3 because of your "visited[i] == False"
I don't kown it‘s a bug or you do it on purpose.
Actually when I read the Beam Search part of the origin paper,I feel confuse too.
Look forward to your reply