debjyoti385/questionanswernlp

Question Answering System with Natural Language Processing for defined domain.

QUESTION ANSWER NLP PROJECT

Author/ Contributor: Debjyoti Paul, [email protected], uNID: u0992708

###NOTE: In CADE machines use following python path: /usr/local/stow/python/amd64_linux26/python-2.7.3/bin/python PLEASE RUN FIRST $ chmod +x prepare.sh $ ./prepare.sh

• PREPARE MODULE

  • Modules require:
    • numpy
    • scipy
    • scikit-learn
    • beautifulSoup4
    • requests
  • to install them locally and run use following command
  • $ ./prepare.sh

• EXECUTION

  • Help
    • /usr/local/stow/python/amd64_linux26/python-2.7.3/bin/python qa.py -h
  • Execution WITHOUT Coreference Resolution
    • /usr/local/stow/python/amd64_linux26/python-2.7.3/bin/python qa.py -i input.txt -o response.txt -c 0
  • Execution WITH Coreference Resolution
    • /usr/local/stow/python/amd64_linux26/python-2.7.3/bin/python qa.py -i input.txt -o response.txt -c 1
  • NOTE:
    • The final F-score obtained for developset is 32.68% (without coref resolution)
    • The final F-score obtained for testset1 is 38.64% (without coref resolution)

• INFORMATION

  • It usually takes some 30 seconds to initialize Question classifier.
  • This script also installs BART anaphora resolution kit and runs a web server with port 8125
  • TESTED on lab1-13.eng.utah.edu
  • In OSX/ Macbook developset execution completes in 12-15 minutes.

questionanswernlp's Issues

keywords2pickle not found

As per prepare.sh added hard coded path like this
cp -r /home/debjyotp/important/qcdata .
echo "## FETCH COMPLETE ##"
echo "## FETCH developset DATA ##"
cp -r /home/debjyotp/important/developset .
echo "## PREPARING input.txt ##"
cd developset

can you add this in project because of this it show following error :
Traceback (most recent call last):
File "qa.py", line 112, in
q_classifier = QuestionClassifier.get_classifier()
File "modules/QuestionClassifier.py", line 217, in get_classifier
classifier = QuestionClassifier(fine_grain)
File "modules/QuestionClassifier.py", line 178, in init
self.keywordlist2 = load_keywords()
File "modules/keywords_pickle_builder.py", line 26, in load_keywords
save_keywords(keywords=keywords)
File "modules/keywords_pickle_builder.py", line 31, in save_keywords
with open('qcdata/keywords2pickle', 'wb') as fout:
IOError: [Errno 2] No such file or directory: 'qcdata/keywords2pickle'

BART anaphora resolution kit NOT FOUND

--2020-05-12 09:57:51-- http://www.sfs.uni-tuebingen.de/~versley/BART/BART-snapshot.tgz
Resolving www.sfs.uni-tuebingen.de (www.sfs.uni-tuebingen.de)... 134.2.129.121
Connecting to www.sfs.uni-tuebingen.de (www.sfs.uni-tuebingen.de)|134.2.129.121|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2020-05-12 09:57:51 ERROR 404: Not Found.

Can u Add the toolkit zip file or directory

thanks