wbj0110/adam_qas

ADAM - A Question Answering System. Inspired from IBM Watson

ADAM -- Question Answering System

A question answering system that extracts answers from Wikipedia to questions posed in natural language. Inspired by IBM Watson and START. We are currently focused on improving the accuracy of the extracted answers. Follow the creator's blog at shirishkadam.com for updates on progress.

Getting Started

Elasticsearch is being used to store and index the scrapped and parsed texts from Wikipedia. Elasticsearch 7.X installation guide can be found at Elasticsearch Documentation. You might have to start the elasticsearch search service.

$ git clone https://github.com/5hirish/adam_qas.git
$ cd adam_qas
$ pip install -r requirements.txt
$ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"

Note: The above installation downloads the best-matching default english language model for spaCy. But to improve the model's accuracy you can install other models too. Read more at spaCy docs.

$ python -m spacy download en_core_web_md

Running with Docker

$ git clone https://github.com/5hirish/adam_qas.git
$ cd adam_qas
$ docker-compose up

Now both conntainers are up and running. Next step is to enter in the python container and run Adam:

$ docker exec -it $(docker ps -a -q  --filter ancestor=adam_qas_adam) bash
$ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"

References

Find more in depth documentation about the system with its research paper and system architecture here

Requirements

• Python 3.X
• Elasticsearch 7.X

Python Package dependencies listed in requirements.txt Upgrading Elasticsearch 6.X:

• Rolling Update 6.2 to 6.8 > ref
• Rolling Update 6.8 to 7.1 > ref

Features

• Extract information from Wikipedia
• Classify questions with regular expression (default)
• Classify questions with a SVM (optional)
• Vector space model used for answer extraction
• Rank candidate answers
• Merge top 5 answers into one response

Current Project State ?

GitHub Issue #36: Invalid Answers

TODO

• Replace Wikipedia APIs with custom scraper
• Storing extracted data in database (elasticsearch)
• SQLite test input data storage
• Anaphora resolution in both questions and answers
• Machine learning query constructor rather than rule-based
• Improve vector space language model for answer extraction

Contributions

Please see our contributing documentation for some tips on getting started.

Maintainers

• @5hirish - Shirish Kadam