mbaddar/scienceinfluencers

An ArXiv paper search tool

Science Infuencers:

An ArXiv paper search tool

• Using Arxiv API to search for papers
• Using Semantic Scholar to get influence details. Not all papers have matching records there.

Content:

1. Running the app:

2. Containerization:

3. Implementation

4. Hurdles:

5. Future work:

1. Running the app:

• Container endpoint: http://scienceinfluencers-3bb32e13-1.c46e36e7.cont.dockerapp.io/

2. Containerization:

• Docker image built on the AWS staging machine
• Using docker cloud to proxy container hosting on AWS
• Image pushed to Docker hub repository: https://hub.docker.com/r/baddar/scienceinfluencers/
• Docker cloud linked with github for continuous intergration. Unfortunately building on docker hub is very unstable. However I have tried to push updates and it automatically triggers a new image build and deply. This also can be controlled by tagging the image and branching on github (To build a staging container for instance)
• Discarded docker cloud build and used an AWS instance to build the image then push it to docker hub
• Used google clould shell (Tutorial here) to build the image and push it to GCP then create a Kubernetes cluster. Still in progress

3. Implementation:

• Arxiv API: Outputs XML. I parsed it into a pandas dataframe and convert it to HTML, then manually inject hrefs for the URLs as pandas cannot do that
• Using JQuery and DataTables to add sort and pagination to the results
• No database implementation
• Google OAuth 2.0 authentication implemented for login.
• Code in main.py. Google code uses auth.py

4. Hurdles:

• Using Docker Cloud to build my docker image.
• Using Alpine Linux image at first. Since Pandas and many of its dependencies has to be built from source.

5. Future work:

• Rearranging results. Expanding summary. More UX etc.
• Adding Timeline: Histogram trends over time
• Include twitter influence analysis. Semantic scholar does this but no API was exposed
• Database
• Add more ArXiv query capabilities: Search by title, author, etc
• Completing Kubernetes cluster configuration