miliar/tech-stack-demand

An example microservice pipeline to visualize the tech stack demand on the current job market

Tech stack demand

An example microservice pipeline to visualize the tech stack demand on the current job market.

Show who is using certain technologies:

Search for company:

Installation

• Install docker/docker-compose
• Assure docker has enough resources (Preferences > Advanced; tested with 4 CPUs and 6 GiB RAM)
• Run docker-compose up
• Open in your browser http://localhost:5000/
• Collect new data (reload page to see progress; this can take up to 20 min)

How to use the UI

• Check out this basic action wiki

Technical sidenotes

Architecture

Browse kafka topics:

• Kafka logs are not mounted
• While collecting new data, check http://localhost:9000/

Manage Neo4j data:

• Data mounted to /neo4j_data_loader/neo4j_data
• Open Neo4j browser: http://localhost:7474/
• Login with user: neo4j password: password

Run tests:

• docker-compose -f docker-compose-tests.yml up

Run keywords api update:

• Delete old data saved in keywords_api/redis_data
• Run: docker-compose -f docker-compose-keywords-update.yml up