Giter Site home page Giter Site logo

spiderpig86 / smoresunderflow Goto Github PK

View Code? Open in Web Editor NEW
12.0 3.0 3.0 55.74 MB

:hotsprings: A scalable StackOverflow clone powered by microservices.

JavaScript 1.97% Shell 0.59% TypeScript 94.94% HTML 0.80% CSS 1.70%
microservices elastic-search koa pm2 redis mongodb database shards react stackoverflow

smoresunderflow's Introduction

SmoresUnderflow


A scalable StackOverflow clone powered by Koa.

Features

  • This is the gist of what it can do:
    • User account registration with email verification.
    • Posting questions and answers with media.
    • Upvoting/downvoting questions and answers.
    • Search and filtering for questions based on timestamp, query, count, and more.

Built With

  • โšก React - a JavaScript library for building UIs.
  • ๐Ÿ’Ž Koa - a next generation web framework for Node.js.
  • ๐Ÿต MongoDb - the most popular database for modern apps.
  • ๐Ÿœ jwt - compact and secure data transmission for stateful services.
  • ๐Ÿ” Elastic Search - scalable open source search.
  • ๐Ÿ’‚โ€โ™‚๏ธ Kibana - data visualization for Elastic stack.
  • ๐Ÿ’ฐ Redis - in-memory data structure store/cache.
  • ๐Ÿš€ Nginx - high performance load balancer.
  • ๐Ÿฐ RabbitMQ - open source message broker.
  • โœจ PM2 - Node.js process manager.

Performance

  • Was able to handle 1000 concurrent users sending a total of 200,000 requests within 10 minutes (averaging ~333 QPS), maintaining a 95th percentile tail latency of 200ms.

Deployment

  • In general, each instance you deploy would consist of all the microservices that are included in the project, which are the folders prefixed with su- and qu-.
  • The option is to deploy them separately and cluster each microservice by type on each machine. This may reduce CPU load on instances that share both the backend microservices and the queueing.
  • What I recommend is doing a mix of machines that take on a couple of microservices and partially specializing the instances.
  • Assuming that you are familiar with Ansible, there are playbooks written to help speed up setting up instances. PM2 is used to deploy and maintain the instances.

App Instances

  • The app will be able to sustain quite a bit of load if all microservices are located on each machine.
    • Some specialization, such as moving the search and questions microservices to separate machines may lower the overhead, since these services are often hit the most.

Database (Mongodb)

  • It is recommended to have a query router for each app instance you have.
  • However, shards and config servers should run by itself in its own instance to avoid performance hits.
  • Deploying 3 shards, 3 config servers and one query router for each app instance should be sufficient.

Caching

  • Redis will work best when running by itself on its own instance.
  • Clustering is possible, but performance is slower than a single process based on testing.

Searching

  • The ELK stack can run on a single machine. Ensure that there is at least 4GB of RAM so the write queue limit can be raised in Elastic Search to handle more requests.

Architecture

Not the best way to do it, but worked nonetheless.

Architecture

How it looks/works

Smores Demo Question Page Answer Component Home Page Ask Page Search Page Search Filter

Contributors

smoresunderflow's People

Contributors

dependabot[bot] avatar spiderpig86 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.