lovindata/easyeda

📊😌 EasyEDA is a REST API for doing common exploratory data analysis. It is powered by Typelevel stack http4s with Cats Effect, Spark and Redis providing fully vertically scaled parallel request processing and extremely fast in-memory non-persistent data manipulation & computation 🚀🤩!

EasyEDA - Easy fast exploratory data analysis

📊😌 EasyEDA is a REST API for doing common exploratory data analysis. It is powered by Typelevel stack http4s with Cats Effect, Spark and Redis providing fully vertically scaled parallel request processing and extremely fast in-memory non-persistent data manipulation & computation. It means none of your source data are saved somewhere and everything are done on the fly 🚀🤩!

Want to contribute ? 😉

• Stack used image can be directly modified via draw.io
• IntelliJ IDEA CE was used to build the source code
• This VSCode extension was used to edit the OpenAPI documentation

Logbook (🙏😣 Work still in progress...)

• Setup database and start coding for EloData authentication (😼 HERE AT THE MOMENT 😼)

  • Give up email verification directly go like Snowflake style (SaaS)
  • Understand OAuth2 and see if it's applicable in your case
  • Solve compilation issue with doobie

• Finalize migration

  • Rework the already in-place UTs
    • Issue with combined AsyncFreeSpec + withObjectMocked => Unit (PAUSED)
  • Job will stay on "Running" it error handling need to be on overall withJob => Done need to test

• Think about cron jobs not terminated but still here sessions ?

  • K8S stateless compatibility too be-careful ⚠️
    • The implementation of "not-continue"

• Start learning ReactJs & FrontEnd dev

  • Continue "on all in-one container" with ReactApp served by HTTP4s / Tapir (😼 HERE AT THE MOMENT 😼)

• Start learning event streams & prepare async routes for Job monitoring

• Start trying Tapir for embedding routes

• Work on Logging (Spark logs annoying + No app logging)

• Continue DEV on basic statistics

• Start DEV on Spark NLP Service

  • Introduce the type Sentence

• SCoverage & CI/CD

Notes

• NoSQL Data modelling patterns & theories
• Jedis JavaDoc
• Redis commands