frouioui/tagenal

Tagenal is a playground app using Vitess, Redis, Traefik and Jaeger on Kubernetes

Tagenal

Description

Tagenal is a playground with a set of tools that enable experimentation in a cloud-native application.

Tagenal uses:

• Relational Database Sharding with Vitess
• Container Orchestration with Kubernetes
• Complete Runtime Tracing and Observation with Jaeger
• Application State Monitoring with Grafana, Alertmanager and Promotheus
• Caching with Redis Cluster
• APIs and Front-End Application

Requirements

To run locally, tagenal needs:

• Have at least 10Gb of available RAM on the host
• Have Kubernetes / Minikube installed
• Have Golang version 1.15.x installed
• Install yq a YAML processor
• Install jsonnet-bundler (jb), allowing us to deal with jsonnet files
• Install jsonnet using your system's packet manager
• Install vtctlclient, the following command go get vitess.io/vitess/go/cmd/vtctlclient can be used
• Install mysql and mysql-client using your system's packet manager
• Install gojsontoyaml, the following command go get github.com/brancz/gojsontoyaml can be used
• Run the shell scripts that are located in ./lib/*.sh. These scripts will download the necessary basic libraries and repositories

Quick Start

In this quick start we will cover the following items:

1. Setup the Kubernetes cluster
2. Setup Jaeger
3. Setup Traefik Proxy
4. Setup the Vitess cluster
5. Setup the Redis cluster
6. Setup monitoring with Grafana, Prometheus and Alertmanager
7. Setup the APIs and frontend

Once the quick start is over, we will have a fully setup application using distributed database systems.

Generate sample data

The documentation and script to generate the sample data can be found here.

Architecture

tagenal's Issues

Add the setup of the monitoring Kubernetes keyspace in main readme

The main readme should also contain the instruction to start the whole monitoring keyspace.

The monitoring keyspace is composed of the Grafana/Prometheus/Alertmanager suite.

Addition of the article sharded table

Adding the article table to the region-based sharded cluster could be a good direction to take. We have multiple choices, either create a whole new key-space e.g. "articles", or add the article table to the users key-space, which will require to change name. Both options have to be considered, the first one seeming (for now) better in terms of scalability.

Articles will be fragmented depending on their category, for now we have 3 categories, 2 categories are located in 2 separated shards, but the 3rd category is supposed to be replicated across the 2 initial shards, which could potentially be an issue regarding the architecture of Vitess, some research needs to be done on the replication system of Vitess before moving forward.

Change MongoDB test implementation folder location

Change the MongoDB test's folder into another pathname, to prepare migration of the MySQL Vitess cluster codebase into the repository's root.

Remove port forwarding

All port forwarding scripts, command, and related documentation can be deleted, since the integration of Traefik #10 port forwarding is no longer necessary.

Implementation of a users API service

Implementation of an API which will handle all users logic. This very simple API will, for now, implement the HTTP protocol so it can easily be tested out, however once a frontend service is created we could restrict this API to using only gRPC and stay fully internal to the Kubernetes cluster.

This API must communicate with the Vitess Cluster, perform some basic insert operations (single and bulk), select operations, and update.

Add tracing between the APIs and Redis Cluster

Some work concerning the integration of tracing all the way to Redis Cluster have already been started in PR #40. However, implementation was not completed.

Migrate from Travis CI to GitHub Actions

A migration from Travis CI to GitHub Actions could be integrated. GitHub Actions would allow uploading docker images directly on GitHub's registry easily.

Having everything on the same same website, rather than two, is also more convenient.

Some new features can be potentially added in the pipeline later on, such as building docker images when a new release/tag is created.

Change "countries" to "regions" for user table sharding

The commands, and configuration files, that are involving the region-based sharding of the user table have to change the "countries" keyword to "regions".

For instance, the Kubernetes ConfigMap in vitess_config_cluster.yaml have to be modified, the ConfigMap's subpath countries.json will be changed to regions.json.

Add a cache layer between the APIs and Vitess cluster

Between both APIs and Vitess cluster, a caching layer can be added. Improving query efficiency from the frontend/apis.

Add documentation for Jaeger

Some documentation could be added in:

• Main README
• Articles API's code
• Users API's code
• Frontend's code

The documentation should cover how to run, stop, access Jaeger. Concerning the code's documentation, in #32 new functions were added in the 3 services (articles, users, frontend), but no further documentation/comments were added.

Addition of Prometheus and Grafana in the Kubernetes cluster

Monitoring solutions such as Prometheus/Granafa could be implemented in the Kubernetes cluster to monitor, for now, the state of the Vitess cluster.

Vitess seems to implement a MySQL Exporter in each VTablet pod.

Update project's main Makefile with new pathnames

After fixing issue #13 , the main Makefile's path and command lines will probably need to be updated.

Simplify the main Makefile

The Makefile looks more and more messy, and could be written in an easier way.

The Makefile could also delegate some of the tasks to shell scripts, for instance the setup of the Vitess cluster contains lot of tasks that could be packed into a shell script.

Migrate MySQL test codebase into the repository's root

The whole codebase located in ./mysql should be migrated to the repository's root. The Vitess Cluster will now be used as the main part of this project.

The Makefile will, possibly, need to be updated according to the new paths.

Use open tracing common tag names for spans

To increase probability and correctness, the common tag names proposed by opentracing (ext package) could be used. They are already partially used in the frontend thanks to echo framework. They still need to be implemented in the two APIs.

Support multiple operating systems for minikube

The Makefile uses the --driver=hyperkit command when it starts Minikube, which could lead to issues on other distribution such as linux/windows.

As referenced here: https://minikube.sigs.k8s.io/docs/drivers/, other drivers could be tested and implemented.

Implement a Hadoop cluster

A small and simple Hadoop cluster could be integrated in the Kubernetes cluster. The project requirements initially mentioned integrating a big data system into our infra.

Fix git clone address in kube-prometheus install script

The suffix of the git clone address for the is spelled .github instead of .git in the following code:

tagenal/lib/get-kube-prom.sh

Line 3 in d45c5e4

git clone --depth 1 [email protected]/prometheus-operator/kube-prometheus.github

Update main readme after migration of mysql prototype to repository's root

After the mysql prototype is merged into the repository's root, the main README will need to be updated. Issue #17 is highly related.

The README will need to host new informations concerning the technologies and tools that are being used. The SQL data models, the updated list of feature, and an updated list of command to run in order to get the Kubernetes cluster started.

Use gRPC between frontend and APIs

The frontend service currently uses HTTP to communicate with the APIs users and articles, however they both implement gRPC endpoints.

One of the issue in the required design to fix this issue is the sharing of the protobuf files. They could either be duplicated, put in a separated go module that we import, or separated git module.

Implementation of the Orchestrator in Kubernetes

The Vitess cluster already implemented in #1 could get benefit from the addition of the tool orchestrator (https://github.com/openark/orchestrator) which is used to manage MySQL replication topologies.

Vitess supports the an easy integration of this tool when using Kubernetes.

Remove old NoSQL data models from main readme

The repository's main README needs to be updated. MongoDB planning on being removed from the repository, the documentation concerning NoSQL must be deleted.