ibm-cloud-architecture/refarch-cloudnative-netflix

Reference application for deploying microservice-based applications onto IBM Bluemix, leveraging the Netflix OSS framework.

IBM Cloud Architecture - Microservices Reference Application for Netflix OSS

Reference applications for deploying microservice-based applications onto IBM Bluemix, leveraging the Netflix OSS framework.

This application has been developed and designed to run in the IBM Bluemix us-south public region. Changes may be required if it is to run on a different IBM Bluemix public region or on a local/dedicated environment.

Architecture

Application Overview

The application is a simple dinner menu that displays available appetizers, entrees, and desserts for a non-existent restaurant. There are several components of this architecture:

• Menu UI & Backend services aggregate all the options and display them to the user
• Individual microservices for menu options among Appetizers, Entrees, and Desserts
• Menu microservices communicate to each other using the Netflix OSS Framework:
  • Zuul provides a proxy layer for the microservices.
  • Eureka provides a service registry. The reusable Java microservices register themselves to Eureka which allows clients to find them.
• Menu microservices can be dynamically configured using the Spring Framework:
  • Spring Cloud Config provides server and client-side support for externalized configuration in a distributed system.

Project Component Repositories

This project runs itself like a microservice project, as such each component in the architecture has its own Git Repository and tutorial listed below.

Infrastructure Components:

1. Eureka - Contains the Eureka application components for microservices foundation
2. Zuul - Contains the Zuul application components for microservices foundation
3. Config - Contains the Config application components

Application Components:

1. Menu UI - User interface for presenting menu options externally
2. Menu Backend - Exposes all the meal components as a single REST API endpoint, aggregating Appetizers, Entrees, and Desserts.
3. Appetizer Service - Microservice providing a REST API for Appetizer options
4. Entree Service - Microservice providing a REST API for Entree options
5. Dessert Service - Microservice providing a REST API for Dessert options

Resiliency Components:

1. Hystrix - Contains the Hystrix Dashboard container version application component for microservices foundation
2. Hystrix CF - Contains the Hystrix Dashboard Cloud Foundry version application component for microservices foundation
3. Turbine - Contains the Turbine application component for microservices foundation

DevOps Components:

1. CF Toolchain - A one click full deployment process which will create a toolchain for the Cloud Foundry version of this architecture
2. IC Toolchain - A one click full deployment process which will create a toolchain for the IBM Bluemix Containers version of this architecture

Branches

1. master - This is the active development branch with the latest code and integrations.
2. BUILD - This is the Milestone 1 branch. It contains base microservice code, Eureka, Zuul, and Config Server components.
3. DEVOPS - This is the Milestone 2 branch. It contains DevOps pipelines for easy deployment to Bluemix, as well as Active Deploy integration scenarios.
4. RESILIENCY - This is the Milestone 3 branch. It contains integration with Netflix Hystrix for circuit breaker implementations, as well as Zipkin/OpenTracing for distributed tracing implementations.
5. MANAGE - This is the (planned) Milestone 4 branch. It will contain cloud-based service management capabilities allowing for 24x7 operational excellence of microservice-based applications.

Run the reference applications locally and in IBM Cloud

Prerequisites: Environment Setup

• Install Java JDK 1.8 and ensure it is available in your PATH

• Install Docker on Windows or Mac

• Acquire the code

  • Clone the base repository: git clone https://github.com/ibm-cloud-architecture/refarch-cloudnative-netflix
  • Clone the peer repositories using the master branch, which is the active development branch: ./clone_peers.sh
  • Or you can specify a branch when cloning the peer projects to clone and checkout a specific milestone branch. By default, master is selected if no parameter is supplied: ./clone_peers.sh BUILD to clone peer projects and use the BUILD branch for all projects

Run locally via Docker Compose

You can run the entire application locally on your laptop via Docker Compose, a container orchestration tool provided by Docker.

Step 1: Build locally

Run one of the following build script to build all the necessary Java projects.

• ./build-all.sh [-d] will build all the components required runnable JARs using Gradle and optionally package them into Docker containers.

• ./build-all.sh -m [-d] will build all the components required runnable JARs using Apache Maven and optionally package them into Docker containers.

Step 2: Run Docker Compose

Run one of the following Docker Compose commands to start all the application components locally:

• docker-compose up to run with output sent to the console (for all 7 microservices)
  or
• docker-compose up -d to run in detached mode and run the containers in the background.

Step 3: Access the application

You can access the application after a few moments via http://localhost/whats-for-dinner! That's easy enough!

The backing services are automatically registered with Eureka and routed to the necessary dependent microservices, upon calling the Menu service.

Run on Bluemix via Cloud Foundry

Run the following script to deploy all the necessary Java projects as Cloud Foundry apps.

./deploy-to-cf.sh

Run on Bluemix via IBM Container Service

Run the following script to deploy all the necessary Java projects as Container Groups.

./deploy-to-ics.sh

refarch-cloudnative-netflix's Issues

documentation need to reflect -d parameter for build_all.sh

In order to get the docker images built for all the microservices, we need to specify the -d param when calling build_all.sh. Documentation does not reflect so at the moment.

default branch in build-all.sh

What's the following line for in build-all.sh?:

#Optional overrides to allow for specific default branches to be used.
DEFAULT_BRANCH=${1:-master}

As far as I understand, build-all.sh will call all build-microservice.sh scripts for each microservice but such script does not have anything to do with branches at all. As far as I can see, it is only the clone-peers.sh script the only one that interacts with branches.

That line does not break anything but I just want to confirm it is useless in the build-all.sh script.

Scaling out

How is scaling out handled for each of these microservices for e.g entree microservice ? Don't see any reference or explanation of that in the reference architecture. Please advise. Thanks

Zipkin support in Docker Compose file for RESILIENCY+ branches

We need to update the Docker Compose files to include the necessary Zipkin images for RESILIENCY & MANAGE branches.

Update utility scripts for more concise repo lists

The following scripts should be updated to single-source the set of repositories and not depend on the GitHub URL (since only clone-peers.sh requires GitHub knowledge)

• clone-peers.sh
• build-all.sh
• deploy-to-cf.sh
• deploy-to-ics.sh

update docker-compose.yml

image names for eureka and zuul microservices do not correspond with the image name these microservices get built with. Need to remove the 'microservices-' from their image names in the docker compose file

Update documentation to account for config-server latency when using Docker Compose

The Config Server is not available when doing a standard docker-compose up and the attached services do not make persistent connections like Eureka.

A simple workaround for this is to run Docker Compose commands in two phases.

1. docker-compose up eureka config - Wait for Eureka and Config server to start completely
2. docker-compose up - Everything else will talk to Eureka and Config server as soon as the other services startup

Update architecture diagram for "BUILD" phase to include Config Server

WFD microservices missing 'deploy-container-group.sh' script from 'bluemix' directory

Similar to the Eureka and Zuul components, we should provide an easy way to individually deploy container groups for each component. These scripts exist already in Zuul and should be mirrored to all the WFD component repositories.

As a possible enhancement, update the deploy-to-ics.sh script to leverage these scripts instead.