logicmonitor/hipster

HipsterShop

Hipster Shop is a cloud-native microservices demo application .Google uses this application to demonstrate use of technologies like Kubernetes/GKE, Istio, Stackdriver, gRPC and OpenCensus. This application works on any Kubernetes cluster, as well as Google Kubernetes Engine. It’s easy to deploy with little to no configuration.

Architecture

Online Boutique is composed of 11 microservices written in different languages that talk to each other over gRPC.

The following table represents functionality of each microservice.

Service	Language	Description
frontend	Go	Exposes an HTTP server to serve the website. Does not require signup/login and generates session IDs for all users automatically.
cartservice	C#	Stores the items in the user's shopping cart in Redis and retrieves it.
productcatalogservice	Go	Provides the list of products from a JSON file and ability to search products and get individual products.
currencyservice	Node.js	Converts one money amount to another currency. Uses real values fetched from European Central Bank. It's the highest QPS service.
paymentservice	Node.js	Charges the given credit card info (mock) with the given amount and returns a transaction ID.
shippingservice	Go	Gives shipping cost estimates based on the shopping cart. Ships items to the given address (mock)
emailservice	Python	Sends users an order confirmation email (mock).
checkoutservice	Go	Retrieves user cart, prepares order and orchestrates the payment, shipping and the email notification.
recommendationservice	Python	Recommends other products based on what's given in the cart.
adservice	Java	Provides text ads based on given context words.
loadgenerator	Python/Locust	Continuously sends requests imitating realistic user shopping flows to the frontend.

Development Guide

This doc explains how to build and run the Hipstershop source code locally.

Screenshot

You can setup the application in two ways :-

1. Docker
2. Local Machine.

Steps to run on Docker

1. Clone the project.

2. If collector is not already setup, setup lmotel collector using following command:

    
   docker run -d -e LOGICMONITOR_ACCOUNT=your_account_name -e LOGICMONITOR_BEARER_TOKEN=your_bearer_token -e LOGICMONITOR_OTEL_NAME="collector_name" -p 55680:55680 -p 55681:55681 -p 4317:4317 -p 4318:4318 logicmonitor/lmotel:latest
    

3. Open the terminal and go to hipster folder. Execute following command.

   export OTLP_FORMAT="OTLP_FORMAT" // values can be HTTP or GRPC
   export SERVICE_NAMESPACE="SERVICE_NAMESPACE"
   export OTLP_ENDPOINT="YOUR_ENDPOINT" //need to append /v1/traces for OTLP HTTP format
   export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_BEARER_TOKEN" //Required for otlp http format if sending traces directly to LM platform
   export OTEL_RESOURCE_ATTRIBUTES="KEYVALUEPAIR"  // values to be comma seperated eg:"key1=value1,key2=value2"
   docker-compose up -d
   

   For AdService exporter endpoint need to be set manually in docker-compose.yml file. Opentelemetry java agent appends /v1/traces for OTLP HTTP format. OTLP_PROTOCOL values can be set either grpc of http/protobuf

4. Access the web frontend through your browser

• Once run above all steps you can access frontend service at http://localhost:8081

Once all four steps are completed successfully,you can view the traces on Logicmonitor Traces page.

When to use OTLP/HTTP or OTLP/gRPC

• OTLP/gRPC
  • OTLP/gRPC is preferred when sending data via collector.
  • This protocol is concerned with reliability of delivery between one pair of client/server nodes and aims to ensure that no data is lost in-transit between the client and the server. Many telemetry collection systems have intermediary nodes that the data must travel across until reaching the final destination (e.g. application -> agent -> collector -> backend)
  • eg: http://localhost:4317
• OTLP/HTTP
  • OTLP/HTTP is preferred while sending data directly to platform using url without sending it to collector.
  • OTLP/HTTP uses Protobuf payloads encoded either in binary format or in JSON format.
  • OTLP/HTTP uses HTTP POST requests to send telemetry data from clients to servers.
  • eg: https://qauattraces01.logicmonitor.com/rest/api/v1/traces

For more information, visit OpenTelemetry Protocol Specification.

Steps to run on Local Machine

Prerequisites

• Docker for Desktop.
• JDK 11
• Installation of Go
• Installation of Python
• Installation of Node
• Installation of Dotnet core.
• Visual Studio

1. Currency Service (Node.js)

   cd currencyservice
   npm i 
   node -r ./tracing server.js 
   

2. Cart Service (C#)

   Open and Run the cart service in Visual Studio.
   OR
   cd cartservice
   dotnet run
   

3. Payment Service (Node.js)

   cd paymentservice
   node -r ./tracing index.js  
   

4. Recommendation Service (Python)

   cd recommendationservice
   pip3 install -r requirements.txt
   export PORT=8082
   export EXPORT_TYPE="OTLP"
   export SERVICE_NAME=LM-RECOMMENDATIONSERVICE
   export OTLP_ENDPOINT=http://localhost:55681/v1/traces
   export SERVICE_NAMESPACE=HIPSTER
   export PRODUCT_CATALOG_SERVICE_ADDR=localhost:4000
   opentelemetry-instrument python3 recommendation_server.py
   

5. Shipping Service (Go)

   cd shippingservice
   go run . .
   

6. ProductCatlog Service (Go)

   cd productcatalogservice
   go run . .
   

7. Checkout Service (Go)

   cd checkoutservice
   go run . .
   

8. Email Service (Python)

   cd emailservice
   pip3 install -r requirements.txt
   export PORT=4009
   export EXPORT_TYPE="OTLP"
   export SERVICE_NAME=LM-EMAILSERVICE
   export OTLP_ENDPOINT=http://localhost:55681/v1/traces
   export SERVICE_NAMESPACE=HIPSTER
   opentelemetry-instrument python3 email_server.py
   

9. Ad Service (Java)

    cd adservice
    gradle build
    java -javaagent:opentelemetry-javaagent-all.jar \
    -Dotel.exporter=otlp \
    -Dotel.resource.attributes=$OTEL_RESOURCE_ATTRIBUTES ,service.name=$SERVICE_NAME,host.name=$HOST_NAME \
    -Dotel.exporter.$EXPORTER.endpoint=http://localhost:4317 \
    -Dotel.exporter.otlp.insecure=true \
    -Dotel.exporter.otlp.protocol=$OTLP_PROTOCOL \
    -jar hipstershop-0.1.0-SNAPSHOT-fat.jar
   

Note : Download opentelemetry-javaagent-all.jar : https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.7.0/opentelemetry-javaagent-all.jar and copy the jar file in folder adservice/tracinglib

10. Access the web frontend through your browser

• Once run above all steps you can access frontend service at http://localhost:8081
• Once all four steps are completed successfully,you can view the traces on Logicmonitor Traces page.

Also, the following env variables are needed to be set explicitly for every service while running locally:

• SERVICE_NAME
• SERVICE_NAMESPACE
• OTLP_ENDPOINT
• OTLP_FORMAT
• EXPORT_TYPE
• OTEL_RESOURCE_ATTRIBUTES
• OTEL_EXPORTER_OTLP_HEADERS

hipster's Issues

Unable to complete docker-compose build on Apple silicon Mac

Hi there.

Having come to build the Hipster shop using the docker instructions I have found the process was unable to complete. On Apple silicon / ARM64 devices - node-gyp is unable to detect Python on the existing build and fails building currency or payment services.

[4/5] RUN npm install:
#0 6.898
#0 6.898 > [email protected] install /app/node_modules/grpc
#0 6.898 > node-pre-gyp install --fallback-to-build --library=static_library
#0 6.898
#0 7.047 node-pre-gyp WARN Using request for node-pre-gyp https download
#0 7.404 node-pre-gyp WARN Tried to download(404): https://node-precompiled-binaries.grpc.io/grpc/v1.24.4/node-v72-linux-arm64-musl.tar.gz
#0 7.404 node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v72 ABI, musl) (falling back to source compile with node-gyp)
#0 7.554 gyp ERR! find Python
#0 7.554 gyp ERR! find Python Python is not set from command line or npm configuration
#0 7.554 gyp ERR! find Python Python is not set from environment variable PYTHON
#0 7.554 gyp ERR! find Python checking if "python" can be used
#0 7.554 gyp ERR! find Python - "python" is not in PATH or produced an error
#0 7.554 gyp ERR! find Python checking if "python2" can be used
#0 7.554 gyp ERR! find Python - "python2" is not in PATH or produced an error
#0 7.554 gyp ERR! find Python checking if "python3" can be used
#0 7.554 gyp ERR! find Python - "python3" is not in PATH or produced an error
#0 7.554 gyp ERR! find Python
#0 7.554 gyp ERR! find Python **********************************************************
#0 7.554 gyp ERR! find Python You need to install the latest version of Python.
#0 7.554 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
#0 7.554 gyp ERR! find Python you can try one of the following options:
#0 7.555 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
#0 7.555 gyp ERR! find Python (accepted by both node-gyp and npm)
#0 7.555 gyp ERR! find Python - Set the environment variable PYTHON
#0 7.555 gyp ERR! find Python - Set the npm configuration variable python:
#0 7.555 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
#0 7.555 gyp ERR! find Python For more information consult the documentation at:
#0 7.555 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
#0 7.555 gyp ERR! find Python **********************************************************
#0 7.555 gyp ERR! find Python
#0 7.555 gyp ERR! configure error
#0 7.555 gyp ERR! stack Error: Could not find any Python installation to use

No local install/path changes could impact this.

I so far have found a workaround and can now successfully run docker-compose build after updating the dockerfile in each impacted service. Adding a line before NPM install:
Dockerfile.txt

RUN apk add --update --no-cache python3 build-base gcc && ln -sf /usr/bin/python3 /usr/bin/python

(see attached example dockerfile.txt from checkoutservice)

Docker containers do not run correctly on Mac M1

Following getting docker-compose to work via workaround the containers presents three faults.

Cartservice:
Attaching to hipster-main-ad-1, hipster-main-cart-1, hipster-main-checkout-1, hipster-main-currency-1, hipster-main-email-1, hipster-main-frontend-1, hipster-main-lmotel-1, hipster-main-payment-1, hipster-main-productcatlog-1, hipster-main-recommended-1, hipster-main-shipping-1
hipster-main-cart-1 | Failed to resolve full path of the current executable [/proc/self/exe]
hipster-main-cart-1 exited with code 133

Emailservice:
2023-01-31 11:28:54 E0131 11:28:54.418165543 26 ssl_transport_security.cc:1446] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.

Adservice:
2023-01-31 11:50:31 Caused by: io.opentelemetry.sdk.autoconfigure.spi.ConfigurationException: OTLP endpoint must be a valid URL: "http://host.docker.internal:4318/v1/traces"
2023-01-31 11:50:31 Caused by: java.net.MalformedURLException: no protocol: "http://host.docker.internal:4318/v1/traces"

Checking the "/proc/self/exe" fault suggest a docker platform level issue hitting Mac silicon so compatibility needs to be confirmed.

As a workaround attempted forcing docker to use the linux/amd64 platform flag on cartservice which can build yet doesn't resolve the issue.

docker-compose.yml file

cart:
build: ./cartservice
platform: linux/amd64
ports:
- 80:80
extra_hosts:
- "host.docker.internal:host-gateway"