scalaconsultants/zio-scala2-quickstart.g8

A Giter8 template for a fully functional, ready to deploy ZIO-based microservice.

zio-scala2-quickstart

A Giter8 template for a fully functional, ready to deploy microservice (or monolith - it's up to you).

This template will result in Scala 2.13.x compatible code. For Scala 3 use zio-scala3-quickstart.

Out of the box you get a set of CRUD endpoints in the framework of you choice (currently ZIO HTTP or Akka HTTP) that integrate with a PostgreSQL database (currently Slick but more coming in the near future).

Other notable integrations include:

• Testcontainers for integration tests
• Flyway migrations
• sbt-native-packager for docker images
• scalafmt
• zio-json for JSON processing.
• zio-logging for logging.
• zio-config for typesafe configuration.
• zio-test for testing.

Setting up the project

sbt new ScalaConsultants/zio-scala2-quickstart.g8
# then follow interactive process to choose project name and other parameters

Run tests

cd <project-name>
sbt test

Launch and interact

sbt run

# create an item
curl --request POST \
  --url http://localhost:8080/items \
  --header 'content-type: application/json' \
  --data '{
	"name":"BigMac",
	"price": 10.0
}'

# get all items
curl --request GET \
  --url http://localhost:8080/items

Docker image

Template provides packaging as docker image out of the box, using sbt-native-packager plugin.

To create an image, run:

sbt docker:publishLocal

This will create a docker image locally, named the same as your project is. You can run it like the following:

docker run -d -p 8080:8080 --name=<project_name> <project_name>:0.1.0-SNAPSHOT

Main components

This sample app has several key components:

• ItemRepository, which is quite a regular slick repo, with the only twist - it works with ZIO and not Futures. Backed up by an H2 database and Hikari connection pool.
• Api - pretty standard akka-http CRUD endpoint, also powered by ZIO.
• interop - package, where all the integration magic is happening.
• ApplicationService - a more higher level service, that works with different error type and uses ZIO environment.
• Boot - wiring all the components together using ZLayer.
• ApiSpec - akka-http endpoint spec using zio-test.

Template license

Written in 2020 by Scalac Sp. z o.o.

To the extent possible under law, the author(s) have dedicated all copyright and related and neighboring rights to this template to the public domain worldwide. This template is distributed without any warranty. See http://creativecommons.org/publicdomain/zero/1.0/.

zio-scala2-quickstart.g8's Issues

Allow the server to start without the database

Server should be able to start without the database running and make scheduled retries until the database is available.
Also if the database is unavailable at some point when the application is running, it shouldn't stop the server, but retry connection.

Document the usage of ZLayer pattern

Developers not familiar with it could have a hard time understanding why services are defined in such way. Some documentation and references to original Zio pattern description won't hurt

Add docker configuration

It would be very useful if the generated app was docker ready.

"Delete" function in repo is doing more then just deleting, SlickRepository need refactor.

Using Delete in SlickRepository there are hidden runs of Event publishing which shouldn't be there. SlickRepository need to be agnostic of Events.

Remove doubled ActorSystem

We've got two ActorSystem delivered by actorSystemLayer and actorSystem layers.

Create an optional WebSocket endpoint

Flaky tests

Although all tests pass locally in all combinations of yes/no, some random CI checks are failing each time (e.g. (yes, yes, yes) passes but (yes, yes, no) fails).

Add optional GraphQL endpoint

Use Caliban

Timeout if using dockerized image

Repro:

(The following on a Mac.)

• a fresh instantiation of this template

  (which btw looks great, in general, thanks for putting it together.)

• No code/config changes whatsoever

• Run postgres in whatever way desired. Here, just using docker: docker run --name devdb -p 5432:5432 -e POSTGRES_DB=items -e POSTGRES_PASSWORD=12345 -d postgres:13

• All good with the service via sbt. In particular, output quickly shows:

[info] running zio_akka_quickstart.Boot
2021-10-23 16:50:01 INFO  com.zaxxer.hikari.HikariDataSource   - Starting...
2021-10-23 16:50:01 INFO  com.zaxxer.hikari.HikariDataSource   - Start completed.
2021-10-23 16:50:01 INFO  akka.event.slf4j.Slf4jLogger  Slf4jLogger started
Server online.
2021-10-23 16:50:02 INFO  o.f.c.i.license.VersionPrinter  Flyway Community Edition 7.14.0 by Redgate
2021-10-23 16:50:02 INFO  o.f.c.i.d.base.BaseDatabaseType  Database: jdbc:postgresql://localhost:5432/items (PostgreSQL 13.4)
2021-10-23 16:50:02 INFO  o.f.core.internal.command.DbValidate  Successfully validated 1 migration (execution time 00:00.020s)
2021-10-23 16:50:02 INFO  o.f.core.internal.command.DbMigrate  Current version of schema "public": 1
2021-10-23 16:50:02 INFO  o.f.core.internal.command.DbMigrate  Schema "public" is up to date. No migration necessary.
Flyway migration completed with: MigrateResult(7.14.0,items,List(),migrate,Some(1),None,None,List(),0)

and all requests are handled normally.

• however, when instead running the service using the sbt docker:publishLocal-generated image, things don't go as smoothly:

$ docker run --name devzio -p 8080:8080 zio-akka-quickstart:0.1.0-SNAPSHOT
2021-10-23 23:57:17 INFO  akka.event.slf4j.Slf4jLogger  Slf4jLogger started
2021-10-23 23:57:17 INFO  com.zaxxer.hikari.HikariDataSource   - Starting...
2021-10-23 23:57:17 INFO  com.zaxxer.hikari.HikariDataSource   - Start completed.
Server online.

• the immediate output doesn't show the Flyby and DB related lines as with sbt run (probably OK).

• Now, upon a request like curlie get http://localhost:8080/items, one can see, after several seconds, the following service output:

2021-10-23 23:58:28 INFO  akka.actor.ActorSystemImpl akka.actor.ActorSystemImpl(zio-akka-quickstart-system) Request timeout encountered for request [GET /items Empty]
2021-10-23 23:58:39 INFO  akka.actor.ActorSystemImpl akka.actor.ActorSystemImpl(zio-akka-quickstart-system) items: Response for
  Request : HttpRequest(HttpMethod(GET),http://localhost:8080/items,List(Timeout-Access: <function1>, Host, User-Agent: curl/7.71.1, Accept: application/json, */*),HttpEntity.Strict(none/none,0 bytes total),HttpProtocol(HTTP/1.1))
  Response: Complete(HttpResponse(500 Internal Server Error,List(),HttpEntity.Strict(none/none,0 bytes total),HttpProtocol(HTTP/1.1)))

• Interestingly, the GraphQL endpoint does continue to work.

Update Caliban to 0.7.7

Separate GET and HEAD healthchecks

GET and HEAD healthchecks should be separated. HEAD healthcheck should not access the database. Its purpose is to just answer as fast as it can with the least resource utilization with the main use case being AWS ECS healtcheck access.

Update ZIO-Config to 1.0.0-RC17

Remove the choice of library versions

When generating the project we have a choice of library versions to go with. It doesn't really make sense as the chosen library might be totally incompatible with the template.
Better way is to stay always consistent and remove the choice.
Also add https://github.com/fthomas/scala-steward to this project (as long as it's able to keep track not only the project's dependencies but also of the template's dependencies).

Add logger configuration file

Add DB connection pool

It's highly probable that the DatabaseProvider in conjuntion with ZIOSupport is creating new connection on every HTTP request. This needs to be confirmed.
If yes, then a solution would be to introduce a connection pool (probably HikariCP as suggested in Slick documentation).
Unless other ideas come up.

Add GraphQLApi test

Add logging

Use zio-logging

Add license

EventSubscriber is not working

Tests for Events are ignored because they were failing by timeout. Need investigation and fix. Probably problem with zio-akka-wrapper

Add ZLayer

Refactor to use ZLayer.

Add Async API definition for WebSocket and SSE endpoints

Add Async API documentation to the optional WebSocket and SSE endpoints.
Also add an endpoint serving generated UI (from Async API Generator)
https://www.asyncapi.com/

Add tests

Use zio-test.
Research might be needed to integrate zio-test and akka testkit.

Add option for Quill database access

We want users of the template to have a choice of database integration library. Right now it's only Slick. This ticket is about introducing Quill.

• add Quill implementation of ItemRepository (named QuillItemRepository) and HealthCheckService (named QuillHealCheckService) in the infrastucture module
• add in-memory implementation of ItemRepository (named InMemoryItemRepository) and HealthCheckService (named InMemoryHealCheckService) in the infrastucture module
• add a choice between Slick and Quill implementation when running the template (analogous to Akka HTTP vs ZIO HTTP choice)
• if the user answers "no" to both Slick and Quill question, then the in-memory implementation should be used

Add Testcontainers

Switch the template to PostgreSQL and Testcontainers for the integration tests.
We could follow instructions from https://hmemcpy.com/2020/08/running-postgres-integration-tests-easily-with-testcontainers-and-zio-test/ .

Add CORS

Need more detailed specification.

https://github.com/lomigmegard/akka-http-cors

win10 use zio-akka-quickstart.g8 template, report error!

Invalid path: src/main/g8/src/main/scala/$package$/domain/$if(add_caliban_endpoint.truthy || add_server_sent_events_endpoint.truthy || add_websocket_endpoint.truthy)$Subscriber.scala$endif$

Fix tests for caliban 0.8.1

Remove dependency on Console from GraphQLApi

Remove dependency on Console from GraphQLApi. If needed replace with proper logging.

Update to ZIO 2.0

use zio-json

Add explicit HikariCP configuration

Setup scala-steward

Check if it's possible for scala-steward to monitor project-inside-project as is the case with giter8 templates.
https://github.com/fthomas/scala-steward

Add zio-shield

add plugin zio-shield + fix issues found

Add configuration

Use zio-config

Add CI build

Since now the project is opened and should be considered opensource - we can setup circle ci or something similar for free

Possibly serious performance issues

This line

      zio.Runtime.default.unsafeRun(routeWithDependencies)

in src/main/scala/com/kzs/lab/ws/package.scala (line 15) blocks the theard until the result is ready and can seriously affect performance

Update ZIO to 1.0.0-RC20

Add tests for WS and SSE endpoints

Optional endpoints were added without any test coverage

Add Open API definition for the REST endpoints

Refactor integration tests

Change integration tests to cover only the repository layer (not the API).

That means removing ItApiSpec and introducing ItemRepositorySpec.

Graphql package is always done, no matter whether user wanted.

in main api Graphql directory is done without using of gitter8

Modify the response to POST /items

• make it respond with 200 and an item object in the response body.
• remove the Location header - it needs more though and it'll probably break in a docker and/or proxy scenarios

Remove optional Id from Item

Figure out a better way to model Item's states without Option[ItemId].

Create docker compose for postgress

It would be nice to have ready postgres by docker compose

Integrating GRPC to routes

Currently, as part of this quickstart, I could find REST(Akka), Graphql(Caliban).

Any thoughts on integrating Akka-GRPC or ZIO-GRPC?

Split into multi-project

Split the application into three sub-projects:

1. api (packages: api, config; also object Boot)
2. domain (packages: application, domain)
3. Infrastructure (packages: infrastructure)

Domain should not depend on other projects.
Infrastructure should depend on domain.
Api should depend on domain and infrastructure.

Documentation

For this template to be useful we need a clear Readme.
Let's gather here what type of information should we put there.

Add authentication

Having out of the box authentication would be useful. The question is what should we support and how much effort will the maintenance take? Are there any abstraction we could use?

Create an optional Server-Sent Event endpoint

Parametrize application.conf variables