azavea/azavea.g8

A Giter8 template for bootstrapping Scala projects at Azavea.

Scala Template for Projects

A slightly opinionated Giter8 template for a scala project. By default, it sets up a project with the following features:

• Uses sbt for builds, tests, and running the server
• tapir API backed by http4s and doobie
• Easy integration with geotrellis-server

Usage

1. Ensure java (>= 8) is installed, if you have trouble with this try using jabba
2. Install sbt (version >= 1.x)
3. Install docker
4. From a terminal shell Run sbt new azavea/azavea.g8
5. Change directory into new project, run ./scripts/setup && ./scripts/server
6. curl http://localhost:8080/api/users/

azavea.g8's Issues

Use testcontainers for test running

We've run into issues in projects based on this template with port conflicts in parallel CI in a Jenkins environment (see raster-foundry/granary#45). We can use testcontainers instead and not have to care about which CI environment we end up in

Add Documentation Site + Examples

Fix what's going on with error logging

Server output / migration output in both the previous bloop setup and the current as of #54 sbt setup is logged at the error level (see comments starting here). We can maybe fix this with an outputStrategy

Provide more useful error messages for bad requests

See the example in azavea/franklin#454. It may be worthwhile to provide json that's readable to machines or maybe the prose in that PR is fine.

Add support to wait for database to be available before running migrate

When using the update script, check to ensure the database is available before attempting to execute migrate.

Upgrade to Scala 2.12.12

By default, the application project uses Scala 2.12.11. The latest scala 2.12 is 2.12.12. It's valuable to stay current because it makes our eventual upgrade path to 2.13 / Scala 3 easier (with 2.13 as a bridge that will include many backports).

What breaks in 2.12.12 is plugin versions mainly.

Provide template for a CLI-only application

Many applications we write are CLI tools and the abstractions provided by https://github.com/azavea/azavea.g8/blob/master/src/main/g8/application/src/main/scala/%24package__packaged%24/api/commands/Commands.scala are useful for those types of applications as well.

It would be awesome to have one of two things:

1. A separate g8 template that configures a CLI app without having to remove all the extra stuff from this one.
2. A mode in this g8 template that does the same as (1)

Move Doobie helpers to Separate Library

Add tree output to CI logs

Based on this feedback in #26, it would be useful to get a visualization of the scaffolded project structure as part of CI. This could simplify debugging file inclusion / exclusion.

We could use tree to output the project directory structure that is created as part of our CI tests. There would be no assertions against this structure, but having it easily accessible in build output could be useful when tracking down discrepancies.

tree is not installed in the GitHub Actions Ubuntu 18.04 runner image, but we could install it ourselves as part of the CI workflow.

Consider alternative method of accepting application configuration

Currently, it appears as though we're making use of decline to accept command line arguments to configure the application (e.g., --db-port). For target environments like Amazon ECS, we've historically supplied application configuration via environment variables (or overwritten settings defaults from a application.conf file via the environment). Maintaining support for that pattern is valuable because it has worked well, aligns with other project types, and allows for additional task definition flexibility.

Add Mechanism for Populating Secrets in Development

In most of our projects we have secrets or other configuration parameters that we do not wish to put in our repositories. In those projects we usually pull down these configurations from S3 and include them as the environment for the processes that need them.

We should add boilerplate that pulls down configuration from S3 given a bucket (could even be a parameter during project setup).

Simplify build and test scripts

At this point bloop is used in the CI. However, it looks like it is more a user development part of the project.
CI can be more agnostic and rely on SBT only. Moreover, SBT 1.4 uses BSP as an alternative to Bloop by default.

How do you feel about making CI only with SBT and the default local dev worflow will be pretty much default in this case?

$ sbt
$ sbt> ;compile; test; etc...

P.S.

Metals usage does not require an explicit bloop usage. It is used and added by default into the project during the project import into metals via VSCode.

metals.sbt generated during such import is usually ingored and not a part of the repository.

Add SortImports to Scalafix Template

See: https://github.com/NeQuissimus/sort-imports

Add support for running tests w/ database interactions in parallel

Support parallel test suite invocations that interact with a database by avoiding the host binding PostgreSQL port 5432. This is particularly noticeable in CI environments that share an instance of Docker.

See: https://github.com/azavea/alimentiv-video-annotation/issues/33

Use OrganizeImports scalafix action for organizing imports

This scalafix plugin adds configurable import organization that works very nicely (including removing unused!): https://github.com/liancheng/scalafix-organize-imports

Also, it works with Metals, so it has a nice integration with VSCode, vim, and emacs: https://scalameta.org/metals/docs/editors/overview.html#editor-support

add sbt welcome message

This seems like a great addition to project templates:

https://github.com/reibitto/sbt-welcome

The default welcome message should explain the template's default functionality and how to change the welcome message

Add support for configuring Scala Steward in private repositories

We've setup Scala Steward in a few projects, but always using an approach that requires the target repository to be public. For this issue, investigate solutions like scala-steward-org/scala-steward-action to keep Scala dependencies up-to-date for private projects.

(There may even be a way to centrally manage all Scala Steward target repositories.)

Add database testing helpers

We use these a lot on every project where we need to run integration tests against the database. We should include this here as well.

Remove tile server

Lots of things don't need a tile server, and the tile server adds a bunch of stuff that would be baffling if you didn't know what you were getting yourself into with an http4s application. It's not essential, so we should remove it to make the example services more comprehensible.

Include examples of ADT error-handling in project template

Right now it's up to users to figure out from tapir docs how to specify error outputs. This is a really common thing to do, so at least one endpoint should have an example of how it works.

Add progress output to Flyway migration process

I haven't tested a run where there are outstanding migrations to apply, but in scenarios where all outstanding migrations have been applied by Flyway, there appears to be no output to STDOUT. Having output in this scenario would be helpful to build confidence that the process completed successfully.

Add CI examples Template

• Jenkinsfile
• CircleCI
• Github Actions