Giter Site home page Giter Site logo

orlangure / gnomock Goto Github PK

View Code? Open in Web Editor NEW
1.3K 14.0 64.0 1.39 MB

Test your code without writing mocks with ephemeral Docker containers ๐Ÿ“ฆ Setup popular services with just a couple lines of code โฑ๏ธ No bash, no yaml, only code ๐Ÿ’ป

License: MIT License

Go 99.76% Dockerfile 0.24%
go golang testing integration-testing end-to-end-testing docker kafka postgres mysql mongo rabbitmq splunk sql-server redis memcached elasticsearch mariadb kubernetes cockroachdb hacktoberfest

gnomock's Introduction

Gnomock โ€“ tests without mocks

๐Ÿ—๏ธ Spin up entire dependency stack

๐ŸŽ Setup initial dependency state โ€“ easily!

๐Ÿญ Test against actual, close to production software

โณ Spend no time writing mocks

๐Ÿ•น๏ธ Test actual program behavior and side effects

PkgGoDev Test Go Report Card codecov

Gnomock is an integration and end-to-end testing toolkit. It uses Docker to create temporary containers for application dependencies, setup their initial state and clean them up in the end. Gnomock allows to test the code with no mocks wherever possible.

The power of Gnomock is in a variety of Presets, each implementing a specific database, service or other tools. Each preset provides ways of setting up its initial state as easily as possible: SQL schema creation, test data upload into S3, sending test events to Splunk, etc.

The name "Gnomock" stands for "no mock", with a "G" for "Go" ๐Ÿ˜ผ. It also sounds like "gnome", that's why the friendly garden gnome artwork (by Michael Zolotov)

Demo

See for yourself how easy and fast it is to write tests that use actual services running in ephemeral Docker containers:

asciicast

Table of contents

Getting started

Gnomock can be used in two different ways:

  • Imported directly as a package in any Go project
  • Accessed over HTTP running as a daemon in any other language

Both ways require an active Docker daemon running locally in the same environment.

External DOCKER_HOST support is experimental. It cannot be reliably tested at this moment, but it might work.

Using Gnomock in Go applications

See the following example to get started:

go get github.com/orlangure/gnomock

Setting up a Postgres container with schema setup example:

package db_test

import (
	"database/sql"
	"fmt"
	"testing"

	_ "github.com/lib/pq" // postgres driver
	"github.com/orlangure/gnomock"
	"github.com/orlangure/gnomock/preset/postgres"
)

func TestDB(t *testing.T) {
	p := postgres.Preset(
		postgres.WithUser("gnomock", "gnomick"),
		postgres.WithDatabase("mydb"),
		postgres.WithQueriesFile("/var/project/db/schema.sql"),
	)
	container, _ := gnomock.Start(p)
	t.Cleanup(func() { _ = gnomock.Stop(container) })

	connStr := fmt.Sprintf(
		"host=%s port=%d user=%s password=%s  dbname=%s sslmode=disable",
		container.Host, container.DefaultPort(),
		"gnomock", "gnomick", "mydb",
	)
	db, _ := sql.Open("postgres", connStr)
	// db has the required schema and data, and is ready to use
}

See package reference. For Preset documentation, refer to Presets section.

Using Gnomock in other languages

If you use Go, please refer to Using Gnomock in Go applications section. Otherwise, refer to documentation.

Official presets

The power of Gnomock is in the Presets. Existing Presets with their supported* versions are listed below.

* Supported versions are tested as part of CI pipeline. Other versions might work as well.

Preset Go package Go API Supported versions arm64
Localstack (AWS) Go package Reference 0.12.2, 0.13.1, 0.14.0 โœ…
Splunk Go package Reference 8.0.2 โŒ
Redis Go package Reference 5.0.10, 6.0.9 โœ…
Memcached Go package Reference 1.6.9 โœ…
MySQL Go package Reference 5.7.32, 8.0.22 โœ…
MariaDB Go package Reference 10.5.8 โœ…
PostgreSQL Go package Reference 10.15, 11.10, 12.5, 13.1 โœ…
Microsoft SQL Server Go package Reference 2017-latest, 2019-latest โŒ
MongoDB Go package Reference 3.6.21, 4.4 โœ…
RabbitMQ Go package Reference 3.8.9-alpine, 3.8.9-management-alpine โœ…
Kafka Go package Reference 3.3.1-L0 โœ…
Elasticsearch Go package Reference 8.7.0, 7.17.9 โœ…
Kubernetes Go package Reference v1.26.3-k3s1 โœ…
CockroachDB Go package Reference v19.2.11, v20.1.10 โœ…
InfluxDB Go package Reference 2.0.4-alpine โœ…
Cassandra Go package Reference 4.0, 3 โœ…
Vault Go package Reference 1.10.11, 1.11.8, 1.12.4, 1.13 โœ…
Azurite Go package Reference 3.22.0 โœ…

It is possible to use Gnomock directly from Go code without any presets. HTTP API only allows to setup containers using presets that exist in this repository.

Similar projects

Gnomock is not the only project that aims to simplify integration and end-to-end testing by using ephemeral docker containers:

  • testcontainers/testcontainers-go
  • ory/dockertest

These projects are amazing, and they give plenty of flexibility and power to their users. There are many things that are possible with them, but are impossible with Gnomock. Still, below is a short list of things that sometimes give Gnomock an advantage:

  • Gnomock tries to provide a batteries-included solution. Gnomock has a growing number of Presets, each one implementing an integration with a popular external service. For every Preset, there already is a number of "invisible" utilities that transparently relieve you from implementing them yourself:
    • Built-in health check function that you don't even need to know it exists. It makes sure you only get control over a container when it is ready to use.
    • Wrappers for some of the configuration exposed by the container, such as default username/password. You can easily provide your own credentials to connect to the container.
    • Seed data ingestion for your convenience. Sometimes you just need to make sure your queries work given some data. Gnomock puts your data in there with a single line of code. Sometimes you only test a program that consumes messages from Kafka, and Gnomock produces the messages for you with another line of code.
  • Simple API that does not expose anything that happens "under the hood" most of the time. Yet Gnomock allows some additional configuration and custom Preset implementation whenever necessary.
  • Gnomock's vision includes being useful not only in Go projects, but in any projects via HTTP. It already supports almost all its features over HTTP layer and has a clear OpenAPI spec.
  • Gnomock has a friendly garden gnome mascot๐Ÿ˜ป

Troubleshooting

Tests with Gnomock take too long and time-out eventually

It happens a lot locally if your internet isn't fast enough to pull docker images used in tests. In CI, such as in Github Actions, the images are downloaded very quickly. To work around this issue locally, pull the image manually before running the tests. You only need to do it once, the images stay in local cache until deleted. For example, to pull Postgres 11 image, run:

docker pull postgres:11

Tests time-out even when the image exists locally

It can happen if the containers can't become ready to use before they time out. By default, Gnomock uses fairly high timeouts for new containers (for starting and for setting them up). If you choose to change default timeout using WithTimeout (timeout in HTTP), it is possible that the values you choose are too short.

Tests pass when run one-by-one, and fail when run in parallel

It happens when you try to start up a lot of containers at the same time. The system, especially in CI environments such as Github Actions, cannot handle the load, and containers fail to become healthy before they time-out. That's the reason Gnomock has a few separate build jobs, each running only a small subset of tests, one package at a time.

Containers fail to setup with a "File not found" error

If you run gnomock as a server, you need to make sure the files you use in your setup are available inside gnomock container. Use -v $(pwd):$(pwd) argument to docker run to mount the current working directory under the same path inside the gnomock container. If you prefer to keep a permanent gnomock container running, you can mount your entire $HOME directory (or any other directory where you keep the code).

Giving back

This is a free and open source project that hopefully helps its users, at least a little. Even though I don't need donations to support it, I understand that there are people that wish to give back anyway. If you are one of them, I encourage you to plant some trees with Tree Nation ๐ŸŒฒ ๐ŸŒณ ๐ŸŒด

If you want me to know about your contribution, make sure to use [email protected] as the recipient email.

Thank you!

gnomock's People

Contributors

0b01001110 avatar aeolus3000 avatar brurucy avatar dependabot[bot] avatar devuo avatar dusansimic avatar haegi avatar jayvdb avatar juneezee avatar marvinzeising avatar orlangure avatar pawndev avatar rajatvig avatar rohitkg98 avatar shado1111w avatar u5surf avatar vjftw avatar zbindenren avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gnomock's Issues

Bug: Docker Daemon and Client version mismatch

Describe the bug
Docker client does not negotiate API version. In machines with client version mismatch, one has to explicitly set DOCKER_API_VERSION

To Reproduce

  • Try gnomock with different Docker Daemon and Client versions.

Expected behavior
Should negotiate the version and automatically set it.

System (please complete the following information):

  • OS: Linux, Ubuntu, Pop OS 20.10
  • Version: v0.14.1
  • Docker version: 19.03.6

Unable to start image stored in private Docker repo

Describe the bug

I am trying to bring up a custom container using gnomock.StartContainer(). The image is hosted on Google Container Registry (gcr.io), which requires authentication.

The container fails to start with the error:

can't start container: can't pull image: can't pull image: Error response from daemon: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication

docker pull for the same image works just fine however:

$ docker pull gcr.io/nadt-184401/bnet-fake-apiserver-0.1.5-2021-06-08
Using default tag: latest
latest: Pulling from nadt-184401/bnet-fake-apiserver-0.1.5-2021-06-08
5dea5ec2316d: Pull complete
cc6ee50445b3: Pull complete
Digest: sha256:3d16cbb203efac08a83022f2517e89e124c3425c62db1905dd9dc2f2721f9ec3
Status: Downloaded newer image for gcr.io/nadt-184401/bnet-fake-apiserver-0.1.5-2021-06-08:latest
gcr.io/nadt-184401/bnet-fake-apiserver-0.1.5-2021-06-08:latest

To Reproduce

func TestGetCharacter(t *testing.T) {
	ctx := context.Background()

	// Spin up a fake for the Battle.Net API server
	bn, err := gnomock.StartCustom(
		"gcr.io/nadt-184401/bnet-fake-apiserver-0.1.5-2021-06-08",
		map[string]gnomock.Port{
			"http": {Protocol: "TCP", Port: 8080},
		},
		gnomock.WithDebugMode(),
	)
	if err != nil {
		t.Fatalf("starting gnomock bnet api fake: %v", err)
	}
	defer gnomock.Stop(bn)
}

Expected behavior

If docker pull works for a given image, gnomock should also be able to start that image.

System (please complete the following information):

  • OS: Ubuntu
  • Version 20.04
  • Docker version 20.10.6 (CE)

Additional context

The root cause is likely that Gnomock passes a fixed, empty set of pull options to the Docker client. If there were an way to pass ImagePullOptions to .StartCustom, then I could probably get around this by base64 encoding the required config.

Ideally, it would be nice if the docker client just used my ~/docker/config.json and "did the right thing", but that doesn't seem to be part of the docker client package. Another option would be to allow the HTTP client to be passed via a Gnomock option, but that's just asking the user to manage the crafting of the X-Registry-Auth header (which is what passing an ImagePullOptions struct with a populated RegistryAuth does for you.)

Bug: gnomock-cleaner not work well in M1(arm64)

Describe the bug
StartCustom cleaner image maybe(random) hang at the M1 system.
docker logs print this

no specific platform was requested
qemu: uncaught target signal 11 (Segmentation fault) - core dumped

To Reproduce
Start any preset in M1

Expected behavior
work well

System (please complete the following information):

  • OS: MacOS
  • Version 12.2
  • Docker version 20.10.7

Please add arm64 platform docker images in the dockerhub

Configure golangci-lint to skip some linters in test files

Right now, there is a bunch of // nolint:funlen or // nolint:bodyclose comments in various tests. These comments tell golangci-lint to exclude the marked tests from linter reports.

Instead of marking each test file separately, there is a configuration of golangci-lint that we can set once in .golangci-lint.yaml. This configuration should apply to all test files.

Linters that should be disabled for test files:

  • bodyclose
  • funlen
  • gosec

Request: More options for MySQL preset

Hey,

I see awesome MySQL preset but at case in my company there are still missing some features. We need three things:

  • Run in root mode.
  • Don't choose database at start because we have multiple db's in single MySQL instance.
  • Mount Docker volumes. We use internal cache for seeds and it's saved in Docker volume.

Here's a quick sneak peek what I'm talking about in code:
https://github.com/orlangure/gnomock/compare/master...zdunecki:new-options-for-mysql-preset?expand=1

If you like this idea I'm able to finish this PR and add more tests, comments etc.

Fixed container names

Allow using a fixed container name, either failing when a container already exists, or killing the existing container before starting.

CockroachDB preset

CockroachDB is another database that would be great to support.

I suggest to discuss the requirement here before starting the work.

Request: Implement Postgis Preset

PostGIS is awesome and it would be great to have a preset for it, I've started one already that i'm using that works well enough for me but its probably not ready to be merged. I can submit a PR if you want to review.

Allow multiple WithQueriesFile options in presets

Sometimes tests will require multiple sql files: one for schema, and one for data. Or maybe more. Currently each WithQueriesFile call replaces existing files. It should add files to the list instead.

Question: using gnomock in CI/CD environment with go applications

How would one use gnomocks go interface in a ci/cd environment without using a docker in docker solution as many have pointed out the pitfalls of doing something like docker in docker
I'm aware that you can also use gnomock as a server for creating presets and starting them, but theres not much documentation on if those containers could be side car along side of a docker image running the tests and exposed via the docker network.

Allow to override Preset options

When using gnomock.StartPreset(preset), there is no way to override options set by preset. Sometimes it might be useful (maybe not now, but in future). Signature should be changed to accept gnomock.Option functions in the end.

Pulsar preset

This is a placeholder for Apache Pulsar preset.

Minimal version should be very straightforward since there appears to be a ready-to-use docker image:
Running in docker

There should be initial state setup available similar to rabbitmq and kafka presets.

Installation script for MacOS

gnomockd should start automatically, for that it needs to have its own launch daemon. To allow easier setup, a script that downloads and configures everything could help.

Implement DynamoDB preset

A short guide on implementing a new preset:
https://github.com/orlangure/gnomock/blob/master/CONTRIBUTING.md#new-presets

MySQL preset for inspiration:
https://github.com/orlangure/gnomock/tree/master/preset/mysql

The objective is to implement a new Preset that spins up DynamoDB docker container, sets up its initial state (user provided data), waits for it to become ready and kills it in the end.

Please note that there are multiple steps required to create a new preset (see the guide above).
I'm here to help๐Ÿ˜ผ

Add artifact upload on release

Use github actions to build and upload gnomockd binary for every release. Initially macos and linux binaries should be supported at minimum.

Remove container on start/init failure

Currently when a container is started, but then fails to become available, this container is returned to the caller alongside the error that occurred somewhere in the process. It requires the caller to always stop the container, even in case of errors, which looks and feels weird. On failure, containers should be stopped in case of errors.

Elasticsearch preset

This is a placeholder for Elasticsearch Preset issue that I am now working on.

Merge all timeouts into one

For simplicity, one timeout should be used when setting up Gnomock. Users should not care whether image pull timed out, or healthcheck, or init.

Bug: Gnomock is not working with a remote docker host

Describe the bug
When starting the docker containers on a remote host (e.g. by setting DOCKER_HOST env var), the containers are starting just fine, but the host field of the container object is still set to localhost which leads to failing healthcheck

To Reproduce

  • Set DOCKER_HOST to a valid remote host
  • Run tests

Expected behavior
The container field host should be set to the remote host from DOCKER_HOST env variable.
An alternative for setting the DOCKER_HOST env variable could be adding the option to set a remote host.

Screenshots
If applicable, add screenshots to help explain your problem.

System (please complete the following information):

  • OS: [e.g. MacOS] MacOS
  • Version [e.g. 0.9.2] 0.12.1
  • Docker version 20.10.5

Additional context
Add any other context about the problem here.

Attach docker output on failed container starts

When a container fails to start, no output is returned to the user, which makes it very hard to debug what is wrong. We should add any information we have from docker to the error message.

Print container logs on failure

When containers go up, but fail to become available for different reasons, the user will have no way to understand what went wrong. In such cases I suggest to include container logs in the output.

Implement Cassandra preset

A short guide on implementing a new preset:
https://github.com/orlangure/gnomock/blob/master/CONTRIBUTING.md#new-presets

MySQL preset for inspiration:
https://github.com/orlangure/gnomock/tree/master/preset/mysql

The objective is to implement a new Preset that spins up Cassandra docker container, sets up its initial state (user provided data), waits for it to become ready and kills it in the end.

Please note that there are multiple steps required to create a new preset (see the guide above).
I'm here to help ๐Ÿ˜ผ

Improve CircleCI builds (align with Github Actions)

CircleCI builds: https://app.circleci.com/pipelines/github/orlangure/gnomock
CircleCI workflow configuration: https://github.com/orlangure/gnomock/blob/master/.circleci/config.yml

Github Actions configuration: https://github.com/orlangure/gnomock/blob/master/.github/workflows/test.yaml

Github Actions is the primary CI platform of this project. But many use CircleCI, and up until now Gnomock didn't work (at least sometimes) there.

Since we used Github Actions for project builds, we kind of constantly verified that Gnomock worked in this environment. So, to ensure that Gnomock also works in CircleCI, the best way would to have similar workflows in all supported platforms.

The goal of this task is to have all presets, core functionality, and SDKs (currently only python) tested in CircleCI builds, just like we have in Github Actions. The separation of jobs does not have to be the same, but it would be better if builds were relatively fast.

Create a Github Action for gnomockd

This is a placeholder task for the future implementation of gnomockd github action. This action should make it easy to run gnomockd for users in languages other than go.

Add Memcached preset

I've wanted a Memcached (a simple Redis alternative) preset so I've implemented one and only then I realized I'm actually supposed to write a feature request ๐Ÿ˜…. Here is the PR for the preset. This preset is based on Redis one since it has most of the same operations.

Allow to import messages from file/directly in RabbitMQ Preset

RabbitMQ Preset is missing a key feature: automated and simplified message ingestion upon startup. See Kafka preset for example:

// WithMessages makes sure that these messages can be consumed during the test
// once the container is ready.
func WithMessages(messages ...Message) Option {
	return func(o *P) {
		o.Messages = append(o.Messages, messages...)
	}
}

// WithMessagesFile allows to load messages to be sent into Kafka from one or
// multiple files.
func WithMessagesFile(files string) Option {
	return func(o *P) {
		o.MessagesFiles = append(o.MessagesFiles, files)
	}
}

Implementing this feature should allow users to provide a text file with messages in JSON format, or specify message objects directly using WithMessages option. When RabbitMQ container is ready, these messages should be already ready to consume.

In this issue, an InitFunc should be added to preset code. This function should be applied only when there are messages or message files, just like in Kafka.

Just like with any other preset development, please try to cover all the cases mentioned in contribution guide.

Implement InfluxDB preset

A short guide on implementing a new preset:
https://github.com/orlangure/gnomock/blob/master/CONTRIBUTING.md#new-presets

MySQL preset for inspiration:
https://github.com/orlangure/gnomock/tree/master/preset/mysql

The objective is to implement a new Preset that spins up InfluxDB docker container, sets up its initial state (user provided data), waits for it to become ready and kills it in the end.

Please note that there are multiple steps required to create a new preset (see the guide above).
I'm here to help ๐Ÿ˜ผ

Bug: Localstack container doesn't stop

Describe the bug
gnomock.Stop(c) does nothing

To Reproduce
I use https://github.com/onsi/ginkgo to write tests and faced an issue with active container even after stop was called

package main

import (
	"fmt"
	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/credentials"
	. "github.com/onsi/ginkgo"
	"github.com/orlangure/gnomock"
	"github.com/orlangure/gnomock/preset/localstack"
	"log"
)

var lsc *gnomock.Container
var config *aws.Config

var _ = SynchronizedBeforeSuite(func() []byte {
	lsp := localstack.Preset(localstack.WithServices(localstack.DynamoDB))
	lsc, err := gnomock.Start(lsp, gnomock.WithLogWriter(log.Writer()))
	if err != nil {
		log.Fatalln(err)
	}

	endpoint := fmt.Sprintf("http://%s/", lsc.Address(localstack.APIPort))
	return []byte(endpoint)
}, func(endpoint []byte) {
	log.Println("Localstack runs on:", string(endpoint))
	config = &aws.Config{
		Region:           aws.String("us-east-1"),
		Endpoint:         aws.String(string(endpoint)),
		S3ForcePathStyle: aws.Bool(true),
		Credentials:      credentials.NewStaticCredentials("a", "b", "c"),
	}
})

var _ = SynchronizedAfterSuite(func() {
	// all nodes - nothing
}, func() {
	log.Println("shutdown localstack")
	err := gnomock.Stop(lsc)
	if err != nil {
		log.Println(err)
	}
})

var _ = Describe("Status", func() {
	It("should return something", func() {
	})
})

Expected behavior
Container should be gone

Screenshots
If applicable, add screenshots to help explain your problem.

When I execute the test, it gives me the log:

=== RUN   TestStatus
Running Suite: Receipt Suite
============================
Random Seed: 1608770795
Will run 1 of 1 specs

Waiting for all LocalStack services to be ready
2020-12-24 00:46:38,248 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.
2020-12-24 00:46:38,252 INFO supervisord started with pid 17
2020-12-24 00:46:39,256 INFO spawned: 'dashboard' with pid 23
2020-12-24 00:46:39,260 INFO spawned: 'infra' with pid 24
2020-12-24 00:46:39,266 INFO success: dashboard entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2020-12-24 00:46:39,266 INFO exited: dashboard (exit status 0; expected)
(. .venv/bin/activate; exec bin/localstack start --host)
Starting local dev environment. CTRL-C to quit.
LocalStack version: 0.12.2
2020-12-24 00:46:40,751 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Waiting for all LocalStack services to be ready
Starting edge router (https port 4566)...
Starting mock DynamoDB service on http port 4566 ...
Starting mock DynamoDB Streams service on http port 4566 ...
Starting mock Kinesis service on http port 4566 ...
[2020-12-24 00:46:45 +0000] [25] [INFO] Running on https://0.0.0.0:4566 (CTRL + C to quit)
2020-12-24T00:46:45:INFO:hypercorn.error: Running on https://0.0.0.0:4566 (CTRL + C to quit)
2020/12/24 01:46:50 Localstack runs on: http://127.0.0.1:55009/
โ€ข2020/12/24 01:46:50 shutdown localstack

Ran 1 of 1 Specs in 14.902 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestStatus (14.90s)
PASS

you can notice shutdown localstack, which is right before the stop, however docker ps gives me that container as an active one. Debug mode doesn't give any insights on what's happening during the stop.

System (please complete the following information):

  • OS: MacOS
  • Version v0.10.1
  • Docker version
$ docker version
Client: Docker Engine - Community
Version:           20.10.0
API version:       1.41
Go version:        go1.13.15
Git commit:        7287ab3
Built:             Tue Dec  8 18:55:43 2020
OS/Arch:           darwin/amd64
Context:           default
Experimental:      true

Server: Docker Engine - Community
Engine:
 Version:          20.10.0
 API version:      1.41 (minimum version 1.12)
 Go version:       go1.13.15
 Git commit:       eeddea2
 Built:            Tue Dec  8 18:58:04 2020
 OS/Arch:          linux/amd64
 Experimental:     true
containerd:
 Version:          v1.4.3
 GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
 Version:          1.0.0-rc92
 GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
 Version:          0.19.0
 GitCommit:        de40ad0

Additional context
Add any other context about the problem here.

Since you aware about testcontainers-go, did you consider their ryuk container? That's a nice approach to kill stale containers no matter what.

p.s. Thanks for the lib, this presets api looks really nice

Yugabyte preset

Yugabyte is another database that would be great to support.

I suggest to discuss the requirement here before starting the work.

A preset generator

A preset generator is needed to help developers develop any customized presets, e.g., REST Api client/server, grpc client/server and so on. A generalized interface could be good enough.

Add debug mode

Currently the only debug feature allows to print container logs to stdout. This is not enough. We need a way to print configuration used to setup every container, and to publish updates on every step in the pipeline. It might require implementing a separate log package, and/or accepting a logger from the outside.

Implement MariaDB preset

A short guide on implementing a new preset:
https://github.com/orlangure/gnomock/blob/master/CONTRIBUTING.md#new-presets

MySQL preset for inspiration:
https://github.com/orlangure/gnomock/tree/master/preset/mysql

The objective is to implement a new Preset that spins up MariaDB docker container, sets up its initial state (tables, data), waits for it to become ready and kills it in the end.

Please note that there are multiple steps required to create a new preset (see the guide above).
I'm here to help๐Ÿ˜ผ

Support Elasticsearch v6 in Elasticsearch preset

Until now I only tried to run Elasticsearch v7 in all tests, and elasticsearch client v7 is used for health checks and initialization. To allow testing earlier versions of elasticsearch, we should seamlessly support v6 images, and make needed adjustments transparently.

There is a chance that the code will work with elasticsearch v as-is. Or maybe we will only need to use a different package version (the client has v6 and v7 packages). There is also a chance that some setup functions or a health check implementation won't work on v6.

In the end, a user should be able to use v6 and v7 versions (WithVersion) and still get a working and ready-to-use container in the end.

Bug: can't connect to container: canceled after error: %!w(<nil>)

Describe the bug
Gnomock fails to connect to containers. I've forked this repo and ran the tests in it and I receive the same error as my own tests.
I suspect it has something to do with docker but i'm not sure what, these tests were working before i updated to the newest docker (prior to that I was a few major versions behind).
To Reproduce
For me, and another colleague just running any tests that call gnomock.Start() causes the error.

Screenshots
Here are some scerens of attempting to run the postgres preset tests, they take forever but eventually they do fail with the following error:
=== RUN TestPreset/10.15
preset_test.go:37:
Error Trace: preset_test.go:37
Error: Received unexpected error:
can't connect to container: canceled after error: %!w()
Test: TestPreset/10.15
--- FAIL: TestPreset/10.15 (312.93s)

image

System (please complete the following information):

  • OS: [e.g. MacOS]
    Windows 10 Professional
  • Version [e.g. 0.9.2]
	github.com/orlangure/gnomock v0.12.0
  • Docker version
Client: Docker Engine - Community
Cloud integration: 1.0.9
Version:           20.10.5
API version:       1.41
Go version:        go1.13.15
Git commit:        55c4c88
Built:             Tue Mar  2 20:14:53 2021
OS/Arch:           windows/amd64
Context:           default
Experimental:      true

Server: Docker Engine - Community
Engine:
 Version:          20.10.5
 API version:      1.41 (minimum version 1.12)
 Go version:       go1.13.15
 Git commit:       363e9a8
 Built:            Tue Mar  2 20:15:47 2021
 OS/Arch:          linux/amd64
 Experimental:     false
containerd:
 Version:          1.4.3
 GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
 Version:          1.0.0-rc92
 GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
 Version:          0.19.0
 GitCommit:        de40ad0

Implement basic authentication in gnomockd

Currently any code running alongside gnomockd can access its API using unprotected HTTP endpoints. In theory, javascript running in browser can also try to access localhost and start a bunch of containers. To avoid this, at least some basic authentication should be added to gnomockd.

Bug: DOCKER_HOST "unix:///run/user/1000/docker.sock" does not work

Describe the bug
I have a rootless docker installation on localhost. I set env variable DOCKER_HOST to "unix:///run/user/1000/docker.sock". Docker work and gnomock does not. If I'm not mistaken, this feature worked when the docker version was old in go.mod.

To Reproduce
Set environment variable DOCKER_HOST="unix:///run/user/1000/docker.sock" and run anything using gnomock.

Expected behavior
The file /run/user/1000/docker.sock will be used for docker sock instead of /var/run/docker.sock.

System:

  • OS: GNU/Linux amd64.
  • Docker version: 20.10.3.

Additional context
I have seen the problem in code: with remote DOCKER_HOST support, the hostname is processed and the path is lost. I can open PR to fix.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.