checkr/openmock

Mock all the services. Intuitive YAML DSL for HTTP, gRPC, Kafka, and AMQP mocks.

openmock's Issues

Add yaml schema docs

Admin endpoint for mocks creation should return 400 if the input is not valid

I think a few functions may need to bubble up the errors so that we can prevent invalid input early on. Incidents like an invalid yaml input could potentially crash the openmock server and may need manual interaction from the redis server.

openmock/pkg/admin/convert.go

Line 62 in 68fadf5

func swaggerToOpenmockMock(swagger model.Mock) (om openmock.Mock) {

panic error when mocking http request > kafka topic

Hi there,

When I try to mock kafka publising to a topic based on a an HTTP GET request, I get the following error:
INFO[0012] try to publish to kafka err="<nil>" out_message= payload="{ \"kafka\": \"OK\", \"data\": {} " topic=hello_kafka_out echo: http: panic serving 172.17.0.1:60042: runtime error: invalid memory address or nil pointer dereference goroutine 6 [running]: net/http.(*conn).serve.func1(0xc000224000) /usr/local/go/src/net/http/server.go:1769 +0x139 panic(0xab7e00, 0x11c9e20) /usr/local/go/src/runtime/panic.go:522 +0x1b5 github.com/checkr/openmock.(*kafkaClient).sendMessage(0x0, 0xc0001dc220, 0xf, 0xc000288480, 0x112, 0x120, 0x0, 0x0) /go/src/github.com/checkr/openmock/kafka.go:43 +0x170 github.com/checkr/openmock.ActionPublishKafka.Perform(0xc0001dc220, 0xf, 0xc000158120, 0x112, 0x0, 0x0, 0xc9b8c0, 0xc00020a070, 0xc00020c4e0, 0x0, ...) /go/src/github.com/checkr/openmock/handle_actions.go:123 +0x1cd github.com/checkr/openmock.(*Mock).DoActions(0xc0000a1ee0, 0xc9b8c0, 0xc00020a070, 0xc00020c4e0, 0x0, 0x0, 0xc0001dc2b0, 0x5, 0x0, 0x0, ...) /go/src/github.com/checkr/openmock/handle_actions.go:28 +0x192 github.com/checkr/openmock.MocksArray.DoActions(0xc0001bd4e0, 0x2, 0x2, 0xc9b8c0, 0xc00020a070, 0xc00020c4e0, 0x0, 0x0, 0xc0001dc2b0, 0x5, ...) /go/src/github.com/checkr/openmock/handle_actions.go:14 +0x96 github.com/checkr/openmock.(*OpenMock).startHTTP.func2.1(0xc9b8c0, 0xc00020a070, 0x20, 0xafbe60) /go/src/github.com/checkr/openmock/http.go:42 +0x2c2 github.com/labstack/echo.(*Echo).Add.func1(0xc9b8c0, 0xc00020a070, 0x2, 0xc79180) /go/src/github.com/checkr/openmock/vendor/github.com/labstack/echo/echo.go:482 +0x87 github.com/labstack/echo/middleware.BodyDumpWithConfig.func1.1(0xc9b8c0, 0xc00020a070, 0x11f3300, 0x3) /go/src/github.com/checkr/openmock/vendor/github.com/labstack/echo/middleware/body_dump.go:81 +0x375 github.com/dafiti/echo-middleware.LogrusWithConfig.func1.1(0xc9b8c0, 0xc00020a070, 0x3, 0xc0000380a4) /go/src/github.com/checkr/openmock/vendor/github.com/dafiti/echo-middleware/logrus.go:97 +0x119 github.com/labstack/echo.(*Echo).ServeHTTP(0xc000208000, 0xc84fc0, 0xc00024c000, 0xc000226300) /go/src/github.com/checkr/openmock/vendor/github.com/labstack/echo/echo.go:594 +0x227 net/http.serverHandler.ServeHTTP(0xc0001fc000, 0xc84fc0, 0xc00024c000, 0xc000226300) /usr/local/go/src/net/http/server.go:2774 +0xa8 net/http.(*conn).serve(0xc000224000, 0xc86e00, 0xc000076240) /usr/local/go/src/net/http/server.go:1878 +0x851 created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:2884 +0x2f4

My template is as follows:
`
- key: test_kafka_1
kind: Behavior
expect:
http:
method: GET
path: /ping
kafka:
topic: hello_kafka_in
actions:
- publish_kafka:
topic: hello_kafka_out
payload: >
{
"kafka": "OK",
"data": {}

- key: test_kafka_2
kind: Behavior
expect:
http:
method: GET
path: /ping
kafka:
topic: hello_kafka_in
actions:
- publish_kafka:
topic: hello_kafka_out
payload: >
{
"kafka": "OK",
"data": {"base_tug":2.0,"avg_delta_tug":0.0,"ship_dim_pca_0":-0.021482626865567126,"ship_dim_pca_1":0.10497156370907207,"bs_ok":0,"hs_ok":0,"loc_score":0.002597402597402597,"ship_group":"Stukgoedschepen","wind_speed":4.0},
"nb_tug": 2.0
}`

Did I do any wrong here?

Usage example of GRPC is unclear

Hi!

I launched openmock by running docker-compose and by setting OPENMOCK_GRPC_ENABLED and OPENMOCK_GRPC_PORT
Then I made a GRPC request to demo_probuf.ExampleService.ExampleMethod
Request timed out and I see following in logs:

     openmock         | {"error":"empty GRPCServiceMap","level":"error","msg":"failed to convert gRPC request body to JSON","time":"2021-02-18T06:42:30Z"}
     openmock         | {"grpc_host":"localhost:50051","grpc_method":"POST","grpc_path":"/demo_protobuf.ExampleService/ExampleMethod","grpc_req":"\u0000\u0000\u0000\u0000\u000e\u0008{\u0012\u0006fooBar\u001a\u0002{{","grpc_res":"{\"message\":\"Internal Server Error\"}\n","level":"info","msg":"","time":"2021-02-18T06:42:30Z"}

I tried to find what that error means in source code and the only place where GRPCServiceMap is assigned is in /grpc_test.go
Does that mean that currently in order to use GRPC mocking one has to write a Go wrapper project?

Thanks!

Sidenote: I'd like to say that it took me some time to find out that one has to enable GRPC by setting OPENMOCK_GRPC_ENABLED
At first I just set OPENMOCK_GRPC_PORT and expected it to work 😅 since a Kafka configuration example which is a few lines below mentions OPENMOCK_KAFKA_ENABLED flag. Perhaps Readme.md should mention OPENMOCK_GRPC_ENABLED

Standardize logging format with enough logging context

We should have better logging to help with the better debugging experience.

GRPC support

Would be awesome to be able to mock GRPC services.

This project seems like it could be a good jumping-off point: https://github.com/jekiapp/gripmock

fatal error: concurrent map writes

psp-stub-crash-concurrent_map_write.log

under moderate load on 1-3 endpoints on a rate of 100 - 150 requests/second

Connection not closed properly

We're using go's "net/http" client to send http requests to the stub.
When sending multiple POST requests to the openmock http server the client fails with an EOF error.

After investigating further we noticed that the this error occurs if the server closed the connection without noticing the client.
https://stackoverflow.com/questions/28046100/golang-http-concurrent-requests-post-eof/34474535#34474535

To reproduce this I created a simple stub:

- key: mystub
  kind: Behavior
  expect:
    http:
      method: POST
      path: /
  actions:
    - reply_http:
        status_code: 200
        body: somepostresponse

And a small go test

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"strconv"
	"strings"
	"testing"
)

func TestPost(t *testing.T) {
	for i := 0; i < 20; i++ {
		if err := sendRequest(i); err != nil {
			panic(fmt.Sprintf("sending request %d failed because of %v", i, err))
		}
	}
}

func sendRequest(i int) error {
	resp, err := http.Post("http://localhost:9091", "text", strings.NewReader("request "+strconv.Itoa(i)))
	if err != nil {
		panic(err)
	}

	defer resp.Body.Close()
	respBody, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		panic(err)
	}

	println(fmt.Sprintf("got response %s for request %d", string(respBody), i))
	return nil
}

Our current workaround is to add the "connection: close" header to each stubbed request to signal the http client that it shouldn't reuse the connection, but is there any way this can be solved in the openmock server?
If not, then this at least might be useful for the next person using go and running into this issue.

Thanks!

wildcard routes can be ambiguous

HTTP routes containing wildcards can be ambiguous with routes that are longer. For example:

- key: hello-wildcard
  kind: Behavior
  expect:
    http:
      method: GET
      path: /hello/*/bar
  actions:
    - reply_http:
        status_code: 200
        body: >
          { "hello": "foo" }

- key: hello-wildcard-2
  kind: Behavior
  expect:
    http:
      method: GET
      path: /hello/*
  actions:
    - reply_http:
        status_code: 200
        body: >
          { "hello": "bar" }

With these routes, calls to /hello/foo/bar will be ambiguous. It looks like once openmock loads in the templates, it is consistent until reloaded. For some loads, the response body will be {"hello": "foo"} and other times it will be {"hello": "bar"}.

Templating returns an INT when yaml value is all digits

We are getting the following error in our templated mocks:

failed to render condition: {{
  (.HTTPBody | regexFindFirstSubmatch "<License>([^<>]+)</License>" | eq .Values.license_number)
}}  err="template: :1:297: executing \"\" at <eq .Values.license_number>: error calling eq: incompatible types for comparison"
echo: http: response.Write on hijacked connection from io.(*multiWriter).Write (multi.go:60)

We are using an AbstractBehavior:

- key: some-template
  kind: AbstractBehavior
  values:
    license_number: G1234567
  actions:
    - reply_http:
        status_code: 200
        body: OK
  expect:
    http:
      method: POST
      path: /some-path
    condition: '{{ (.HTTPBody | regexFindFirstSubmatch "<License>([^<>]+)</License>" | eq .Values.license_number) }}

The mocks look like:

- key: some-key
  extend: some-template
  values:
    license_number: G1234567
  actions:
    - redis:
...

After some investigation, the problem turns out to be that one of the mocks have license_number of 000000000, and it was templated into a int instead of a string.

Would it make sense to always return a string type?

@WilsonGiese

Missing template dir results in no errors

Openmock does not bubble up any errors if TemplatesDir is not present in the FS.

This happens because the error passed into the WalkFn by Walk does not check for or return the error at L101.

openmock/load.go

Lines 96 to 109 in 2a309c9

	func loadYAML(searchDir string) ([]byte, error) {
	logrus.Infof("Start to load templates from: %s", searchDir)
	w := &bytes.Buffer{}
	err := filepath.Walk(searchDir, func(path string, f os.FileInfo, err error) error {
	if f != nil && (strings.HasSuffix(f.Name(), ".yaml") || strings.HasSuffix(f.Name(), ".yml")) {
	content, err := ioutil.ReadFile(path)
	if err != nil {
	return err
	}
	w.Write(content)
	}
	return nil
	})

Walk will pass nil to WalkFn's f os.FileInfo if there is an error, and this has the affect of ignoring the error since our WalkFn returns nil if f == nil

Looking at the tests, this seems like expected behavior:

openmock/load_test.go

Lines 74 to 78 in 2a309c9

	t.Run("path not exists", func(t *testing.T) {
	bytes, err := loadYAML("")
	assert.NoError(t, err)
	assert.Len(t, bytes, 0)
	})

However this leads to misleading output when booting Openmock, the following logs are emitted:

time="2019-07-30T19:06:23Z" level=info msg="Start to load templates from: /file/not/exists"
time="2019-07-30T19:06:23Z" level=info msg="Done with loading templates from: /file/not/exists"

It seems to me that if the TemplatesDir is missing an error should be bubbled up and Openmock should probably fail to boot. Thoughts?