spectolabs/hoverfly

Lightweight service virtualization/ API simulation / API mocking tool for developers and testers

API simulations for development and testing

Hoverfly is a lightweight, open source API simulation tool. Using Hoverfly, you can create realistic simulations of the APIs your application depends on.

• Replace slow, flaky API dependencies with realistic, re-usable simulations
• Simulate network latency, random failures or rate limits to test edge-cases
• Extend and customize with any programming language
• Export, share, edit and import API simulations
• CLI and native language bindings for Java
• REST API
• Lightweight, high-performance, run anywhere
• Apache 2 license

Hoverfly is developed and maintained by SpectoLabs.

Quickstart

• Download and installation
• Read the docs
• Join the mailing list

Hoverfly Cloud

Hoverfly Cloud - API simulations as a service. Sign up here.

Contributing

Contributions are welcome!

To contribute, please:

1. Fork the repository
2. Create a feature branch on your fork
3. Commit your changes, and create a pull request against Hoverfly's master branch
4. In your pull request, include details regarding your change, i.e
   1. why you made it
   2. how to test it
   3. any information about testing you have performed

To read more about forking model, check out this link: forking workflow.

Setup for local development

Build

1. Install Go. You should follow the download and installation instructions for your OS from official Go website. If you have installed Go before using either apt-get or homebrew, you should uninstall those first.

2. Clone the hoverfly project:

   git clone https://github.com/SpectoLabs/hoverfly.git
   # or: git clone https://github.com/<your_username>/hoverfly.git

3. Finally you can run the build

   cd hoverfly
   make build 

Notice the binaries are in the target directory.

Test

You should be able to execute all unit and functional tests with:

make test

Some middleware tests may fail if you don't have ruby and python setup in your environment. If you are using Mac, you can install them with Homebrew:

brew install ruby
brew install python

License

Apache License version 2.0 See LICENSE for details.

(c) iOCO 2024.

hoverfly's Issues

2way SSL is not supported

Hi Team,

I greatly like your product; which to our need. We appreciate your effort for making it open source. But we found that hoverfly doesn't support 2-way-ssl.

B'coz of this we are not able to do virtualize any of our backend service in Cap1. Hope you will provide a fix for this which will enable us to use the tool smoothly.

unable to use hoverfly with firefox / https

Getting hoverfly to work as a proxy for http connections with firefox was quite simple. However, I'm unable to get it to work as a proxy for firefox with https connections. I have no idea where to even start with the debugging.

I added the certificate to my system with the following commands:

 $ sudo cp ca.pem /usr/local/share/ca-certificates/hoverfly.crt
 $ sudo update-ca-certificates

In firefox, I went into the preferences for connection settings and manually set a proxy using "localhost" and port 8500. I added the ca.pem certificate, also through the preferences panel.

I started hoverfly with the following:

$ ./hoverfly_linux_amd64  --capture -v

When I go to https://yahoo.com the page says "Secure Connection Failed" and this is what I see on stdout:

$ ./hoverfly_linux_amd64  --capture -v
{"databaseName":"requests.db","level":"info","msg":"Initiating database","time":"2016-02-25T09:14:48-06:00"}
{"level":"debug","msg":"new counter created, registration successful","time":"2016-02-25T09:14:48-06:00"}
{"Destination":".","Mode":"capture","ProxyPort":"8500","level":"info","msg":"Proxy prepared...","time":"2016-02-25T09:14:48-06:00"}
{"AdminPort":"8888","level":"info","msg":"Admin interface is starting...","time":"2016-02-25T09:14:48-06:00"}
[negroni] listening on :8888
2016/02/25 09:14:52 [001] INFO: Running 2 CONNECT handlers
2016/02/25 09:14:52 [001] INFO: on 0th handler: &{2 <nil> 0x570300} yahoo.com:443
2016/02/25 09:14:52 [001] INFO: Assuming CONNECT is TLS, mitm proxying it
2016/02/25 09:14:52 [001] INFO: signing for yahoo.com
2016/02/25 09:14:52 [001] WARN: Cannot handshake client yahoo.com:443 remote error: bad certificate

Are there extra or different configuration steps I need to take?

Enable data replication across multiple hoverfly instances

Failed to retrieve response from cache using BoltDB

When running Hoverfly as a webserver using a dataset persisted as a requests.db, I am unable to retrieve a response from the cache. The only way to get the data working is to export the data and then reimport it while Hoverfly is configured to be a webserver

Binary data response with synthetic middleware

With python middleware setting a text response body is simple

payload_dict['response']['body'] = "hello"

Or if the response body is in a file

body_path = "path/to/response_body.txt"
body_file = open(body_path, "r")
payload_dict['response']['body'] = body_file.read()

How would I respond with a binary data file? For example an image

body_path = "path/to/image.jpg"
body_file = open(body_path, "r")
payload_dict['response']['body'] = body_file.read()

Above fails with

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

hoverctl loads hoverfly from PATH, should look in current directory first

hoverctl and hoverfly seem to often get bundled together, however hoverctl seems to look for the hoverfly binary in the path.

This can lead to strange situations, where hoverctl does not find the hoverfly binary, or loads the wrong hoverfly binary.

Maybe hoverctl should first try to load hoverfly from the directory it is in, before loading it from the path.

dir, _ := filepath.Abs(filepath.Dir(os.Args[0]))
hfbin := filepath.Join(dir, "hoverfly")
if _, err := os.Stat(hfbin); err == nil {
  // use hfbin
} else {
  // load from path as before
}

Running into "No match found" error

I have just recorded the request/response pairings from API endpoints tests, and switching to "simulate" but getting "No match found" errors when re-running the tests.
Here's the error:

{"destination":"hoverfly-user:8500","error":"key \"2d48c51b560946d5ae7a64739311fdc9\" not found \n","key":"2d48c51b560946d5ae7a64739311fdc9","level":"warning","method":"POST","msg":"Failed to retrieve response from cache","path":"/cards","query":"","time":"2016-08-23T13:11:23Z"}

And the stored req / response is:

"response": {
        "status": 200,
        "body": "{\"_embedded\":{\"cards\":[{\"longNum\":\"59535****4448\",\"expires\":\"08/19\",\"ccv\":\"***\",\"_links\":{\"card\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830ae\"},\"self\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830ae\"}}},{\"longNum\":\"5544154011345918\",\"expires\":\"08/19\",\"ccv\":\"958\",\"_links\":{\"card\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830b1\"},\"self\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830b1\"}}},{\"longNum\":\"0908415193175205\",\"expires\":\"08/19\",\"ccv\":\"280\",\"_links\":{\"card\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830b4\"},\"self\":{\"href\":\"http://user/cards/57a98d98e4b00679b4a830b4\"}}},{\"longNum\":\"5429804235432\",\"expires\":\"04/16\",\"ccv\":\"432\",\"_links\":{\"card\":{\"href\":\"http://user/cards/57a98ddce4b00679b4a830d2\"},\"self\":{\"href\":\"http://user/cards/57a98ddce4b00679b4a830d2\"}}}]}}\n",
        "encodedBody": false,
        "headers": {
          "Content-Length": [
            "799"
          ],
          "Content-Type": [
            "application/json; charset=utf-8"
          ],
          "Date": [
            "Tue, 23 Aug 2016 13:10:44 GMT"
          ],
          "Hoverfly": [
            "Was-Here"
          ]
        }
      },
      "request": {
        "path": "/cards",
        "method": "GET",
        "destination": "user-dev:8084",
        "scheme": "http",
        "query": "",
        "body": "",
        "headers": {
          "Connection": [
            "close"
          ],
          "User-Agent": [
            "Dredd/1.5.0 (Linux 4.4.16-moby; x64)"
          ]
        }
      }
    },

I am running docker version of hoverfly, image: spectolabs/hoverfly

Is there anything obviously wrong?

Thanks.

Log input middleware will receive at verbose level

To aid debugging failures in a middleware, the JSON payload that was sent to the executable should be logged

simulations.json is different each time it gets curled

Dear SpectoLabs,

This is to inform you of a bug.

Version: v0.9.2

Repro Steps:

hoverctl stop
hoverctl start
hoverctl mode capture
for i in {1..5}
do
  curl -x http://localhost:8500 http://echo.jsontest.com/key/${i}
done
curl http://localhost:8888/api/v2/simulation | python -m json.tool > sim1.json
curl http://localhost:8888/api/v2/simulation | python -m json.tool > sim2.json
diff sim1.json sim2.json
hoverctl stop

Expected Result:

No difference in the json.

Actual Result:

23c23
<                     "path": "/key/4",
---
>                     "path": "/key/1",
29c29
<                     "body": "{\"key\": \"4\"}\n",
---
>                     "body": "{\"key\": \"1\"}\n",
42c42
<                             "Sat, 24 Dec 2016 10:37:31 GMT"
---
>                             "Sat, 24 Dec 2016 10:37:30 GMT"
51c51
<                             "61bf2a2d584db437722fbc1bb02c5a3a"
---
>                             "7c8ef7b70a0e613205a4b42900f5feae"
73c73
<                     "path": "/key/5",
---
>                     "path": "/key/2",
79c79
<                     "body": "{\"key\": \"5\"}\n",
---
>                     "body": "{\"key\": \"2\"}\n",
101c101
<                             "b3de2c0e06bc77e8f2b7eb9a68227a28"
---
>                             "13df72751a32cb9d4097a5a524ff80d2"
123c123
<                     "path": "/key/1",
---
>                     "path": "/key/3",
129c129
<                     "body": "{\"key\": \"1\"}\n",
---
>                     "body": "{\"key\": \"3\"}\n",
142c142
<                             "Sat, 24 Dec 2016 10:37:30 GMT"
---
>                             "Sat, 24 Dec 2016 10:37:31 GMT"
151c151
<                             "7c8ef7b70a0e613205a4b42900f5feae"
---
>                             "b1ca31fb24337554706f13cd68ac2909"
173c173
<                     "path": "/key/2",
---
>                     "path": "/key/4",
179c179
<                     "body": "{\"key\": \"2\"}\n",
---
>                     "body": "{\"key\": \"4\"}\n",
201c201
<                             "13df72751a32cb9d4097a5a524ff80d2"
---
>                             "61bf2a2d584db437722fbc1bb02c5a3a"
223c223
<                     "path": "/key/3",
---
>                     "path": "/key/5",
229c229
<                     "body": "{\"key\": \"3\"}\n",
---
>                     "body": "{\"key\": \"5\"}\n",
251c251
<                             "b1ca31fb24337554706f13cd68ac2909"
---
>                             "b3de2c0e06bc77e8f2b7eb9a68227a28"

(the json differs)

Error Message:

None.

Further Info:

The results are as expected when exporting from hoverctl.

Regards,

Shyal

Hoverctl to Support -cert and -key

Is there a way to run hoverctl with the -cert and -key arguments in that same way I am able to run the hoverfly binary with these arguments?

Also, thank you for this great product.

Different behavior in downloaded binary and self-compiled code

Hi Team,

The pre-compiled binary (downloaded from the links in the documentation) fails silently. Below is all that I can see when my application tries to connect to the box api (https://api.box.com).

2016/06/29 11:35:09 [029] INFO: Running 2 CONNECT handlers 2016/06/29 11:35:09 [030] INFO: Running 2 CONNECT handlers 2016/06/29 11:35:09 [031] INFO: Running 2 CONNECT handlers 2016/06/29 11:35:09 [032] INFO: Running 2 CONNECT handlers 2016/06/29 11:35:09 [033] INFO: Running 2 CONNECT handlers 2016/06/29 11:35:09 [034] INFO: Running 2 CONNECT handlers

Interestingly, I am able to get hoverfly to work partially using a self-compiled binary which is strange as I haven't made any code changes. The only change I have made is to the jwt-go package import. jwt-go was updated to v3.0.0 a few days ago. This was a breaking update and caused build errors while trying to compile the hoverfly source code as-is. I was able to get it to compile by using the gopkg.in redirect to the old (v2) version of jwt-go.

Using my self-compiled code, I get the message "response from external service got successfuly!" along with a lot of other info and debug messages. However, now I am unable to fully export the service-data captured. A call to http://localhost:8888/api/records gives me the below error:

PANIC: interface conversion: *negroni.responseWriter is not negroni.ResponseWriter: missing method Before goroutine 300 [running]: github.com/codegangsta/negroni.(*Recovery).ServeHTTP.func1(0xa6ab28, 0xc8205e9a00, 0xc8201aa810) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/recovery.go:39 +0x231 panic(0x4e1f80, 0xc82035e500) /usr/local/go/src/runtime/panic.go:443 +0x4e9 github.com/meatballhat/negroni-logrus.(*Middleware).ServeHTTP(0xc8201a2420, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0, 0xc820442140) /Users/awaise/go_code/HoverflyTest/src/github.com/meatballhat/negroni-logrus/middleware.go:138 +0x6dd github.com/codegangsta/negroni.middleware.ServeHTTP(0xa65170, 0xc8201a2420, 0xc820193d20, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0xd8 github.com/codegangsta/negroni.(middleware).ServeHTTP-fm(0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0x63 github.com/codegangsta/negroni.(*Static).ServeHTTP(0xc8201aa840, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0, 0xc820442060) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/static.go:53 +0x323 github.com/codegangsta/negroni.middleware.ServeHTTP(0xa65118, 0xc8201aa840, 0xc820193d00, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0xd8 github.com/codegangsta/negroni.(middleware).ServeHTTP-fm(0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0x63 github.com/codegangsta/negroni.(*Logger).ServeHTTP(0xc82002c258, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0, 0xc820442000) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/logger.go:25 +0x241 github.com/codegangsta/negroni.middleware.ServeHTTP(0xa650f0, 0xc82002c258, 0xc820193ce0, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0xd8 github.com/codegangsta/negroni.(middleware).ServeHTTP-fm(0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0x63 github.com/codegangsta/negroni.(*Recovery).ServeHTTP(0xc8201aa810, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0, 0xc8200d5fe0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/recovery.go:54 +0x78 github.com/codegangsta/negroni.middleware.ServeHTTP(0xa650c8, 0xc8201aa810, 0xc820193cc0, 0xa6ab28, 0xc8205e9a00, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:33 +0xd8 github.com/codegangsta/negroni.(*Negroni).ServeHTTP(0xc8201aa8a0, 0xa65350, 0xc8200640d0, 0xc8204089a0) /Users/awaise/go_code/HoverflyTest/src/github.com/codegangsta/negroni/negroni.go:73 +0xbb net/http.serverHandler.ServeHTTP(0xc82017d000, 0xa65350, 0xc8200640d0, 0xc8204089a0) /usr/local/go/src/net/http/server.go:2081 +0x19e net/http.(*conn).serve(0xc820336500) /usr/local/go/src/net/http/server.go:1472 +0xf2e created by net/http.(*Server).Serve /usr/local/go/src/net/http/server.go:2137 +0x44e

I am an absolute beginner in Go (planning to use the binary out of the box along with a JavaScript/Python middleware). Please suggest a workaround if possible.

Thanks!

Add support for ZeroMQ or RabbitMQ

This has been discussed and came up in HN. It's not clear we would do it in Hoverfly or when we could do it.

Please add use-cases here.

hoverfly decoding the encode url

Hi Team,

Hoverfly is decoding the encoded url while sending back to the backend; where as the backend is expecting an encoded url to process.

e.g:
encoded url --> Jnm4MOzS8yh1qxo7DVEpkLpqK9ZAqky913QJ6A7T0OOk6pqwV%2FLnBC8x4CBEXhKX

hoverfly is decoding the %2F to / which is causing the api call to fail.

Implement two way SSL

Proposed solution for this is to allow Hoverfly to be configurable with a client certificate to use with two way SSL.

It will then be up to the end user to decide on whether to generate a new client certificate for Hoverfly, or re-use an existing client certificate. The former is most likely to be used as it means avoiding sharing of private keys between existing clients and Hoverfly, but also means that the server will have to be configured to trust the new certificate used by Hoverfly.

Partially matching requests

There are a couple of different use cases that we've been discussing for more configurable mapping of requests to responses:

• Matching on a set of user-specified headers ( issue #203 )
• Looser matching of individual request components, e.g. parameterised URL paths, comparison of json/xml fields in request body

Currently Hoverfly matches all incoming requests to responses by lookup in a key-value store based on a hash of the scheme, method, host, path, query and body (but not on headers).

The proposed change will allow users to modify which request fields they want to match with on a per-request basis, creating what are effectively request templates. If Hoverfly doesn't find an exact match for the request in the key-value store, then it would iterate through the set of request templates associated with the current simulation, looking for a match.

We think that this will give the maximum flexibility in how matching will work. For the first pass we are assuming that:

1. The number of request templates will be relatively small compared to the number of requests recorded / requests made
2. Since the number of request templates is small, its not going to be too much work for people to create them by directly modifying json ( although we also plan to look at how these can be captured directly )

Woudl be interested to hear any feedback or related use cases that people might have

param -v don't show output where script crash

In previous versions when something fails in the script the out of that fail was showed on console.
Now I'm getting a crash but i can't see the line where the problem is.

Edit request with graphic interface

Hi should be nice, edit the request from ui interface, Is there any plan on that?

debugging hoverfly

Hi,

Is there currently a method in place for debugging Hoverfly?

https://golang.org/doc/gdb

Regards,

Shyal

Run Hoverfly in simulate mode as web server

The proposition would be to configure Hoverfly as a web server as opposed to a proxy when using it in a development. The reason behind this is it's a property change for applications to point to a new web server, as opposed to proxy configuration. The applications are the left untouched and behave in the same way (ie, not talking via a proxy). My instincts tell me this would only make sense when simulating - during capture seems like it would be easier to remain as a proxy. I'd be interested to get some input.

middleware delay not working for PT --> beyond 60TPS

Hi Team,

We have recently done a LoadTest of hoverfly for one of our api, using the -middleware (both python & go) with introducing a delay of 60ms. We observed that after 60 TPS, the hoverfly started queuing of the python/go middleware multiple processes; there by causing the CPU% to go beyond 65-70%. Hence the response time started increasing beyond 3-4 secs and the TPS didn't go beyond 120.

For Python middleware used:

#!/usr/bin/env python
from time import sleep
SLEEP_SECS = 0.060
def main():
    sleep(SLEEP_SECS)
if __name__ == "__main__":
    main()

command run:

sudo nohup ./hoverfly_v0.5.17_linux_amd64 -v -pp 8082 --middleware "./virtDelayResponse.py" -dest HOST:PORT 2>&1 1>hoverflystdout_date +%Y-%m-%d_%H:%M:%S:%N.log &

For GO middleware used:

package main
import (
    "flag"
    "time"
)
func main() {
    var duration int64
    flag.Int64Var(&duration, "duration", 100, "100 ms sleep")
    flag.Parse()
    time.Sleep(time.Duration(duration) * time.Millisecond)
}

Command Run (both with compile and without compile):

sudo nohup ./hoverfly_v0.5.17_linux_amd64 -pp 8082 --middleware "./goPerfDelay –duration 60" -dest HOST:PORT 2>&1 1>hoverflystdout_date +%Y-%m-%d_%H:%M:%S:%N.log &

Note: is there a way to introduce the delay within the hoverfly; rather than using the middleware?

Lookup response by key only

In my use case it is not practical for me to record the responses, I want to write them manually.

I've looked around a bit and it doesn't seem like it is currently possible, would it possible to make hoverfly lookup a response from import file by key / id only?

When you do a proxy call and a response is not found the message is:

Hoverfly Error! Could not find recorded request, please record it first!. 
Got error: key "7a4e6d5c94e616d64255313523d9699a" not found 

I then want to create an import file without specifying what the request is:

{
    "data": [
        {
            "response": {
                "status": 500,
                "body": "1234: Some error",
                "headers": {"Content-Type": ["text/html; charset=utf-8"]}
            },
            "id": "7a4e6d5c94e616d64255313523d9699a"
        }
    ]
}

Delays does not follow convention set by RequestMatchers

e.g:

Actual endpoint - http://REALHOST:REALPORT/user/data/{userid}

We have captured the req by setting the proxy. (localhost:8500)

Then we start the server as ./hoverfly -webserver

Set the response delay as below:

 curl -H "Content-Type application/json" -X PUT -d '{"data":[{"urlPattern":"localhost\\:8500","delay":1000}]}' http://${HOVERFLY_HOST}:8888/api/delays

Now when i ran the req http://localhost:8500/user/data/{userid} the response delay is not working.

Hoverfly as http proxy

Hi, hoverfly looks amazing but i'm failing trying to set it up as a proxy to modify the response for certain urls.

My python script is the same to the one here https://www.specto.io/service-virtualization-modifying-traffiic/

But I don't get it works.

./hoverfly --modify --middleware="./myscript"

I expect modify all HTTP GET responses but when I use any browser every response is not customized.

Issues with POST requests

I am trying hoverfly in combination with Dredd + OpenAPI spec.

I am running tests against hoverfly in capture mode.

My GET requests get through fine, however, for POST requests hoverfly returns 301s and Dredd won't follow through.
Moreover, the location header is set to /endpoint, without a host prefix.

Example:

request:
method: POST
uri: /addresses
headers:
    User-Agent: Dredd/1.5.0 (Linux 4.4.16-moby; x64)
    Content-Type: application/json
    Content-Length: 95

body:
{
  "address": {
    "street": "teststreet",
    "number": "15",
    "country": "The Netherlands",
    "city": "Den Haag"
  }
}


expected:
headers:

body:

statusCode: 200
bodySchema: {"title":"Post status response","type":"object","properties":{"status":{"type":"boolean"}},"required":["status"]}


actual:
statusCode: 301
headers:
    content-length: 0
    content-type: text/plain; charset=utf-8
    date: Tue, 23 Aug 2016 09:59:18 GMT
    hoverfly: Was-Here
    location: /addresses
    connection: close

body:

What's the policy for POST requests, are they somehow special?

Inspecting in middleware script

Hi,

I want intercept just certains http calls so I decide use if-then-else to decide who url's will be mocked.

But when I inspect the request details a part of payload comes empty, why? :(
I want check the method, query, path, destination to decide what I want to mock.

data = sys.stdin.readlines()
payload = data[0]
logging.debug(payload)

DEBUG:root:Middleware is called
DEBUG:root:{"response":{"status":0,"body":"","headers":null},"request":{"path":"/posicionIntegrada/datosPosicionAgrupados/0002/N603638/10648812898","method":"GET","destination":"180.197.78.39:8097","scheme":"http","query":"agrupacion=ADV","body":"","remoteAddr":"[::1]:53527","headers":{"Accept":["*/*"],"Accept-Encoding":["gzip, deflate, sdch"],"Accept-Language":["en-US,en;q=0.8,es;q=0.6"],"Cache-Control":["no-cache"],"Postman-Token":["4277b47c-f92c-2a8f-cd96-dd5bd7a99a4d"],"Proxy-Connection":["keep-alive"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36"]}},"id":""}
DEBUG:root:Middleware is called
DEBUG:root:{"response":{"status":200,"body":"{\"investmentProposalGrouping\":[{\"agrupacionDesc\":\"Carteira de Custodia com Perfil\",\"agrupacionId\":\"555511987\",\"agrupacionTipo\":\"CUSTCP\",\"agrupacionSaldo\":3.05,\"agrupacionPcrt\":1.4727422522060096E-4,\"investmentProposalGrouping\":[{\"agrupacionDesc\":\"Alternativo\",\"agrupacionId\":\"IN\",\"agrupacionSaldo\":3.05,\"agrupacionPcrt\":1.4727422522060096E-4,\"investmentProposalGrouping\":[{\"agrupacionDesc\":\"Alternativo\",\"agrupacionId\":\"IN\",\"agrupacionSaldo\":3.05,\"agrupacionPcrt\":1.4727422522060096E-4,\"distribucion\":[{\"producto\":{\"idProducto\":\"TESOURARIA\",\"nombreProducto\":\"Tesouraria\",\"idCartera\":\"555511987\",\"descCartera\":\"Carteira de Custodia com Perfil\",\"tipoCartera\":\"CUSTCP\",\"perfilCartera\":\"CONS\",\"tipoActivo\":\"0001\",\"descTipoActivo\":\"ALTERNATIVO\",\"divisaProducto\":\"BRL\",\"tipoCambio\":1.0,\"fechaSaldo\":\"2015-11-10\",\"reqProducto\":0,\"iLiquidezProducto\":1,\"comGestion\":1.1,\"comDistribucion\":0.6,\"cotizacion\":9.12,\"tipoAdvisory2\":\"ALTR\",\"tipoProducto\":\"0012\",\"entProducto\":\"0002\",\"isinProducto\":\"0\",\"zonaGeografica\":\"0001\",\"productosRefer\":\"N\",\"complejoBasico\":\"SIMP\",\"recomendado\":\"9999\",\"recomEmisor\":\"REBL\",\"comercializable\":\"N\",\"rent12Meses\":0.0,\"rent5Anio\":0.0,\"leadingValue\":0,\"rentEsperada\":12.0,\"productoEmision\":\"N\",\"maxConcentracion\":100,\"fechaCreacion\":\"2016-01-12\",\"fechaModificacion\":\"2016-01-12\",\"idPregunta3Test\":\"12\",\"emergente\":\"S\",\"fmi\":\"N\",\"focusList\":\"ACOES\",\"importeMinimo\":1000,\"tipoAdvisory2Desc\":\"Alternativo\",\"tipoAdvisory1\":\"IN\",\"impMinimoAdicional\":5000,\"iliquidezProducto\":1},\"saldo\":3.05,\"saldoPcrt\":1.4727422522060096E-4}]}]}]}],\"totalPosicionGlobal\":2070966.59,\"totalInversion\":2070966.59,\"totalFinanciacion\":0.0,\"fechaSaldo\":\"2015-11-10\"}","headers":{"Access-Control-Allow-Headers":["x-requested-with, Cache-Control, Accept, Content-type"],"Access-Control-Allow-Methods":["POST, GET, PUT, DELETE"],"Access-Control-Allow-Origin":["*"],"Access-Control-Max-Age":["3600"],"Content-Type":["application/json;charset=UTF-8"],"Date":["Mon, 08 Feb 2016 12:40:17 GMT"],"Hoverfly":["Was-Here"],"Server":["Apache-Coyote/1.1"],"X-Application-Context":["position-backend-div:local:8097"]}},"request":{"path":"","method":"","destination":"","scheme":"","query":"","body":"","remoteAddr":"","headers":null},"id":""}

add authentication to admin UI

Set response delays per URL based on URL pattern

Currently, delays can be set per host using hostPattern to match to the URL host. Allowing delays to be set per URL using urlPattern (for example) would allow for more fine-grained control over individual URLs on the same host.

Homebrew formula

It'd be easier to install hoverfly through a homebrew formula.

--import flag will not redirect to retrieve HTTP JSON

v0.8.2 fixed an issue regarding Hoverfly automatically resolving redirects.

This has caused a very minor bug regarding the startup flag --import. When trying to import a JSON simulation file, if the URL responds with a redirect, Hoverfly will fail to import.

This bug has no affect on using the flag to import a local JSON simulation and also has no affect on using the /api/records endpoint in any way.

This bug will only impact those who specify a remote resource on startup if there is a redirect involved in resolving that resource.

Include request headers in middleware payload

Middleware may wish to inspect a header from the request, so example returning JSON or XML depending on what the client accepts.

Following installation getting : {error:Not Found}

Following installation (Mac OSX / zsh) I get an error:

/usr/local/bin/hoverctl: line 1: {error:Not Found}: command not found

Snippet from terminal:

...

Installation complete. To get started:
1. Run 'hoverctl pull benjvi/hello-world:latest'
2. Set your http_proxy environment variable: 'export http_proxy=http://localhost:8500/'
3. Run 'hoverctl start'
3. Run 'hoverctl import benjvi/hello-world:latest'
4. Run 'curl -L http://lab.specto.io/static/hello-world.html', or open in the browser on linux

...

$ hoverctl pull benjvi/hello-world:latest
/usr/local/bin/hoverctl: line 1: {error:Not Found}: command not found
$ hoverctl
/usr/local/bin/hoverctl: line 1: {error:Not Found}: command not found
$ hoverctl start
/usr/local/bin/hoverctl: line 1: {error:Not Found}: command not found

Review diagnostics in case of panic!

Providing a generic solution for all potential failures would probably be difficult but issue #255 suggests there are circumstances where Hoverfly has sufficient context and we could dump context (URI, REST headers, request, template, response, middleware, etc) perhaps to the log or to a separate file.

Runtime error: invalid memory address or nil pointer reference

From last couple of days, hoverfly has been crashing continuously and intermittently with following error logs:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0xfc90f]

goroutine 1103 [running]:
panic(0x4bed00, 0xc82000c0b0)
/usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/SpectoLabs/hoverfly/core.(_Hoverfly).captureRequest(0xc820163340, 0x0, 0x55bd50, 0x0, 0x0)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/hoverfly.go:313 +0x12cf
github.com/SpectoLabs/hoverfly/core.(_Hoverfly).processRequest(0xc820163340, 0xc82064e1c0, 0xc82002aab8, 0x1)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/hoverfly.go:203 +0xb6
github.com/SpectoLabs/hoverfly/core.NewProxy.func2(0xc82064e1c0, 0xc820bb2000, 0xc820bb2000, 0x1)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/proxy.go:51 +0x2a
github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.FuncReqHandler.Handle(0xc820da36e0, 0xc82064e1c0, 0xc820bb2000, 0xc5001, 0xc820b4a0a0)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/actions.go:19 +0x30
github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.(_ReqProxyConds).Do.func1(0xc82064e1c0, 0xc820bb2000, 0x11, 0x2f3501)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/dispatcher.go:200 +0x141
github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.FuncReqHandler.Handle(0xc8207869e0, 0xc82064e1c0, 0xc820bb2000, 0x0, 0x0)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/actions.go:19 +0x30
github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.(_ProxyHttpServer).filterRequest(0xc820014400, 0xc82064e1c0, 0xc820bb2000, 0xc82064e1c0, 0x0)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/proxy.go:56 +0xc7
github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.(_ProxyHttpServer).handleHttps.func1(0xa2ef00, 0xc820894060, 0xc82086a000, 0xc820bb2000, 0xc820cca1c0, 0xc820014400)
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/https.go:184 +0x891
created by github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy.(_ProxyHttpServer).handleHttps
/home/ubuntu/.go_workspace/src/github.com/SpectoLabs/hoverfly/core/vendor/github.com/rusenask/goproxy/https.go:236 +0xe7a

Matching request query should ignore the order of query parameters

Given the following simulation data in Hoverfly

"request": {
          "requestType": "recording",
          "path": "/api/bookings",
          "method": "POST",
          "destination": "www.my-test.com",
          "scheme": "",
          "query": "destination=newyork&class=business&class=premium",
          "body": "{\"flightId\": \"1\"}",
          "headers": {
            "Content-Type": [
              "application/json"
            ]
          }
        }
      }

It's supposed to match the following requests with same set of query parameters in any order:
http://www.my-test.com/api/bookings?destination=newyork&class=business&class=premium
http://www.my-test.com/api/bookings?class=business&class=premium&destination=newyork
http://www.my-test.com/api/bookings?class=premium&destination=newyork&class=business

But Hoverfly only does exact matching of the query string.

Is there any architecture document?

I am interested in Hoverfly and I want to know how it works, which components exist in it, how these components interact with others.

Is there any architecture document?

Hoverctl bash auto-completion

Would be nice to have auto-completion when using the command line tool.

build and run instructions missing from README

Recording request headers should be optional

@mogronalol has raised the concern of potentially exposing sensitive information from recording request headers.

Perhaps the request headers recording should be disabled by default, and options for a user to record specific headers would be provided.

Expand documentation to cover utilisation as a proxy

readthedocs_query outdated

Hi,
I tried the readthedocs_query example, but I think that it is outdated.

In the starting URL "format=json" is missing, and then in the json list also "subdomain" is missing. Could you verify and fix?

Partially match in templates

With the addition of template partial matching I'm able to create templates for my requests. However I couldn't find in the docs how to create a template match for a particular path.

Let's say I have a path that looks like this:

GET
/user/2312323/status

How can I match any user id on my template? I was expecting to be able to type something similar to this but it didn't work:

"requestTemplate": {
            "path": "/user/*/status",
            "method": "GET",
            "scheme": "http"
        },

Thanks!
-Ivan

virtualized from oneTool/machine didn't work on other machine/Tool

Hi Team,

We captured an request by using SOAP UI from one machine; and when we tried to simulate the same request from other machine using the SOAP UI; we got the error message of key not found.

How to force use python3

Hi, I miss a lot work with python3. Is there any way to select python3 instead of work with python2 by default?

Include stderr output of middleware when logging middleware failure

My middleware is currently failing and the stderr is being lost
Logs:

{"error":"exit status 1","level":"error","msg":"Failed to process pipeline","time":"2016-02-24T16:18:34Z"}
{"error":"exit status 1","level":"error","middleware":"/path/to/an/executable","msg":"Error during middleware transformation, not modifying payload!","time":"2016-02-24T16:18:34Z"}

http/2 support

This likely should be tagged a feature request.

I have a c++ client that is talking to a java server, and they speak http2 (implictly https). The feature set hoverfly offers is great, and I've used it on http projects, but (please correct me if I'm missing something obvious here) there is no apparent http/2 support.

When trying out an initial test case, I get an error related to a malformed http response (reasonable if http/2 support is lacking), but when I convert the test case to use http, it works just fine.

If this ends up being a feature request, I'd be interested in the proposed timeline. I may be able to contribute to this effort in order to get it going.

hoverfly as webserver doesn't provide the right content-type

My api response content-type is expected as application/json; where as hoverfly gives back as text/plain
Hoverfly response -

HTTP/1.1 200 OK
Req: /user/aaa111
Resp: 124
Date: Wed, 24 Aug 2016 04:42:21 GMT
Content-Length: 124
Content-Type: text/plain; charset=utf-8

{"userId":"727a87f5-03cb-4f69-9778-6585d37461c5","eid":"aaa111","name":"Test User","email":"[email protected]","team":"Foo"}

Actual response -

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 124
ETag: W/"7c-vKwqaAK/uwMcGkLv6JIknA"
Date: Wed, 24 Aug 2016 18:00:26 GMT
Connection: keep-alive

{"userId":"727a87f5-03cb-4f69-9778-6585d37461c5","eid":"aaa111","name":"Test User","email":"[email protected]","team":"Foo"}

And also observer that in the ui - http://localhost:8888; under the count section not count increases if i use as webserver mode.

Set response delays based on HTTP Method

For a specific URL (matched with a regular expression), it should be possible to set different response delays for different HTTP methods.

Set response delays per URL based on URL pattern

Currently, delays can be set per host using hostPattern to match to the URL host. Allowing delays to be set per URL using urlPattern (for example) would allow for more fine-grained control over individual URLs on the same host.

Spelling Mistake

Hi,
There is spelling mistake in hoverfly.go at L:20

https://github.com/SpectoLabs/hoverfly/blob/develop/hoverfly.go#L20

const SynthesizeMode = "sinthesize"

Thanks

Cache to match on request headers

Is it possible to allow for certain headers to be matched on? For example, I'd like to hit the same URL with two different auth headers, have it return different data and cache both responses. The current behavior seems to overwrite the previous cache result if the URLs match.