waldonl/reocca

Recorder / replayer of (initially) rest responses, on top of Spray

REOCCA

Test/loadtest recorder/replayer. Built on top of spray (spray.io), written in Scala.

A typical use case for REOCCA would be to serve mock responses to an application under load test. REOCCA is a Spray microservice that

• serves those mock responses when in 'replay' mode
• forwards to configured URLs when in 'forward' mode
• caches the response returned by a forward in 'record' mode
• caches can be imported, exported or edited
• testers can specify a delay time per response
• different testers can share a single REOCCA service and autonomously manage their own cache, (aka cache target)

How to run

Assuming Scala and Sbt are installed and the git repo is cloned, at the root directory startup SBT

    sbt -Dservice.port=9999

and inside the SBT console enter

    run

This starts REOCCA on your localhost at port 9999. Now you can load a cache via a rest PUT with header Content-Type = application/json; charset=UTF-8 on

    http://localhost:9999/REOCCA/myexample

Here is an example of a cache to post:

        [  { "target" : {
            "name" : "/todos/toforward",
            "replay" : true, "forward" : true, "record" : true,
            "minSimDelay" : 1000, "maxSimDelay" : 2500,
            "keyGeneratorName" : "default",
            "keyGeneratorParameter" : "pathFilter{1}",
            "url" : "http://localhost:8889/one/todos/urgent",
            "filterNamespace" : true,
            "skipHttpHeaders" : true,
            "entries" : [
             {    "key" : "",
                  "method" : "get", 
                  "keyRequestHeader" : [
                        {   "Content-Type" : "application/json"},
                        {   "Accept-encoding" : "UTF-8"}
                  ],
                  "requestHeader" : [                {"Accept":"*/*"}
                  ],
                  "responseHeader" : [
                        {   "Content-Type" : "application/json"},
                  ],
                  "response" : {"objective" : "this is just a simple task"}
             },
             {    "key" : "inprogress",
                  "method" : "get", "requestHeader" : "tbd", "responseHeader" : "tbd",
                  "response" : {"objective" : "get this working"}
             },
             {   "key" : "/inprogress",
                  "method" : "get", "requestHeader" : "tbd", "responseHeader" : "tbd",
                  "response" : {"objective" : "get this working late"}
             }      ]
          }}]

Studying the structure of this json should give you a good overview of REOCCA's possibilities. Note, however that some keywords refer to functionality which is not working at the time of writing. The open issue list shows what is still to come.

If you now do a GET on http://localhost:9999/myexample/todos/toforward the response will be

    {"objective" : "this is just a simple task"}

Notice how the path after REOCCA in the PUT request becomes the name of the cache target, and the context root via which to reach your cache entries. While this is still in beta (or alpha?), if REOCCA receives a request it can not handle, it will return the entire cache (i.e. all cache targets).

###FAQ

• Is REOCCA bound to port 9999?
  • Obviously not ;) Forwarding already works, so it is perfectly feasible to have 2 servers on different ports, where one server forwards to the other. Of course you can also test forwarding with a rest service of your own. Make sure its response is json, and the content-type is according.
• Why Spray and not Akka HTTP?
  • When I started this (Q2-2015) I assumed the maturity of Spray would lead to less surprises, and hence would be the safest route to a result that could demonstrate functionality and scalability. Porting to Akka HTTP in the near future is a viable option.
• Why Scala and not Java 8?
  • I believed that it would be easier and more natural to make use of Spray (itself being written in Scala) using Scala. I believed the reactive principles are easier to follow in Scala. My recent experience seems to confirm both.

reocca's Issues

recognise post methods

error: if no path entries exist for some method, NPE follows

give proper error when forwarding fails

make spray use the new route after cache upload and route rebuild

record all request headers

BREAKING CHANGE: include http headers in the cache to upload via json

NOTE: BREAKING CHANGE!

Up to this point, request and response headers were marshalled as simple strings, but from this issue on, each cache target will have three http header lists where a list contains http header entries which in json are fields where the name denotes the header name and the value the header value.
The three lists are:

1. key request headers: each header in this list must match with an incoming request header
2. request headers: these headers are not for matching incoming requests but rather to be included during recording.
3. response headers: will be included during replay, unless skip headers is true.

Note that the previous json structure ceases to be accepted!

restore feature: if a request does not match any entry, as a debugging feature, return the entire cache map

upload a cache should not replace the previous cache map

update the route with the response acquired from a forward

extend target entry key with request parameters

A request including request parameters has to be matched with target entries. Note that the order of the request parameters is irrelevant

Fix json serialization of http headers

Per issue #38, the json representation of the cache, that reocca returns, reflects the name - value structure of the reocca case class, not the name header values that are in the json to upload a cache. This issue shall fix this deviation.

allow per field cache updates

e.g. PUT localhost:9990/REOCCA/mycache2/tudus/keras/late?replay=true

the path is the concatenation of the cache name, entry name and key. Note that that needs to be unique per cache entry.

write a test for #13 forwarding test

sort entries to prevent shadowing of deeper paths

export a named cache

GET REOCCA/{cachename}

must result in a json response fit to be PUT back (i.e. under another name)

refactoring: rename apiservice to routemanager

record all response headers

fix http unbind - bind sequence

to activate the new route, I bind a new actor with a new route to Http actor, after an Http.Unbind.
This does not have the desired effect

MAJOR FEATURE: BROWSER INTERFACE

extend the default key to include request headers

increase path segment depth

Currently reocca supports just one path (segment). This limitation must be removed

restructure cache to share a single target url configuration by multiple response entries

introduce cache actor

the cache is now shared and not thread safe. by having all updates on the cache done via a separate actor, no updates will go lost, without having any waits.

implement minimum delay according to configuration

fix runtime warnings regarding explicitly set response headers

move initial cache to test spec and start reocca with an empty cache

allow portnumber ( or, rather : ) inside the forwarding url

currently, when the forwarding url is passed in the http client GET, the colon is translated to %3A

extend configuration updates with response

the issue is that the response is json, so proper converting has to take place, withiout disrupting the form field unmarshalling

forwarding GET

publish (in some way) monitoring statistics

like

  implicit val statsMarshaller: Marshaller[Stats] =
    Marshaller.delegate[Stats, String](ContentTypes.`text/plain`) { stats =>
      "Uptime                : " + stats.uptime.formatHMS + '\n' +
      "Total requests        : " + stats.totalRequests + '\n' +
      "Open requests         : " + stats.openRequests + '\n' +
      "Max open requests     : " + stats.maxOpenRequests + '\n' +
      "Total connections     : " + stats.totalConnections + '\n' +
      "Open connections      : " + stats.openConnections + '\n' +
      "Max open connections  : " + stats.maxOpenConnections + '\n' +
     "Requests timed out    : " + stats.requestTimeouts + '\n'
    }

to be found at :
Spray examples

give the cache a name and prefix path with that name

tell apart put from get requests

allow remaining segments as key

cache entry with path="todos/urgent"
and key="inprogress"
hits on request with path "cachename/todos/urgent/inprogress"

spread delay over min-max range

currently delay is only considering minimum delay. instead, the delay of each request should be a random value between min and max delay

PM user defined key generation

allow upload of json cache via rest PUT

/reocca/{cacheName}
body contains complete cache

explore oncomplete of the future[http response]

establish whether the oncomplete() plays well with the complete() processing of that same future

further test recording entries of empty cache targets

During fixing #21 I noticed an issue regarding cache targets without entries, I made some code changes particularly for #21, which could also help when recording requests. This needs to be investigated.

when forward and record is on, and a request hits a name but misses all keys, add the request's key