itv/scala-pact

A Scala implementation of CDC using the Pact standard

Scala-Pact

A Consumer Driven Contract testing library for Scala and ScalaTest that follows the Pact standard.

Scala-Pact is intended for Scala developers who are looking for a better way to manage the HTTP contracts between their services.

If you are just starting out on your pact journey in scala, we recommend checking out pact4s. This is built directly on top of pact-jvm, and provides support for writing and verifying contracts using scalaTest, weaver-test, and munit-cats-effect-3.

Latest version is 4.4.0

Scala-Pact currently only supports v2 of the pact specification. To use pacts with v3+ of the pact specification, use pact4s instead. This project will not support versions beyond v2.

Scala-Pact now has two branches based on SBT requirements.

SBT 1.x compatible (Latest 4.4.0)

All development going forward begins at 2.3.x and resides on the master branch. For the sake of the maintainer's sanity, version 2.3.x and beyond will only support Scala 2.12 and SBT 1.x or greater. The project is currently cross-compiled across scala 2.12.12 and 2.13.4.

SBT 0.13.x compatible (Latest 2.2.5)

The reluctantly maintained EOL maintenance version of Scala-Pact lives on a branch called v2.2.x. These versions support Scala 2.10, 2.11, and 2.12 but are limited by only supporting SBT 0.13.x.

More information

Please visit our official documentation site for more details and examples.

There is also an example project setup for reference.

Getting setup

Scala-Pact goes to great lengths to help you avoid / work around dependency conflicts. This is achieved by splitting the core functionality out of the library requirements which are provided separately. This allows you to align or avoid conflicting dependencies e.g. If your project uses a specific version of Circe, tell Scala-Pact to use Argonaut! One big change between 2.2.x and 2.3.x is that dependencies are now provided by TypeClass rather than just static linking. Please refer to the example setup.

You're using SBT 1.x:

There are two approaches to using the scala-pact dsl. The new approach uses a single dependency and mix-ins to use the dsl. The old approach allows more freedom in which http and json library versions are being used, but requires more dependencies and imports.

Mix-ins approach

Add the following lines to your build.sbt file to setup the test framework:

import com.itv.scalapact.plugin._

enablePlugins(ScalaPactPlugin)
        
libraryDependencies ++= Seq(
  "com.itv"       %% "scalapact-scalatest-suite"   % "4.4.0" % "test",
  "org.scalatest" %% "scalatest"                   % "3.2.9"  % "test"
)

Add this line to your project/plugins.sbt file to install the plugin:

addSbtPlugin("com.itv" % "sbt-scalapact" % "4.4.0")

Both the import and the plugin come pre-packaged with the latest JSON and Http libraries (http4s 0.21.x, and circe 0.13.x).

In your consumer test suites, have the test class extend PactForgerSuite. In your provider test suites, have the test class extend PactVerifySuite.

Without mix-ins

If your project needs more control over the dependencies used by scala-pact, add the following lines to your build.sbt file to setup the test framework:

import com.itv.scalapact.plugin._

enablePlugins(ScalaPactPlugin)
        
libraryDependencies ++= Seq(
  "com.itv"       %% "scalapact-circe-0-13"   % "4.4.0" % "test",
  "com.itv"       %% "scalapact-http4s-0-21"  % "4.4.0" % "test",
  "com.itv"       %% "scalapact-scalatest"    % "4.4.0" % "test",
  "org.scalatest" %% "scalatest"              % "3.2.9" % "test"
)

Add this line to your project/plugins.sbt file to install the plugin:

addSbtPlugin("com.itv" % "sbt-scalapact" % "4.4.0")

This version of the plugin comes pre-packaged with the latest JSON and Http libraries. Thanks to the way SBT works, that one plugin line will work in most cases, but if you're still having conflicts, you can also do this to use your preferred libraries:

 libraryDependencies ++= Seq(
   "com.itv" %% "scalapact-argonaut-6-2" % "4.4.0",
   "com.itv" %% "scalapact-http4s-0-21"  % "4.4.0"
 )
 
 addSbtPlugin("com.itv" % "sbt-scalapact-nodeps" % "4.4.0")

In your test suite, you will need the following imports:

The DSL/builder import for Consumer tests:

  import com.itv.scalapact.ScalaPactForger._

Or this one for Verification tests:

  import com.itv.scalapact.ScalaPactVerify._

You'll also need to reference the json and http libraries specified in the build.sbt file:

  import com.itv.scalapact.circe09._
  import com.itv.scalapact.http4s18._

Alternatively, in case your project has both scalapact-http4s and scalapact-circe as dependencies, you could also use the following:

  import com.itv.scalapact.json._
  import com.itv.scalapact.http._

You're using SBT 0.13.x:

Add the following lines to your build.sbt file to setup the test framework:

import com.itv.scalapact.plugin._

enablePlugins(ScalaPactPlugin)
        
libraryDependencies ++= Seq(
  "com.itv"       %% "scalapact-circe-0-9"     % "2.2.5" % "test",
  "com.itv"       %% "scalapact-http4s-0-18-0" % "2.2.5" % "test",
  "com.itv"       %% "scalapact-scalatest"     % "2.2.5" % "test",
  "org.scalatest" %% "scalatest"               % "3.0.5" % "test"
)

Add these lines to your project/plugins.sbt file to install the plugin:

libraryDependencies ++= Seq(
  "com.itv" %% "scalapact-argonaut-6-2"  % "2.2.5",
  "com.itv" %% "scalapact-http4s-0-16-2" % "2.2.5"
)

addSbtPlugin("com.itv" % "sbt-scalapact" % "2.2.5")

In you're test suite, you will need the following import for Consumer tests:

  import com.itv.scalapact.ScalaPactForger._

Or this one for Verification tests:

  import com.itv.scalapact.ScalaPactVerify._

Note that you can use different versions of Scala-Pact with the plugin and the testing framework, which can make Scala 2.10 compat issues easier to work around while we get the SBT 1.0 release sorted out.

scala-pact's Issues

how do I run pact-test for part of my tests?

I have many tests, some of them are pact tests. I run sbt pact-test to generate pact files, then run sbt pact-stubber to run the stub, at last, I run sbt test to run other tests that rely on the stub.

but when I run the first sbt pact-test, all tests will be run, so those depends on stub will fail.

what should I do to make things better? or am I misusing pact?

Should Scala-Pact support type matchers for header values?

Apparently the Ruby and JS versions do, although it isn't in the pact spec (v2 anyway).

Update contributing.md

Include:

• Notes about rebasing for people within ITV.
• The testing procedure.

Add docs for the standalone stubber

The standalone stubber works in exactly the same way as the pact stubber sbt command. However, you need to build it yourself, turn it into a JAR and then run it... as a JAR.
Words to that effect need adding to the docs site somewhere.
Might also be worth building a script that does the build and creates a launch script?

Prevent publishing of SNAPSHOT pacts by default

Deleting pacts from pact broker is hard work and currently a snapshot is published with a '.x' suffix that breaks retrieval of the latest pact file.

Think we should have an flag that you can override if you really want to.

Add configurable versions to pact-verify

The sbt pact-verify settings will only pull the latest pacts from the broker, can this be made configurable in pact.sbt to allow specific pact versions to pulled by the provider.

consumerNames := Seq("User Domain CI", "iOS")
versionedConsumerNames := Seq(("Consumer 1" , "1.0.0"), ("Consumer 2", "2.0.0"))

Upgrading 1.0.2 -> 2.1.0 no matching request

I have a repository of pact tests that run and pass fine on 1.0.2

Upon updating to 2.1.0 I get some errors of the form:
No matching request for: InteractionRequest(Some(POST),Some(/users/password/reset),Some(email=uon@exa),Some(Map(Connection -> keep-alive, Content-Length -> 0, Accept -> application/vnd.user.password.v1+json, Content-type -> application/x-www-form-urlencoded, Accept-Encoding -> gzip,deflate, User-Agent -> scalaj-http/1.0, Host -> localhost:51465)),None,None)

Changing order of test execution did not change what tests fail, the same tests fail every time.

Some Post and Put requests had htis error.

A failed match against the stubber should NOT return 404.

I think the Ruby version return 598 or something?
It's a red herring when you're trying to debug you're pact tests.

How to pass verification result form provider to pact-broker?

Hi all,
I'm verifying the pact which is present in pact-broker from the provider using "SBT test".
When I run the above command my pacts are getting verified but the verification result is not getting pushed to pact-broker. Can someone help me to resolve this issue(Do I need to add any extra parameters or am I missing any)?

Fail the test if the provider state fails

When a provider state fails or returns false a warning is sent but does not fail the test. It would aid debugging the pact if it is obvious that the set up failed rather than the test execution.

providerStates := Seq( ("The user with the ID SOME_ID exists", (key: String) => { false }) )

Give the provider state config access to other scopes

So that code from the main / test code base can be reused.

Implement permissive equality matching for XML body content

Optional publishing of pact files

Could you please enable optional publishing of files so that I can write tests that do not get published to the pact broker?

Implement body matchers

Create a wrapper to launch the stubber nicely, programmatically.

Related to discussions about starting and stopping Http4s nicely itself.

https://gitter.im/http4s/http4s?at=58d50e7419cdf803632bf663

Maya and Bill have done work around this independently, should compare notes.

SPIKE: Reworking body matching logic

The basic premise is to try and lift XML and JSON into a common intermediate representation and do the matching on that.

This would remove the need to maintain two different matching approaches so they can be more consistent.

Also if we bear in mind current matching issues we might be able to solve a few of them along the way e.g. meaningful reporting of where in a complex body a match failed.

Move verifer code into core and test

Allow relationships to be version at the consumer test level

At the moment pact versioning is at a project level, set in the .sbt file.

Would be nice if you could say that the relationship between consumer X and provider Y is at v1 while consumer X and provider Z is at v2.

Some("") and None should be equal in Permissive body matching.

Provide a way to set the client timeout

The client request timeout is by default at 1 sec and I don't see a way to override this default.

Consider moving to Tasks instead of Commands for the plugin

Doesn't work with chunked requests

.withBackendConfig(BackendConfig(
allowChunkedRequestBody = false
))

When a verify fails, there is no context of which interaction failed

This is not as trivial as it sounds (unless something sneaky can be done). But in a nutshell we need to output the description on failure.

Add JSON body matchers to DSL

Add 'OPTIONS' HTTP Method

Currently we have constants for get, put, post, and delete but not options.

Verifier in scalapact plugin turns any request with a body into POST

This is the default behaviour of scalaj-http, HttpRequest.postData.

def postData(data: Array[Byte]): HttpRequest = {
    val postFunc: HttpConstants.HttpExec = (req, conn) => {
      conn.setDoOutput(true)
      conn.connect
      conn.getOutputStream.write(data)
    }
    copy(method="POST", connectFunc=postFunc)
  }

Add strict mode to DSL

TLS/SSL Support

Hi,

Is there a way to configure the stubber server to run with ssl/https support?

Add ability to set pact version number

Generated pact versions are currently bound to the version defined in build.sbt. Can we get the ability to set the pact version independently as a field in pact.sbt.

Pact-publish prevents pact tests being written as integration tests

I'm working on an existing project, and the layout of the code is mandated by the policy of the group I am a part of. Unit tests are under 'src/test/scala', and integration tests are under 'src/it/scala'. Pact tests are integration tests, so they have to go under src/it/scala.

Unfortunately 'pact-publish' as well as actually publishing the pact files also does a clean and a test before it does a publish, which means that the pacts are blown away by the clean, and not recreated by the test, so it is impossible for me to use pact-publish to actually publish the pacts. I can of course do it manually with a curl or something, but I would rather do it from sbt

I've looked at the code with the help of David Smith (thanks) and the lines of potency are here. https://github.com/ITV/scala-pact/blob/master/scalapact-sbtplugin/src/main/scala/com/itv/scalapact/plugin/tester/ScalaPactTestCommand.scala#L25-L26

I'm going to have a go at 'fixing it'. One way to do that is to do a 'pact-unsafe-publish' task which would be a task with a single responsibility: publishing the pacts. That would create a composable task that would allow people to create their build pipelines as they chose.

Improve mock config object

Mock server returns incompatible body object.

We had to use Scalaz runLog instead of runLast because org.http4s.Response.bodyAsText.runLog = Vector{"good body", }. Seems that the ScalaPact mock server is adding the extra bit.

Check line 30 from the attached screen shot:

what's the default path of pact files when call pact-verify

hi,

I put pact files in project top level pact folder(same level as src). when I set --source explicitly,
sbt "pact-verify --host localhost --protocol http --port 1234 --source pacts"
it works.

if I just call sbt pact-verify, it only says:

Verifying against 'localhost', port '1234'

then nothing happened.

I also tried "delivered_pacts" folder, as examples, it doesn't work.

Basic explaination of where the match failed

During verification, if there is a match failure, you see the actual versus the expected. Which is not ideal but ok. It would be very helpful if the verifier told you at least where if failed to match: status, headers, or body.

Better, would be the match each section separately and display an aggregate of errors to avoid having to go back and forth fixing one mismatch at a time.

XML body matchers

Replace WireMock

Need to find a way to replace wiremock. We have all the matching logic so anything we do with WireMock is duplicate effort. Maybe we should just spin up an Http4s pact stubber instance?

SBT 1.0.x Compatibility

If this PR ever gets merged - all the problems are solved:
lucidsoftware/sbt-cross#10

Otherwise, the work is to re-work all the cross publishing...

Improve test framework DSL/Builder API

There are a number of long standing niggles I've had with the test framework's api. Principally that:

• It's possible to misunderstand what to do in some places e.g. Some("") and None are not equal but are perhaps both reasonable ways to want to express an empty body?
• The way you add interactions and possibly matching rules is horridly hard to work out without docs and examples. I did it that way to enforce order but now I think it would be better achieved with phantom types or similar.

Failure to download from the pact broker should fail the test

Running pact-verify against a pact broker url does not fail the test if the pact is not present or times out loading from the broker.

Should it be a hard failure?

Suggestion to allow more robust state providers

I've been looking into this repo for one of my own projects, and have found a need for more flexibility in how state providers are handled by the runner. I've looked at making this change for my own purposes. Perhaps there would be interest in making this change in master.

State providers currently do direct string comparison to parse to different handlers. Given example

providerStates := Seq(
  ("Resource with ID 1234 exists", (key: String) => {
    println("Injecting key 1234 into the database...")
    // Do some work to ensure the system under test is
    // in an appropriate state before verification
    true
  })
)

Limitation is specific resource 1234 must be agreed by both ends before hand.

My suggestion is to change direct string comparison ( str1 == str2 ) to pattern matching. This can be done by simply making ProviderState a String => Boolean type instead of Seq[(String, String => Boolean)].

This allows some more language features for parsing strings. Example of my use:

// Regex matcher
val Resource = """Resource:([a-z]+)\{([^\{\}]+)\}""".r

providerStates := {
  case key: String if key.startsWith("Resource with ID 1234 exists") =>
    println("Injecting key 1234 into the database...")
    // Do some work to ensure the system under test is
    // in an appropriate state before verification

    true
  case Resource("user", id: String) => // Given Sample:  'Resource:User:id:1234'
   // Request to setup user with id
    true
 case "Resource with ID 4321 exists"
    true

}

Note we have ability for regex and boolean statements. Direct matching is still supported. Regex matching allows me to pass user id to the provider state logic to, for example, make http request to my server to create user with provided id before executing the test.

Main concern is this is a breaking change, so updating the library will incur errors for existing provider state definitions.

If interested I can put up a PR for this.

A large suite of tests fails with 'error writing to server'

I have a large suite of Pact tests (22 tests to be precise) that each utilises 'forgePact' With test number 22 failing, and any further tests past that also fail.

The response given by the mock is 500 'Error writing to server'
Changing the order of tests doesn't change this, whatever test is executed 22nd has this 500 response code from the mock

An example sub-project

Makes sure it imports a logging dependancy.

provider_state vs. providerState

It seems that most other implementations use provider_state rather than providerState when writing the pact files.

We need to confirm this is true (I can't find any docs, our evidence is just stuff we've seen in the wild). I'll ask on the pact-dev group.

The fix (if true) is to adapt the ScalaPactReader class to accept both formats, and then change the writer to only emit provider_state.

Provider test suite

Add the ability to run verification as a test on the provider side so that the provider can isolated the routing + marshalling aspect of their code without having to do complex stubbing.

Suggestion: summarize results after pact-verify

It would be useful to get a summary after running pact-verifiy that tells you how many tests passed/failed/cancelled like sbt does:

Allow ports for pact broker address

Please allow broker access via a defined port.

• I got a broker running on port 5000.
• pactBrokerAddress := "http://localhost:5000" inside build.sbt

It throws the following erro on sbt pact-publish:

Pact broker address does not appear to be valid, should be of form: http://my.broker-address.com

Add set up and tear down functionality to provider states

Current provider state functions will only be called before a pact is executed, Ruby pact supports the concept of set up and tear downs when specifying a provider state as well as global and default provider states.

See:
https://github.com/realestate-com-au/pact/wiki/Provider-states

Support multiple pact brokers

Publish for Scala 2.12

The main roadblock here is that Http4s is not yet out for 2.12 either. They have an issue for that here:

http4s/http4s#613

BUG in 2.2.0: Publishing fails thanks to Java toString()....

This is an accidentally error and will be fixed in the next imminent release. Should add more tests to the plugin too... Unfortunately a ValidPactBrokerAddress("..url...") is being toString'd inappropriately. Fix is already on master.