Giter Site home page Giter Site logo

go-vcr's Introduction

go-vcr

Build Status Go Reference Go Report Card codecov

go-vcr simplifies testing by recording your HTTP interactions and replaying them in future runs in order to provide fast, deterministic and accurate testing of your code.

go-vcr was inspired by the VCR library for Ruby.

Installation

Install go-vcr by executing the command below:

$ go get -v gopkg.in/dnaeon/go-vcr.v3/recorder

Note, that if you are migrating from a previous version of go-vcr, you need re-create your test cassettes, because as of go-vcr v3 there is a new format of the cassette, which is not backwards-compatible with older releases.

Usage

Please check the examples from this repo for example usage of go-vcr.

You can also refer to the test cases for additional examples.

Custom Request Matching

During replay mode, You can customize the way incoming requests are matched against the recorded request/response pairs by defining a MatcherFunc function.

For example, the following matcher will match on method, URL and body:

func customMatcher(r *http.Request, i Request) bool {
	if r.Body == nil || r.Body == http.NoBody {
		return cassette.DefaultMatcher(r, i)
	}

	var reqBody []byte
	var err error
	reqBody, err = io.ReadAll(r.Body)
	if err != nil {
		log.Fatal("failed to read request body")
	}
	r.Body.Close()
	r.Body = ioutil.NopCloser(bytes.NewBuffer(reqBody))

	return r.Method == i.Method && r.URL.String() == i.URL && string(reqBody) == i.Body
}

func recorderWithCustomMatcher() {
	rec, err := recorder.New("fixtures/matchers")
	if err != nil {
		log.Fatal(err)
	}
	defer rec.Stop() // Make sure recorder is stopped once done with it

	rec.SetReplayableInteractions(true)
	rec.SetMatcher(customMatcher)

	client := rec.GetDefaultClient()
	resp, err := client.Get("https://www.google.com/")
	...
	...
	...
}

Hooks

Hooks in go-vcr are regular functions which take an HTTP interaction and are invoked in different stages of the playback.

You can use hooks to modify a request/response before it is saved on disk, before it is returned to the client, or anything else that you might want to do with it, e.g. you might want to simply log each captured interaction.

You often provide sensitive data, such as API credentials, when making requests against a service.

By default, this data will be stored in the recorded data but you probably don't want this.

Removing or replacing data before it is stored can be done by adding one or more Hooks to your Recorder.

There are different kinds of hooks, which are invoked in different stages of the playback. The supported hook kinds are AfterCaptureHook, BeforeSaveHook and BeforeResponseReplayHook.

Here is an example that removes the Authorization header from all requests right after capturing a new interaction.

r, err := recorder.New("fixtures/filters")
if err != nil {
	log.Fatal(err)
}
defer r.Stop() // Make sure recorder is stopped once done with it

// Add a hook which removes Authorization headers from all requests
hook := func(i *cassette.Interaction) error {
	delete(i.Request.Headers, "Authorization")
	return nil
}
r.AddHook(hook, recorder.AfterCaptureHook)

Hooks added using recorder.AfterCaptureHook are applied right after an interaction is captured and added to the in-memory cassette. This may not always be what you need. For example if you modify an interaction using this hook kind then subsequent test code will see the edited response.

For instance, if a response body contains an OAuth access token that is needed for subsequent requests, then redacting the access token using a AfterCaptureHook will result in authorization failures in subsequent test code.

In such cases you would want to modify the recorded interactions right before they are saved on disk. For that purpose you should be using a BeforeSaveHook, e.g.

r, err := recorder.New("fixtures/filters")
if err != nil {
	log.Fatal(err)
}
defer r.Stop() // Make sure recorder is stopped once done with it

// Your test code will continue to see the real access token and
// it is redacted before the recorded interactions are saved on disk
hook := func(i *cassette.Interaction) error {
	if strings.Contains(i.Request.URL, "/oauth/token") {
		i.Response.Body = `{"access_token": "[REDACTED]"}`
	}

	return nil
}
r.AddHook(hook, recorder.BeforeSaveHook)

Passing Through Requests

Sometimes you want to allow specific requests to pass through to the remote server without recording anything.

Globally, you can use ModePassthrough for this, but if you want to disable the recorder for individual requests, you can add Passthrough handlers to the recorder.

The function takes a pointer to the original request, and returns a boolean, indicating if the request should pass through to the remote server.

Here's an example to pass through requests to a specific endpoint:

// Passthrough the request to the remote server if the path matches "/login".
r.AddPassthrough(func(req *http.Request) bool {
    return req.URL.Path == "/login"
})

License

go-vcr is Open Source and licensed under the BSD License

go-vcr's People

Contributors

annismckenzie avatar asad-urrahman avatar blesmol avatar brentdrich avatar cflewis avatar craig08 avatar davars avatar dhirschfeld avatar disintegrator avatar dnaeon avatar dougnukem avatar draganm avatar dthadi3 avatar falcon78 avatar fjorgemota avatar flamefork avatar haibin avatar jaymecd avatar jeanmertz avatar judy2k avatar lox avatar marco-m avatar mikesnare avatar moolitayer avatar msabramo avatar nickpad avatar retnuh avatar stephengroat avatar vangent avatar xcoulon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-vcr's Issues

Throw error on realHttpTransport connection

Is there a way to throw an error when an HTTP connection is made on realHttpTransport during ModeReplay. I imagine this is up to our own implementation of realHttpTransport - is there a recipe we could include in the readme.md? I find this a super helpful feature of Ruby VCR.

Thanks.

Add ability to skip certain interactions from getting saved to disk

Describe the problem you'd like to have solved

I'd like to have a way to specify which kind of Interaction to skip from being recorded on disk inside a cassette. This would be useful for example if I don't want the cassette spammed with 429s.

Describe the ideal solution

Perhaps something like:

recorder.AddHook(
	func(i *cassette.Interaction) error {
		if i.Response.Code == http.StatusTooManyRequests {
			i.Skip()
			return nil
		}
		return nil
	},
	recorder.BeforeSaveHook,
)

Alternatives and current workarounds

Current workaround would be to manually remove them from the cassette file.

Additional context

Thanks a lot for maintaining this! It's a great project. 👍🏻

Filter out "unplayed" Interactions?

Hi! When replaying interactions for tests, I want to be able to throw some sort of error if there are any Interactions left unplayed (aka the flow of the test has changed, and therefore the Interactions need to be rerecorded) - my initial thought was to filter based on the replayed value, except it's private. With the package as it is now, what would be the recommended way to check the Interactions in a Cassette for replayed=false, or do the equivalent of that?

I appreciate the help, and thank you for your awesome work on this package!

Gopkg.toml references etcd

The Gopkg.toml file references a constraint on etcd, but this package is not dependent on etcd from best I can tell and only uses etcd as an example. Could the etcd constraint be removed so that when using it doesn't get pulled in unnecessarily?

go-vcr/Gopkg.toml

Lines 28 to 30 in f8a7e8b

[[constraint]]
name = "github.com/coreos/etcd"
version = "3.3.4"

Feature Request: Add `AddSaveFilter` function to redact data in responses

Certain response bodies from the API we're testing contain sensitive data like OAuth access tokens. Attempting to redact this data using rec.AddFilter breaks our tests because it seems the edited response is used by subsequent test code that needs the access token.

This is what we have today for a helper:

package testsetup

import (
	"flag"
	"fmt"
	"path"
	"regexp"
	"testing"

	"github.com/dnaeon/go-vcr/cassette"
	"github.com/dnaeon/go-vcr/recorder"
)

var update = flag.Bool("update", false, "Set to update VCR recordings")

var tokenRegexp = regexp.MustCompile(`"access_token":\s*".+?"`)

func NewRecorder(t *testing.T) (*recorder.Recorder, error) {
	t.Helper()
	filename := path.Join("recordings", t.Name())
	mode := recorder.ModeReplaying
	if *update {
		mode = recorder.ModeRecording
	}

	rec, err := recorder.NewAsMode(filename, mode, nil)
	if err != nil {
		return nil, fmt.Errorf("failed to set up vcr recorder: %w", err)
	}

	rec.AddFilter(func(i *cassette.Interaction) error {
		delete(i.Request.Headers, "Authorization")

		if i.Request.Form.Get("client_id") != "" {
			i.Request.Form.Set("client_id", "[redacted]")
		}

		if i.Request.Form.Get("client_secret") != "" {
			i.Request.Form.Set("client_secret", "[redacted]")
		}

		// 👇 👇 👇 👇 👇 is breaking our tests
		i.Response.Body = tokenRegexp.ReplaceAllString(i.Response.Body, `"access_token": "[redacted]"`)

		return nil
	})

	t.Cleanup(func() {
		if rec == nil {
			return
		}

		stopErr := rec.Stop()
		if stopErr != nil {
			t.Fatal(fmt.Errorf("failed to stop VCR recorder: %w", err))
		}
	})

	return rec, err
}

Proposal

Option 1

I propose adding redaction functionality in the form of an *Recorder.AddSaveFilter method that is invoked while saving interactions. This happens on upon calling *Recorder.Stop so it seems like a great place to redact sensitive data in response.

I'm not sure if the naming is appropriate so open to suggestions there.

package testsetup

import (
	"flag"
	"fmt"
	"path"
	"regexp"
	"testing"

	"github.com/dnaeon/go-vcr/cassette"
	"github.com/dnaeon/go-vcr/recorder"
)

var update = flag.Bool("update", false, "Set to update VCR recordings")

var tokenRegexp = regexp.MustCompile(`"access_token":\s*".+?"`)

func NewRecorder(t *testing.T) (*recorder.Recorder, error) {
	t.Helper()
	filename := path.Join("recordings", t.Name())
	mode := recorder.ModeReplaying
	if *update {
		mode = recorder.ModeRecording
	}

	rec, err := recorder.NewAsMode(filename, mode, nil)
	if err != nil {
		return nil, fmt.Errorf("failed to set up vcr recorder: %w", err)
	}

	rec.AddFilter(func(i *cassette.Interaction) error {
		delete(i.Request.Headers, "Authorization")

		if i.Request.Form.Get("client_id") != "" {
			i.Request.Form.Set("client_id", "[redacted]")
		}

		if i.Request.Form.Get("client_secret") != "" {
			i.Request.Form.Set("client_secret", "[redacted]")
		}

		return nil
	})

	// 👇 👇 👇 👇 👇 proposed function
	rec.AddSaveFilter(func(i *cassette.Interaction) error {
		i.Response.Body = tokenRegexp.ReplaceAllString(i.Response.Body, `"access_token": "[redacted]"`)
		return nil
	})

	t.Cleanup(func() {
		if rec == nil {
			return
		}

		stopErr := rec.Stop()
		if stopErr != nil {
			t.Fatal(fmt.Errorf("failed to stop VCR recorder: %w", err))
		}
	})

	return rec, err
}

Option 2

Run the current filters in *Cassette.Save instead of recorder.requestHandler. This requires a breaking change in API since the filters need to be stored in Cassette instead of Recorder.

How to use go-vcr with "github.com/satori/go.uuid" getting back Nil values for uuid

When I use go-vcr with structs containing uuid.UUID github.com/satori/go.uuid I get back no values. Testing other structs without uuid.UUID types works as expected. Any advice is appreciated.

Thank you!

Testing with Ginkgo and Gomega (ommitted test code for brevity)

Go version:
go version go1.12 darwin/amd64

type Test2 struct {
	ID          uuid.UUID `json:"id"`
	Name        string    `json:"name"`
	Description string    `json:"description"`
}

func (s *UserService) Test2() *Test2 {
	url := fmt.Sprintf("%v:%v/test", s.ServiceHost, s.ServicePort)
	var client = &http.Client{Timeout: 10 * time.Second}
	req, err := http.NewRequest("GET", url, nil)
	req.Header.Add("Content-Type", "application/json")
	req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", s.ServiceToken))
	resp, err := client.Do(req)
	if err != nil {
		return nil
	}
	defer resp.Body.Close()

	var t Test2
	json.NewDecoder(resp.Body).Decode(&t)
	return &t
}

Fixture:

---
version: 1
interactions:
- request:
    body: ""
    form: {}
    headers:
      Authorization:
      - Bearer 123abc
      Content-Type:
      - application/json
    url: http://localhost:8080/test2
    method: GET
  response:
    body: |
      {"id":"5600966e-4ad6-4f79-8e01-8db13ba5c212","name":"Foo","description":"Bar"}
    headers:
      Content-Length:
      - "79"
      Content-Type:
      - application/json
      Date:
      - Fri, 19 Apr 2019 18:49:03 GMT
    status: 200 OK
    code: 200
    duration: ""

Test code:

	us := UserService{
			ServiceHost:  "http://localhost",
			ServicePort:  8080,
			ServiceToken: "123abc",
	}
	url = fmt.Sprintf("%v:%v/test", us.ServiceHost, us.ServicePort)
	req, err = http.NewRequest("GET", url, nil)
	Expect(err).To(BeNil())
	req.Header.Add("Content-Type", "application/json")
	req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", us.ServiceToken))
	resp, err = client.Do(req)
	Expect(err).To(BeNil())
	defer resp.Body.Close()
	t := us.Test()
	Expect(t.Description).To(Equal("Bar"))

	r, err = recorder.New("fixtures/test2")
	Expect(err).To(BeNil())
	defer r.Stop()

	client = &http.Client{
	   Transport: r,
	}

	url = fmt.Sprintf("%v:%v/test2", us.ServiceHost, us.ServicePort)		
	req, err = http.NewRequest("GET", url, nil)
	Expect(err).To(BeNil())
	req.Header.Add("Content-Type", "application/json")
	req.Header.Add("Authorization", fmt.Sprintf("Bearer %s", us.ServiceToken))
	resp, err = client.Do(req)
	Expect(err).To(BeNil())
	defer resp.Body.Close()
	t2 := us.Test2()

       iDString := "5600966e-4ad6-4f79-8e01-8db13ba5c212"
	id, err := uuid.FromString(iDString)
	Expect(err).To(BeNil())
	Expect(t2.ID).To(Equal(id))

Fails with

   �[91mExpected
          <uuid.UUID>: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
      to equal
          <uuid.UUID>: [86, 0, 150, 110, 74, 214, 79, 121, 142, 1, 141, 177, 59, 165, 194, 18]�[0m

Verify HTTP request was issued

Is there a way to verify that each recorded request is reproduced during the playing phase?

The use case:

  1. I write a test and record the http requests.
  2. Re-run the test and it passes
  3. I refactor the code and introduce a bug which prevents issuing a specific request.
  4. Re-run the test, it passes. It shouldn't.

recorder.ModeReplaying not working as expected

I was working on setting up testing for CI. I was hoping that if a cassette file was not present, the recorder would throw an error, which would prompt the developer to commit the missing file before their PR could be merged. This would let us run tests on CI while ensuring that no actual HTTP calls would occur.

We'd set the mode through a utility function, but the pseudo code looks like this:

import "github.com/dnaeon/go-vcr/v2/recorder"

vcr, err := recorder.NewAsMode("foo", recorder.ModeReplaying, nil)
if err != nil {
  log.Fatal(err)
}
// test using VCR
vcr.Stop()

No error occurred, and a new cassette file foo.yaml was created.

It looks like the bug may be in:

if mode != ModeDisabled {
// Depending on whether the cassette file exists or not we
// either create a new empty cassette or load from file
if _, err := os.Stat(cassetteFile); os.IsNotExist(err) || mode == ModeRecording {
// Create new cassette and enter in recording mode
c = cassette.New(cassetteName)
mode = ModeRecording

The mode is not ModeDisabled and no cassette file exists, so the mode is automatically reset to ModeRecording, then vcr.Stop() persists the cassette file, since the mode was changed to ModeRecording.

go-vcr messes with a Body set to http.NoBody

According to net/http (https://golang.org/pkg/net/http/):

http.NoBody can be used in an outgoing client request to explicitly signal that a request has zero bytes. An alternative, however, is to simply set Request.Body to nil.

recorder.go checks for r.Body != nil before inserting a new Reader (a TeeReader used to save the Body into a buffer in order to record it). However, if r.Body is http.NoBody, it still inserts the new Reader. This can result in the HTTP request being made without a Content-Length=0 header.

Use official OSI approved license?

The current license contains custom text such that it isn't recognised by go-licenses:

E0719 17:22:35.138662    3359 library.go:122] Failed to find license for gopkg.in/dnaeon/go-vcr.v3/cassette: cannot find a known open source license
E0719 17:22:35.171726    3359 library.go:122] Failed to find license for gopkg.in/dnaeon/go-vcr.v3/recorder: cannot find a known open source license

The text seems closest to the BSD-2-Clause license so perhaps just use that?

Other official (recognised) licenses are:

Implement http.RoundTripper on recorder.Recorder directly?

Just throwing this one here for discussion.

Right now recorder.Recorder contains a Transport field which implements the http.RoundTripper interface, and this is the transport we give to clients for using as their transport.

Within a recorder.Recorder we contain the Cassette and mode of the recorder. However the Transport struct also contains Cassette and mode fields, which kind of leads to unnecessary repetition and more complexity.

What do you think of implementing http.RoundTripper directly on the recorder.Recorder type?

This would remove the internal Transport we are currently using and avoid passing around the same Cassette and mode all over the place?

Of this is a breaking API change, so clients will have to address this, but we can probably handle that as part of a new release of go-vcr.

Any thoughts? @davars ? Anyone?

Thanks,
Marin

Allow for Mode to be interrogated

I would be very useful to be able to check the recorder to see if it's in Record or Playback mode (i.e. because a cassette already exists or not.) A method on recorder would be sufficient.

go-vcr does not work with `google/go-github` and/or `golang.org/x/oauth2` library

I'm attempting to record and playback interactions with the Github API (using the google/go-github library). However my interactions are not being recorded to cassettes. The "same" code is tested and working with other libraries, so I'm not sure what's going on.
For additional context, I had previously used the https://github.com/seborama/govcr library, and it was able to record the interactions correctly (though I removed it because it seems to be unmaintained).

Here's a simplified version of my test suite. It passes, but does not create any recordings. If I had to guess, there's some internal magic in the oauth2.Transport that go-vcr can't hook into.

package example_test

import (
	"context"
	"github.com/dnaeon/go-vcr/cassette"
	"github.com/dnaeon/go-vcr/recorder"
	"github.com/google/go-github/v33/github"
	"github.com/stretchr/testify/require"
	"golang.org/x/oauth2"
	"net/http"
	"path"
	"testing"
)

func TestGithub(t *testing.T) {
	//custom http.Transport, since github uses oauth2 authentication
	ts := oauth2.StaticTokenSource(
		&oauth2.Token{AccessToken: "YOUR_GITHUB_TOKEN"},
	)

	tr := &oauth2.Transport{
		Base:  http.DefaultTransport,
		Source: oauth2.ReuseTokenSource(nil, ts),
	}

	// Start our recorder
	vcrRecorder, err := recorder.NewAsMode(path.Join("testdata", "fixtures", t.Name()), recorder.ModeRecording, tr)
	require.NoError(t, err)


	// Filter out dynamic & sensitive data/headers
	// Your test code will continue to see the real access token and
	// it is redacted before the recorded interactions are saved
        // =====> commenting out this section has no impact on missing recording
	vcrRecorder.AddSaveFilter(func(i *cassette.Interaction) error {
		delete(i.Request.Headers, "Authorization")
		delete(i.Request.Headers, "User-Agent")
		i.Request.Headers["Authorization"] = []string{"Basic UExBQ0VIT0xERVI6UExBQ0VIT0xERVI="} //PLACEHOLDER:PLACEHOLDER

		return nil
	})

        // custom http.client
	httpClient := &http.Client{
		Transport: vcrRecorder,
	}

	ghClient := github.NewClient(httpClient)

	// =====> actual test, should create cassettes, but does not.
	_, _, err = ghClient.Users.Get(context.Background(), "")

	require.NoError(t, err)
}

Support read-modify-reread interactions

Currently, recorder.requestHandler, via Cassette.GetInteraction, always returns the first matching interaction.

This does not work when the client first reads original state, then updates it, and then re-reads the modified state, with both reads using the same URL. That interaction is recorded correctly (Cassette.AddInteraction simply appends without checking for duplicates), but the replay always returns the first read, i.e. the modified state is not visible.

A possible, somewhat hacky, fix, could be for the Recorder to keep an index of last used interaction, and for Cassette to start searching from that index+1, possibly wrapping over.

Alternatively, the replay could be made “strict” in the sense that only exactly the same sequence of http.Requests in that exact order would be available during the replay, and any reordered use would result in ErrInteractionNotFound or similar.

Unsupported protocol scheme error when trying to use https

Thanks for the great work with go-vcr - it is exactly what I was looking for. I'm trying to understand how to solve this issue but my still limited go skills are getting in the way so I figured out it would be helpful to create a ticket.

    r, err := recorder.New("../../testdata/githubapi")
    if err != nil {
        t.Fatal(err)
    }
    defer r.Stop() // Make sure recorder is stopped once done with it

    // Create an HTTP client and inject our transport
    client := &http.Client{
        Transport: r.Transport, // Inject our transport!
    }

    response, err := client.Get("https://example.com")

When trying to use a https URL, go-vcr will fail complaining that the protocol scheme is unsupported and it will fail with:

2015/12/29 09:42:11 Failed to process request for URL //example.com:443: unsupported protocol scheme ""

I will keep looking and open a PR in case I find a solution.

Cheers,

Upgrade gopkg.in/yaml.v2 dependency due to CVE-2019-11254

Hi,

I would kindly like to ask whether you could upgrade your dependency to module gopkg.in/yaml.v2 from v2.2.1 to v2.2.8?

There were improvements to the performance in specific cases, fixed by go-yaml/yaml#555, and versions of yaml.v2 without that fix even have a CVE entry, although the description there is a little bit confusing, as it is mainly talking about the K8S API server...

Still, updating from v2.2.1 to v2.2.8 should neither introduce compatibility issues nor other problems. I also did the update locally, and the test suite ran fine. I can also provide a PR that updates the go.mod file and contents of the vendor dir if you want.

Thanks!

CancelRequest recursively calls itself leading to `fatal error: stack overflow`

// CancelRequest implements the github.com/coreos/etcd/client.CancelableTransport interface
func (r *Recorder) CancelRequest(req *http.Request) {
	r.CancelRequest(req)
}

https://github.com/dnaeon/go-vcr/blob/master/recorder/recorder.go#L214

Should probably be:

// CancelRequest implements the github.com/coreos/etcd/client.CancelableTransport interface
func (r *Recorder) CancelRequest(req *http.Request) {
        // copy the interface to not import etcd/client
        type cancelableTransport interface {
             CancelRequest(req *http.Request)
        }
        if ct, ok := r.realTransport.(cancelableTransport); ok {
            ct.CancelRequest(req)
        }
}

Strip Authentication string from headers

Hello!

I was wondering, is there any way of modifying what is being dumped to the recorded response? For example, in the original ruby vcr gem there is an option to filter_sensitive_data, but for go-vcr package I can't figure out a way how to do the same.

Currently, I'm doing this as a quick workaround, but I don't think that it's the best way to do that:

if copiedReq.Header["Authorization"] != nil {
  copiedReq.Header["Authorization"] = []string{"<FILTERED>"}
}

// Add interaction to cassette
interaction := &cassette.Interaction{
  Request: cassette.Request{
    Headers: copiedReq.Header,
    ...
  },
  ...
}

Could anyone suggest how to do that better? Maybe we can introduce configurable Interaction struct which will be an optional parameter of a New?

Provide higher level recorder utilties for recording all tests in a package

It could be helpful to provide a higher level functionality to managing a set of recorders in a given test package.

  • For example it'd be nice to inject a Recorder in a TestMain() so that all tests in a package making HTTP requests are recording/playing back requests.
  • Resulting in a httpfixtures/TestName.yml for each Test.* in the package automatically
  • Also we can allow for an env var to re-record tests when things change e.g. go-vcr.mode=record

main_test.go

func TestMain(m *testing.M) {
    var (
        code        = 0
        // recordMode can be specified via env var: "replay" (default) | "record" | "disabled"
        recorderMode = parseRecorderMode(os.Getenv("go-vcr.mode"))
    )

    testRecorder = recorder.NewTestRecorder(recorderMode, http.DefaultTransport)

    // inject recorder transport
    http.DefaultTransport = testRecorder.Transport()

    // Run package tests
    code = m.Run()

    // Stops the recorder and writes recorded HTTP files
    testRecorder.Stop()
    os.Exit(code)
}

func parseRecorderMode(s string) recorder.Mode {
    s = strings.ToLower(s)
    switch s {
    case "record", "recording":
        return recorder.ModeRecording
    case "replay", "replaying":
        return recorder.ModeReplaying
    case "disable", "disabled":
        return recorder.ModeDisabled
    default:
        return recorder.ModeReplaying
}
  • The recorder.TestRecorder would manage a collection of recorder.Recorder mapped to each Test.* being run
  • I've found it necessary in test environments to provide additional HTTP transport to inject a X-TEST-SEQ and match the body to ensure POST requests that don't have unique request params (but result in different responses)
// RoundTrip implements http round tripper
func (l *TestRecorder) RoundTrip(req *http.Request) (*http.Response, error) {

    ctxt := l.GetContext()
    transport := ctxt.recorder.Transport

    // Don't append sequence numbers for GET requests as they should return the same information (POST etc may return new response data each time called e.g. /create)
    if req.Method == http.MethodGet {
        return transport.RoundTrip(req)
    }

    seq := ctxt.NextSequence()
    req.Header.Add("X-TEST-SEQ", strconv.Itoa(seq))
    return transport.RoundTrip(req)

 }
httpRecorder.SetMatcher(func(r *http.Request, i cassette.Request) bool {
    if r.Method == http.MethodGet {
        return cassette.DefaultMatcher(r, i) && (reflect.DeepEqual(r.Header, i.Headers))
    }

    var b bytes.Buffer
    if _, err := b.ReadFrom(r.Body); err != nil {
        return false
    }
    r.Body = ioutil.NopCloser(&b)
    return cassette.DefaultMatcher(r, i) && (b.String() == "" || b.String() == i.Body) && (reflect.DeepEqual(r.Header, i.Headers))
})

check for cancelled context when in replay mode

I have a test that calls an API that makes HTTP requests underneath with a canceled context.

When in "record" mode, the API (correctly) returns an error, and no HTTP request is made (so, nothing is added to the cassette).

However, when in "replay" mode, go-vcr doesn't check the context, so it ends up looking for the HTTP request in the cassette, and returning either some other matching HTTP request, or a "not found" error.

It would be a truer replay experience if the replay mode checked the context.

allowing matcher to see the order of requests

i just hit a scenario where having the order of the requests is more useful than complex matcher.

An easy change to the matcher function adding the position in the array of the requests, allow you to check that the order of the requests is correct and use a simple matcher.

If you are interested in this I can arrange a pull request quite easly.

ContentLength is incorrectly set to 0 for HEAD requests

https://github.com/dnaeon/go-vcr/blob/master/recorder/recorder.go#L222
always sets Response.ContentLength to buf.Len(), where buf is the returned body.

Per HTTP/1.1, for HEAD requests ContentLength should be set to the size of the body that would have been sent for a GET: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13.

Could this field be set based on the Content-Length header for HEAD requests? I'd be happy to send a PR.

Add a new mode: ModeRecordingOrReplaying

I'd like to have a new mode: ModeRecordingOrReplaying. It will do the recording if cassette returns ErrInteractionNotFound. Otherwise it does replaying. It is useful to me. How about others?

version "v2.0.0" invalid

go-vcr doesn't follow V2 go modules

Doing :

require (
	github.com/dnaeon/go-vcr v2.0.0
)

Gives

go: errors parsing go.mod:
/Users/thibault.delor/workspace/go-search-proxy/go.mod:10:2: require github.com/dnaeon/go-vcr: version "v2.0.0" invalid: should be v0 or v1, not v2

I have renovate configured and its trying to use github.com/dnaeon/go-vcr/v2 v2.0.0 which leads to

go: github.com/dnaeon/go-vcr/[email protected]: go.mod has non-.../v2 module path "github.com/dnaeon/go-vcr" (and .../v2/go.mod does not exist) at revision v2.0.0

v2.1.0 NewAsMode fails with ModeReplaying and missing cassette file

The latest minor version update contained a breaking change in the NewAsMode function.

// Check if the cassette exists
if _, err := os.Stat(cassetteFile); os.IsNotExist(err) {
// Replaying mode should fail if no cassette exists
if mode == ModeReplaying {
return nil, cassette.ErrCassetteNotFound
}

In the previous version we were using (v.2.0.1), this function just created an empty cassette if there was no cassette file found with the given name. With v2.1.0 this behavior changed and calling NewAsMode using ModeReplaying in combination with a non-existing cassette filename returns an error.
Especially in test scenarios where the recorder is just setup in a SetupTest method for all tests, independent of whether all test methods execute an outgoing http call, this is an undesired behavior.

Could we get this function backwards compatible by introducing a flag which defines whether a missing cassette file should return an error?

Thanks in advance!

go-vcr and using it to test GitHub Client with Oauth 2

Hi there,

I'd like to use this library to test with the GitHub client.

From their docs I would need to inject an Oauth2 client to handle the Oauth2 dance
https://github.com/google/go-github/blob/master/example/tokenauth/main.go#L29

Is it possible to use go-vcr with an Oauth2 client to save responses and replay them later?

My understanding is that the recorder needs to be injected into the client as the Transport https://github.com/dnaeon/go-vcr/blob/master/example/https_test.go#L46

I'm still poking around both libraries but don't know a lot about the internals of the Oauth2 and http libraries. Is this even possible?

Any assistance would be appreciated

Thanks

add filter after reading the cassette

sometime we wants to modify the recorded values dynamically.

for example some 3rd party library generates a requestID on the HTTP header and verify it when it got the response back.

for example maybe a ReplyFilter can be added to give the caller a chance to modify the request or response

Add hook to allow roundtripping values

We have request to a server that include a unique ID that needs to be included in the response:

Request:

{ "type": "request", "id": "abc123", "message": "hello!" }

Response:

{ "type": "response", "id": "abc123", "message": "nice to meet you!" }

The request is generated with code like the following:

request := Request{Type: "Request", Id: id.Random().String(), Message: "hello!"}

Right now, the first interaction will be recorded with the first random ID. All future replays will include the original id.

  • Neither the AfterCaptureHook nor BeforeSaveHook trigger on replays, so they can't patch up the response with the correct id
  • The BeforeResponseReplayHook does trigger on replays, but the i.Request contains the first recorded id value, so it can't patch up the response with the correct id

What is the correct way to go about fixing up an interaction in this way?

Filter should not mutate the real response or request

Currently, if you use a filter and remove a response header, the actual response header is mutated. This prevent using filter to remove sensitive data that would be returned through headers.

	r.AddFilter(func(i *cassette.Interaction) error {
		// This works fine, although if you inspect the request at the client level,
                // it will not have the 'X-Auth-Token' present anymore
		delete(i.Request.Headers, "X-Auth-Token")

                // This will mutate the original response object and will not pass the 'X-Subject-Token'
                // to the client
		delete(i.Response.Headers, "X-Subject-Token")
		return nil
	})

Tests fail if filter modifies URL

Adding this filter to my recorder makes tests to fail.

		r.AddFilter(func(_ *cassette.Interaction) error {
			u, _ := url.Parse(i.Request.URL)
			q := u.Query()
			q.Set("access_key", "no-op")
			u.RawQuery = q.Encode()
			i.Request.URL = u.String()
			return nil
		})

The service I'm building this library for insists on using the access key as a URL param since they've been doing that since launch nearly 10 years ago.

Since I'm snapshot testing the output of the requests, I don't want the access key getting logged.

But, modifying the interaction causes subsequent tests to fail.

What can I do to fix this?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.