HTTP Test Recording with Go and go-vcr

When testing code that interacts with HTTP APIs, it is often cumbersome to test them, and especially to automatically test them with real data as part of a continuous deployment process.

To tackle this, it is useful to record your API requests and responses. This can be done in Go by using go-vcr, which gives you a http.Transport you can use in a custom http.Client.

go-vcr uses the concept of a recorder and a cassette. The recorder will save a request-response mapping. This is stored in a cassette.

Creating http.Transport with go-vcr

Creating a custom http.Transport is very straightforward. In fact, the recorder struct implements the http.Transport interface. So it suffices to create a recorder.
In my example I change the mode of the recorder (recording vs replaying) based on whether the UPDATE environment flag is set.
The cassette is stored in testdata/go-vcr, because testdata directories are ignored by Go.

// UpdateCassette ENV variable so we know when to update the cassette.
_, UpdateCassette := os.LookupEnv("UPDATE")

recorderMode := recorder.ModeReplaying
if UpdateCassette {
    recorderMode = recorder.ModeRecording
}

// Setup recorder
r, err := recorder.NewAsMode("testdata/go-vcr", recorderMode, nil)
if err != nil {
    return nil, nil, err
}

Deleting Sensitive Information

Since your tests are version controlled, it is very important to delete all sensitive data from the cassette. This can be done by adding filters to the recorder. Below, I delete the x-auth-token and authorization headers. But depending on the HTTP API you are testing you might need to delete more sensitive data.

// Add a filter which removes Authorization and x-auth-token headers from all requests
r.AddFilter(func(i *cassette.Interaction) error {
    delete(i.Request.Headers, "x-auth-token")
    delete(i.Request.Headers, "authorization")
    return nil
})

Doing HTTP Requests with the Custom Transport

Now you have a recording transport, you can start writing tests. First create a HTTP client with the custom transport. Once you have done this, performing HTTP requests and performing tests on them is business as usual…

// Create new http.Client where transport is the recorder
httpClient := &http.Client{Transport: r}

req, err := http.NewRequest("GET", "http://api.example.com/some/object", nil)
if err != nil {
    // handle error
}
resp, err := httpClient.Do(req)
if err != nil {
    // handle error
}

// perform tests on the response...