Snapshot testing .NET code with Verify
An introduction to using Verify for .NET snapshot testing
Snapshot testing has become a really popular way of asserting data has the correct shape and values in software unit tests. My first introduction to these kinds of tests was with ApprovalTests for .NET. You may also have encountered them in JavaScript with Jest. But in my opinion the standout library for snapshot testing is Simon Cropp’s excellent Verify.
I’ve written previously about Verify, specifically an extension I created for Verify to facilitate snapshot testing with MongoDB. One thing that sets Verify apart is the ability to customise and configure how it works for each test, particularly in the way it serialises and sanitises/scrubs the data.
Data scrubbing (see Scrubbers) is how you deal with values that may change between unit test runs, but within a run they must be consistent. You can choose to either remove values entirely, or replace them with special placeholders.
For example, consider the following C# code that creates a dynamic JSON object:
var json = new
{
id = Guid.NewGuid(),
time = DateTimeOffset.Now,
name = "David Gardiner",
currentUser = Environment.UserName
};
var text = System.Text.Json.JsonSerializer.Serialize(json);
var settings = new VerifySettings();
settings.ScrubUserName();
// Ensure received and verified files have .json suffix (otherwise it will be .txt)
settings.UseStrictJson();
return VerifyJson(text, settings);
The actual value of the JSON object will change every time the code is run, as it includes values like a random GUID, the current date and time, and the current username. If you just compared a file that you’d serialised this object to, then it wouldn’t match if you compared it again a few seconds later.
Instead, by the use of scrubbers and sanitisers, you end up with a file like this:
{
"id": "Guid_1",
"time": "DateTimeOffset_1",
"name": "David Gardiner",
"currentUser": "TheUserName"
}
Notice that the GUID, time and current user values have all been replaced with placeholders names. The clever thing is that if you had other properties with the same original values, then Verify is smart enough to swap those all over too (so you could easily confirm that Guid_1
was used in all the right places).
This means that your test will continue to pass if you run it 5 seconds later, or if you run it on a build server on the other side of the world with a different process username.
The underlying values will have changed, but the shape and usage of them hasn’t, and that’s what you care about.
There’s a default set of scrubbers and sanitisers that you get out of the box, but you can enable additional ones or write your own. In the example above I opted in to username scrubbing by calling ScrubUserName()
.
If necessary, you can adjust how Dates and GUIDs are handled. There’s even more options in Serializer Settings and Scrubbers.
Snapshot tests dramatically simplify your unit tests - the alternative would be lots of specific property asserts, which can get pretty messy.
The other aspect of Verify that really shines is how it can integrate with any of a number of popular GUI diff tools when you’re running it interactively and a comparison fails.
In that scenario, it will automatically launch your diff tool (Beyond Compare is my diff tool of choice) showing you the actual and expected (verified) text and you can easily identify if the failure represents a bug that needs to be fixed, or alternatively it might be an expected change so you can easily update the verified content.
It’s worth noting that Verify is smart enough to realise when it is running non-interactively (like on a build server), and in that case it just reports the comparison failed, causing the related unit test to fail. No GUI diff tools are launched.
In my next post I’ll show how I took Verify and found a way to use it outside of unit tests.
Side note: The code samples in this blog post were included by using one of Simon’s other projects - https://github.com/SimonCropp/MarkdownSnippets. A really clever tool for embedding code samples in Markdown files, which has the advantage that the code samples are more likely to be valid code as you can compile the original source files