Giter Site home page Giter Site logo

swisslife-oss / snapshooter Goto Github PK

View Code? Open in Web Editor NEW
292.0 11.0 28.0 2.89 MB

Snapshooter is a snapshot testing tool for .NET Core and .NET Framework

Home Page: https://swisslife-oss.github.io/snapshooter/

License: MIT License

C# 100.00%
testing net snapshot snapshot-testing xunit json csharp test tool

snapshooter's Introduction

Snapshooter

Nuget GitHub Release Build Status Coverage Status Quality

Snapshooter is a snapshot testing tool for .NET Core and .NET Framework

Snapshooter is a flexible snapshot testing tool to simplify the result validation in your unit tests in .Net. It is based on the idea of Jest Snapshot Testing.

To get more detailed information about Snapshooter, go to the Snapshooter Docs

Getting Started

To get started, install the Snapshooter Xunit or NUnit nuget package:

XUnit

dotnet add package Snapshooter.Xunit

NUnit

dotnet add package Snapshooter.NUnit

MSTest

dotnet add package Snapshooter.MSTest

Get Started

Assert with Snapshots

To assert your test results with snapshots in your unit tests, follow the following steps:

1. Add snapshot assert statement

Insert a snapshot assert statement Snapshot.Match(yourResultObject); into your unit test.

Example:

/// <summary>
/// Tests if the new created person is valid.
/// </summary>
[Fact]
public void CreatePersonSnapshotTest()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestPerson person = serviceClient.CreatePerson(
        Guid.Parse("2292F21C-8501-4771-A070-C79C7C7EF451"), "David", "Mustermann");

    // assert
    Snapshot.Match(person);
}

2. Run the unit test to create a new Snapshot

The Snapshot.Match(person) statement creates a new snapshot of your result object and stores it in the __snapshots__ folder. The __snapshots__ folder is always next to your executed unit test file.

Snapshot name: <UnitTestClassName>.<TestMethodName>.snap

3. Review new snapshot

Review your new snapshot file __snapshots__/<UnitTestClassName>.<TestMethodName>.snap.

4. Run unit test to assert

Now the Snapshot.Match(person) statement will create again a snapshot of your test result and compare it against your reviewed snapshot in the __snapshots__ folder. The __snapshots__ folder is always next to your executed unit test file.

Mismatching Snapshot Handling

If your result object has changed and the existing snapshot is not matching anymore, then the unit test will fail. The unit test error message will point to the exact mismatching position within the snapshot.

In addition, in the snapshot folder __snapshots__ a subfolder with name __mismatch__ will be created. In this folder you can find the actual snapshot which is mismatching with the existing snapshot in the __snapshots__ folder. Therefore it is possible to compare the two snapshots with a text compare tool.

If the snapshot in the mismatching folder __mismatch__ is correct, just move it to the parent __snapshots__ folder (override the existing one).

Read More

Different Match-Syntax Possibilities

The default match syntax for snapshots is:

    Snapshot.Match(person);

However, we also could use the fluent syntax:

    person.MatchSnapshot();

Or we can use FluentAssertion's should() syntax:

    person.Should().MatchSnapshot();

For NUnit we will support the Assert.That syntax (Coming soon):

    Assert.That(person, Match.Snapshot());

Features

Ignore Fields in Snapshots

If some fields in your snapshot shall be ignored during snapshot assertion, then the following ignore options can be used:

[Fact]
public void CreatePersonSnapshot_IgnoreId()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestPerson person = serviceClient.CreatePerson("Hans", "Muster");

    // assert
    Snapshot.Match<Person>(testPerson, matchOptions => matchOptions.IgnoreField("Size"));
}

The fields to ignore will be located via JsonPath, therefore you are very flexible and you can also ignore fields from child objects or arrays.

Ignore Examples:

// Ignores the field 'StreetNumber' of the child node 'Address' of the person
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreField("Address.StreetNumber"));

// Ignores the field 'Name' of the child node 'Country' of the child node 'Address' of the person
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreField("Address.Country.Name"));

// Ignores the field 'Id' of the first person in the 'Relatives' array of the person
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreField("Relatives[0].Id"));

// Ignores the field 'Name' of all 'Children' nodes of the person
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreField("Children[*].Name"));

// Ignores all fields with name 'Id'
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreField("**.Id"));

Ignore All Fields by name

If we want to ignore all fields by a specific name, then we have two options:

Option 1: Use the ignore match option 'IgnoreAllFields()' and add the name.

// Ignores all fields with name 'Id'
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreAllFields("Id"));

Option 2: Use the default ignore match option 'IgnoreFields(**.)' with the following JsonPath syntax **.

// Ignores all fields with name 'Id'
Snapshot.Match<Person>(person, matchOptions => matchOptions.IgnoreFields("**.Id"));

Hash Fields in Snapshots

If some fields of our snapshot are too big, for example a binary field with a lot of data, then we can use the HashField option. The HashField option creates a Hash of the field value and therefore each time only the hash is compared. If there is a change in the field value, then the snapshot match will fail.

[Fact]
public void ImageSnapshot_HashImageBinary()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestImage image = serviceClient.CreateMonaLisaImage();

    // assert
    Snapshot.Match(image, matchOptions => matchOptions.HashField("Data"));
}

Example Snapshot with Hash

{
  "Id": 3450987,
  "OwnerId": "0680faef-6e89-4d52-bad8-291053c66696",
  "Name": "Mona Lisa",
  "CreationDate": "2020-11-10T21:23:09.036+01:00",
  "Price": 951868484.345,
  "Data": "m+sQR9KG9WpgYoQiRASPkt9FLJOLsjK86UuiXKVRzas="  
}

The field(s) to hash can be located via JsonPath or via field name.

Hash Field Examples:

// Hash the field 'Data' of the child node 'Thumbnail' of the person
Snapshot.Match<Person>(person, matchOptions => matchOptions.HashField("Thumbnail.Data"));

// Hash the field 'Data' of the first thumbnail in the 'Thumbnails' array of the image
Snapshot.Match<Person>(person, matchOptions => matchOptions.HashField("Thumbnails[0].Data"));

// Ignores the field 'Data' of all 'Thumbnails' nodes of the image
Snapshot.Match<Person>(person, matchOptions => matchOptions.HashField("Thumbnails[*].Data"));

// Ignores all fields with name 'Data'
Snapshot.Match<Person>(person, matchOptions => matchOptions.HashField("**.Data"));

Assert Fields in Snapshots

Sometimes there are fields in a snapshot, which you want to assert separately against another value.

For Example, the Id field of a 'Person' is always newly generated in a service, therefore you receive in the test always a Person with a new id (Guid). Now if you want to check that the id is not an empty Guid, the Assert option can be used.

/// <summary>
/// Tests if the new created person is valid and the person id is not empty.
/// </summary>
[Fact]
public void CreatePersonSnapshot_AssertId()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestPerson person = serviceClient.CreatePerson("Hans", "Muster"); // --> id is created within the service

    // assert
    Snapshot.Match<Person>(testPerson, matchOption => matchOption.Assert(
                    fieldOption => Assert.NotEqual(Guid.Empty, fieldOption.Field<Guid>("Id"))));
}

The fields to assert will be located via JsonPath, therefore you are very flexible and you can also ignore fields from child objects or arrays.

Assert Examples:

// Assert the field 'Street' of the 'Address' of the person
Snapshot.Match<Person>(person, matchOption => matchOption.Assert(
                    fieldOption => Assert.Equal(15, fieldOption.Field<int>("Address.StreetNumber"))));

// Asserts the field 'Code' of the field 'Country' of the 'Address' of the person
Snapshot.Match<Person>(person, matchOption => matchOption.Assert(
                    fieldOption => Assert.Equal("De", fieldOption.Field<CountryCode>("Address.Country.Code"))));

// Asserts the fist 'Id' field of the 'Relatives' array of the person
Snapshot.Match<Person>(person, > matchOption.Assert(
                    fieldOption => Assert.NotNull(fieldOption.Field<string>("Relatives[0].Id"))));

// Asserts every 'Id' field of all the 'Relatives' of the person
Snapshot.Match<Person>(person, > matchOption.Assert(
                    fieldOption => Assert.NotNull(fieldOption.Fields<string>("Relatives[*].Id"))));
 
// Asserts 'Relatives' array is not empty
Snapshot.Match<Person>(person, > matchOption.Assert(
                    fieldOption => Assert.NotNull(fieldOption.Fields<TestPerson>("Relatives[*]"))));

The Snapshooter assert functionality is not limited to Xunit or NUnit asserts, it also could be used Fluent Assertions or another assert tool.

Concatenate Ignore & Asserts checks

All the ignore, isType or assert field checks can be concatenated.

[Fact]
public void Match_ConcatenateFieldChecksTest_SuccessfulMatch()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestPerson person = serviceClient.CreatePerson("Hans", "Muster");

    // act & assert
    Snapshot.Match<TestPerson>(testPerson, matchOption => matchOption
            .Assert(option => Assert.NotEqual(Guid.Empty, option.Field<Guid>("Id")))
            .IgnoreField<DateTime>("CreationDate")
            .Assert(option => Assert.Equal(-58, option.Field<int>("Address.StreetNumber")))
            .Assert(option => testChild.Should().BeEquivalentTo(option.Field<TestChild>("Children[3]")))
            .IgnoreField<TestCountry>("Address.Country")
            .Assert(option => Assert.Null(option.Field<TestCountry>("Relatives[0].Address.Plz"))));
}

Using Snapshooter in CI-Builds

When running snapshooter tests in a CI-build you might want to ensure that a snapshots are correctly checked-in since otherwise tests without a snapshot will just create the initial snapshot and become green.

In order to fail tests that are without a snapshot on your CI-build you can set the snapshooter behavior to strict-mode by setting the environment variable SNAPSHOOTER_STRICT_MODE to on or true.

Community

This project has adopted the code of conduct defined by the Contributor Covenant to clarify expected behavior in our community. For more information, see the Swiss Life OSS Code of Conduct.

snapshooter's People

Contributors

aaronpowell avatar alisan3 avatar daghsentinel avatar fgreinacher avatar glucaci avatar gojanpaolo avatar hallepas avatar huysentruitw avatar joslat avatar koditkarvedant avatar mcliment avatar michaelstaib avatar nscheibe avatar onecyrus avatar rohrerf avatar sbica avatar simoncropp avatar swisslife-bot avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

snapshooter's Issues

Matching some elements

It would be interesting if we could match a data structure to just some elements of the snapshot.

For example, I want to check that just the first 6 elements of a snapshot containing a list are there.

The business case would be a permission list where some of them are key, fundamental and are the ones in the original snapshot. But the list will naturally grow with more custom permissions... so the tests will fail over and over, becoming a very brittle test.

I would like that I could check the current collection of permissions against this initial snapshot and determine that they exist.

For this I would like two things:

  • That I can cut the elements in the initial snapshot to a limit (ex: element 0 until element 6).
  • That I can check the current list contains the elements in the snapshot.

For example, I have a snapshot of 10 permissions.

{
    element: "permission 1",
    .....
},
{
    element: "permission 2",
    .....
},
...

And a List<permissions> PermList which contains the current and always changing persons... could contain 100 or more..

I'd like something as PermissionList.MatchSnapshot( matchOptions => matchOptions.MatchUntilElement(2));

So only the two first permissions in the snapshot are "matched".

And the test would work forever and ever ;)

Allow customising the serializer settings

Is your feature request related to a problem? Please describe.
Allow adjustment of the JsonSerializerSettings in SnapshotSerializer, so that snapshots can be made more readable e.g. when using custom JsonConverters they are taken into account.

e.g. an example of the current behaviour

"myList": [
    {
      "Name": "My SmartEnum",
      "Value": "My Value that should be used instead"
    }
],

after allowing to add converters for instance

"myList": [
      "My Value that should be used instead"
],

Describe the solution you'd like

Not yet full thought through, but maybe

  • Allow defining a convention based class e.g. **CustomSnapshotSerialzation** that could expose either the complete JsonSerializerSettings or hooks to be called by the default implementation when adding converters.

What would be great is to at least be able to add the following

 serializerSettings.Converters.Add(new SmartEnumValueConverter<MySmartEnum, string>());

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Related to #115

Support diff tools to compare snapshots

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Shallow Snapshot

Sometimes snapshot files can get very big (like 90000 lines for one snapshot). However I'm only interested in the first level or second level of properties and not in all levels.

It would be nice if I could control how big the stored snapshot gets. For example I'm only interested in a shallow snapshot which means not in all properties of any child objects.

In the following example I have a person with one child. In this particular case I'm interested in the person's name and the amount of children, but not in the children's name and so on.

{
    name: "Person 1",
    children: [
        {
            name: "Child 1"
        }
    ]
}

So the stored snapshot could optimized like this.

{
    name: "Person 1",
    children: [ "Object Child" ]
}

Also it would be nice to define the depth of the shallow snapshot to control how precise the comparison should be.

Support for Mixed Object Types

Is your feature request related to a problem? Please describe.
I can't find/work out how to have mixed types (e.g. a List or some sort of generic override) and ignore fields on objects only if they exist. Is this supported?
Currently, if the Ignored Field path does not exist, an exception is thrown which is a little frustrating as i'm trying to ignore it.

Describe the solution you'd like
Possibly support the syntax MatchOptions.IgnoreFields("Data.[*].RowId", strictMode: false) or something similar to allow the field to be ignored if it is not set.

Describe alternatives you've considered
I looked at "IsType" etc but i don't really want to compare all the other fields of the type, i just want to ignore the path i'm trying to set.

Additional context
I may just be missing something here, any help would be appreciated.

Reuse Snapshot across multple tests.

Sometimes multiple tests produce equal snapshots...
And it needs to be checked.

How about add possiblity to match against different test snapshot or common snapshot?

Xunit expected and actual wrong way round?

Describe the bug

The snapshot that I have saved has field "Amount": "0.00", my API is returning "Amount": "0", but the test output appears to have these the wrong way round:

  Error Message:
   Assert.Equal() Failure
                                 โ†“ (pos 3854)
Expected: ยทยทยท        "Amount": "0",\n              "Display": "ยฃ0.00"\n     ยทยทยท
Actual:   ยทยทยท        "Amount": "0.00",\n              "Display": "ยฃ0.00"\n  ยทยทยท
                                 โ†‘ (pos 3854)

The generated mismatch snap file contains "Amount": "0".

I've had a look through the snapshooter source and can't see any obvious problem.

To Reproduce

Using Snapshooter.Xunit 0.7.1

Expected behavior

I would expect the output to say

Expected: ยทยทยท        "Amount": "0.00"
Actual:   ยทยทยท        "Amount": "0"

Desktop (please complete the following information):

  • OS: Windows
  • Version 10

Snapshot is equal

Usually the object.MatchSnapshot(...) or Snapshot.Match(...) throws an exception if the snapshot does not match.

Now people asked for an option without exception, but with a boolean as return value.

I thought about the following syntax:

  • bool Snapshot.IsEqual(...)
  • bool object.Snapshot().IsEqual()
  • bool object.IsEqualSnapshot()

Feat: String snapshot

A basic string snapshot would be very useful, i.e.:

Snapshot.Match("Some illegal json text")

This errors with Error reading JToken from JsonReader.

Or at least a note in the documentation that this only works with Json serializable objects.
I don't mind if there would be an additional method, ie:

Snapshot.MatchStr("This does work");

Exclude from serialize process the ignore fields

If we haven an object with ignored field e.g.:

expected.MatchSnapshot(options => options.IgnoreField("Content"));

the field "Content" should not be serialized because may have a type which is not serializable.

Add support for image and binary files snapshots

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

FluentAssertion's "Subject" is not removed when matching a Dictionary

Describe the bug
Hi, I ran into a similar issue as #56. I'm trying to use FluentAssertions with a custom object that looks like this:

Dictionary<string,List<Dictionary<string,object>>

The snapshot looks like this:

{
  "Subject": {
    "Customer": [
      {
        "ID": 1,
        "Name": "Bla",
        "EmailAddress": "[email protected]",
        "ContactId": null,
        "CreatedAt": "2021-01-10T19:03:59.075874+00:00",
        "LastUpdatedAt": "2021-01-10T19:03:59.075874+00:00"
      }
    ]
  }
}

I set a breakpoint in the ObjectWrapperRemover and get the namespace FluentAssertions.Collections for the dictionary. Since this namespace is not part of the if condition, the Subject is not removed.

private static object RemoveFluentAssertionWrapper(this object objectToRemoveWrappers)
{
Type resultType = objectToRemoveWrappers.GetType();
if (resultType.Namespace != null
&& resultType.Name != null
&& resultType.Namespace.Equals("FluentAssertions.Primitives")
&& resultType.Name.Equals("ObjectAssertions"))
{
PropertyInfo prop = resultType.GetProperty("Subject");
object actualvalue = prop.GetValue(objectToRemoveWrappers);
return actualvalue;
}
return objectToRemoveWrappers;
}

To Reproduce
Steps to reproduce the behavior:

  1. Use FluentAssertions on a Dictionary<string,List<Dictionary<string,object>>
  2. Check the snapshot and see that "Subject" is the top level field

Expected behavior
"Subject" is removed

Tried on Snapshooter 0.5.8

Ability to customize file extension instead of `.snap` (e.g. `.html`)

Hello, we currently have a bunch of tests comparing html files. We'd like to use snapshooter to compare those files since we're already using snapshooter on other projects (this way we can have a consistent workflow). It would be nice if the generated snapshot file extension can be changed to .html (or any other extension), for convenience and also a "correct" file extension.
Thank you!

Add option to ignore the order of elements in arrays when using Snapshot.Match

Is your feature request related to a problem? Please describe.
I have the following json response and I want to ignore the order of the elements of metadata array.
I'm using the Snapshooter.Xunit version 0.5.2 which allows me to ignore some fields (e.g. ids) but the order of the elements should be ignored as the elements will checked

Renerated response

{
  "data": {
    "createItem": {
      "item": {
        "id": 1,
        "name": "Dummy item",        
        "metadata": [
          {
            "id": 1,            
            "key": "answer",
            "value": "42"
          },
		  {
            "id": 2,
            "key": "question",
            "value": "the answer to life, the universe and everything"
          }
        ]
      }
    }
  }
}

Snap file

{
  "data": {
    "createItem": {
      "item": {
        "id": 1,
        "name": "Dummy item",        
        "metadata": [
          {
            "id": 1,
            "key": "question",
            "value": "the answer to life, the universe and everything"
          },
	  {
            "id": 2,            
            "key": "answer",
            "value": "42"
          }
        ]
      }
    }
  }
}

Describe the solution you'd like
When matching arrays should be possible to set an option to ignore the order and the elements should be checks one by one to see if a matched element is found in the array.

Describe alternatives you've considered
Allow injecting a function in the Snapshot.Match( to order the arrays before the comparison is executed

Additional context
The failure message
Message:
Assert.Equal() Failure
Expected: ยทยทยท "key": "question",\n
Actual: ยทยทยท "key": "answer",\n

How can I check that an array is not empty?

  {
    "number": 100000,
    "id": "017c3679-fb15-4720-a046-d88f50f07e39",
    "date": "0001-01-01T00:00:00.000Z",
    "version": "Version 1",
    "arrayWithIds": [
      "017c0902b9b441c0aeb273b376390830",
      "017c0902c1034c17a4ee696505cf67ce"
    ]
  }

How can I check that arrayWithIds is not empty?

I tried:

.Assert(fieldOption => fieldOption.Field<Guid>("Data.myobject[*].arrayWithIds").Should().NotBeEmpty())
But that's giving me : ----> System.ArgumentException : Can not convert Array to Guid.

I tried with

.Assert(fieldOption => fieldOption.Field<Array>("Data.myobject[*].arrayWithIds").Should().NotBeEmpty())
But that's giving me: ----> Newtonsoft.Json.JsonSerializationException : Could not create an instance of type System.Array.

Trying with the first element worksData.myobject[*].arrayWithIds[0]
but with Data.myobject[*].arrayWithIds[*] it returns an error as well:

----> Snapshooter.Exceptions.SnapshotFieldException : The field of the path 'Data.myobject[*].arrayWithIds[*]' has an array as return value, Please use the FieldOption for fields array (Fields).

Cannot create snapshots on anonymous types.

Describe the bug
The sharpshooter has problems with anonymous types.

To Reproduce
The following code will lead to an exception.

new { foo = "bar" }.MatchSnapshot();

Using anonymous types is sometimes nice since we can define the structure of our snapshot data.

Expected behavior
There should be no exception and anonymous types should be treated like any other type.

Html compare

I need to compare 2 html email templates that contains a div element's with generated id's and date for debug purpose. The current comparison fails since this div element couldn't be ignored.

The compare engine is based now on Json. It would be nice to have a possibility to compare html with functionality like ignore html elements by id or class name or even XPath.

Snapshot MatchOption Exclude

At the moment the snapshot match option "ignore" serializes the value of a field of a snapshot and just ignores this value during comparison. Now we should have the possibility to exclude a certain field completely. The excluded field shall not be serialized within the snapshot and of course shall not be compared.

Example:
Snapshot.Match(object, matchOptions => matchOptions.ExcludeField(""))

Strict-Mode for Ci-Server

Is your feature request related to a problem? Please describe.
Currently when the snapshot is not existent snapshooter creates a new one without the test failing. This behavior is nice if you are developing on your local machine and speeds op development. However, if the test is run on a CI-Server, the test should fail if the snapshot does not exist.

Describe the solution you'd like
It should be possible to have an environment variable like SNAPSHOOTER_STRICT_MODE set which tells snapshooter to fail if the snapshot does not exist.

Mismatch error output not well formatted

The error message of the snapshot mismatch exception is not really readable, especially if you have large files.

The goal shall be to see the line number and the position for each mismatch within the comparison.

System.Text.Json support

Currently, snapshooter is using Newtonsoft.Json (also called Json.Net) serializer to compare JSON files.
Microsoft introduced its own JSON Serializer in the System.Text namespace, which is now recommended for simple use cases for .NET Core 3.1 and up.
However, the feature set is still far from the very poweful Newtonsoft.Json as the migration steps explains.
https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to

Especially, using MS serializer requires strictly formatted Json with "": non-string values for string properties are NOT allowed anymore.
For instance, {"str1": 1, "str2": true} is ok for Newtonsoft when linked with an object having only string, but raise Exception with MS which required {"str1": "1", "str2": "true"}

I just migrated one project from Newtonsoft to Text.Json.
For my unit tests using Snapshooter, the unit tests failed because Snapshooter internal routine using Newtonsoft is only reading the Newtonsoft.Json.JsonProperty attribute and ignore the replacement System.Text.Json.Serialization.JsonPropertyName

Thus, I could not 100% remove Newtonsoft dependency from my code and had to use both attributes as a workaround for Snapshooter, e.g. for one property in my model:

[System.Text.Json.Serialization.JsonPropertyName("deviceType")]
[Newtonsoft.Json.JsonProperty("deviceType")]
public string Type { get; set; }

Is there any plan to fully support System.Text.Json.Serialization on top of Newtonsoft.Json?
Or at some point get rid of Newtonsoft?

Extended Ignore Option for all fields with the same name.

Sometimes there are snapshots which contain several fields with the same name which we want to ignore..., for example "duration":

{
  "status": "Healthy",
  "duration": "00:00:00.0361599",
  "entries": {
    "SuspendedWorkflows": {
      "data": { "total": "0" },
      "duration": "00:00:00.0310035",
      "status": "Healthy"
    }
  }
}

Now, it would be nice to have a match option like IgnoreAnyField("xxxx"), which ignores then every field with the specific name or json path (in our example "duration").

SnapshotTestException is thrown when <DebugType> set to none

Describe the bug
When the DebugType project property is set to none snashooter throws a SnapshotTestException : Could not resolve directory for SnapShot.

[xUnit.net 00:00:00.50]     releaseModeSnapshot.UnitTest1.Test1 [FAIL]
  Failed releaseModeSnapshot.UnitTest1.Test1 [20 ms]
  Error Message:
   Snapshooter.Exceptions.SnapshotTestException : Could not resolve directory for SnapShot
  Stack Trace:
     at Snapshooter.LiveUnitTestingDirectoryResolver.CheckForSession(SnapshotFullName snapshotFullName)
   at Snapshooter.Xunit.XunitSnapshotFullNameReader.ReadSnapshotFullName()
   at Snapshooter.SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension)
   at Snapshooter.Snapshooter.ResolveSnapshotFullName(String snapshotName, SnapshotNameExtension snapshotNameExtension)
   at Snapshooter.Xunit.Snapshot.FullName()
   at Snapshooter.Xunit.Snapshot.Match(Object currentResult, Func`2 matchOptions)
   at Snapshooter.Xunit.Snapshot.Match[T](T currentResult, Func`2 matchOptions)
   at releaseModeSnapshot.UnitTest1.Test1()

To Reproduce

  1. add <DebugType>none</DebugType> to your csproj
  2. Run a test in this project which uses a snapshot

Expected behavior
If this is the expected behavior this should be documented. Either in the wiki or just through this ticket.

Extensions for Xunit's Assert and FluentAssertion's ObjectAssertions

Is your feature request related to a problem? Please describe.
Hi,

We're encountering the following warning in Visual Studio 19 when using Sonar for additional code quality checks.
image

The sonar rule is available here: https://rules.sonarsource.com/csharp/RSPEC-2699
As explained in their docu, they check for asserts from the following frameworks:

  • MSTest
  • NUnit
  • XUnit
  • FluentAssertions (4.x and 5.x)
  • NSubstitute

Describe the solution you'd like
For Fluent:
An Extension Method to assert Snapshots for fluent's ObjectAssertions class would probably do the trick.

myObject.Should().MatchSnapshot()

Note: today this already sort-of works, the OjectAssertions Object gets properly serialized and snapshot and only contains a wrapper around the object that we are actually interested in.

For XUnit's Assert:
TBD, probably not easily feasible.

Version 0.5.0 not on Nuget

Hello

We use Snapshooter in our projects and we love it. The new strict mode is something we are very interested in. Although it was released on Github, I cannot find the version on Nuget. Am I missing something.

Greetings

MatchOptions.IgnoreFields() throws SnapshotFieldException

Describe the bug
I'm trying to set the ignorefields with this syntax

result.MatchSnapshot(matchOptions=>matchOptions.IgnoreFields("$.[?(@.Result != null)].Result.Id"))

it failed on certain test cases

This case passes

[
  {
    "Error": "",
    "IsEmpty": false,
    "Result": {
      "Id": "cabcf393-e26e-4adb-af69-7125ca6774e8",
      // additional properties here abbreviated for concise
    },
    "ReplyCase": "Result"
  },
  {
    "Error": "",
    "IsEmpty": true,
    "Result": null,
    "ReplyCase": "IsEmpty"
  }
]

This case also passes

[
  {
    "Error": "",
    "IsEmpty": false,
    "Result": {
      "Id": "a9039bbb-3e65-485b-85ae-b752a8169401",
      // additional properties here abbreviated for concise
    },
    "ReplyCase": "Result"
  },
  {
    "Error": "",
    "IsEmpty": false,
    "Result": {
      "Id": "58e3b1fa-3d2b-4521-a241-6f58948ab411",
      // additional properties here abbreviated for concise
    },
    "ReplyCase": "Result"
  }
]

However, this case will fail

[
  {
    "Error": "",
    "IsEmpty": true,
    "Result": null,
    "ReplyCase": "IsEmpty"
  }
]

with this exception

Snapshooter.Exceptions.SnapshotFieldException
The fields of '$.[?(@.Result != null)].Result.Id' of the compare context caused an error. No fields of the path '$.[?(@.Result != null)].Result.Id' could not be found.

Expected behavior
all presented test cases should pass

Iterative test case with multiple Snapshot.Match fails in 0.5.4 but worked in 0.5.2

Describe the bug

We have a test case that consists of a for loop. In each iteration, an operation is performed and its result is compared to a snapshot. The name of the snapshot is dynamically created in each loop.

[Fact]
public void IterativeTest()
{
    const string nameBase = "TestCase";

    for (var i = 0; i < 2; i++)
    {
        var contents = $"this is iteration = {i}";

        var name = $"{nameBase}_{i}";

        Snapshot.Match(contents, Snapshot.FullName(SnapshotNameExtension.Create(name)));
    }
}

When I run this test with version 0.5.2, the files are created and compared as expected. When I upgrade to 0.5.4, the test case fails because in the second iteration, the contents are compared to the file of the first iteration.

In 0.5.2

  • i = 0: name = "TestCase_0", contents are compared to TestCase_0.snap -> successful
  • i = 1: name = "TestCase_1", contents are compared to TestCase_1.snap -> successful
  • -> test succeeds

In 0.5.4

  • i = 0: name = "TestCase_0", contents are compared to TestCase_0.snap -> successful
  • i = 1: name = "TestCase_1", contents are compared to TestCase_0.snap -> fails
  • -> test fails

To Reproduce

Steps to reproduce the behavior:

  1. Add the test case I posted above
  2. Run with 0.5.2, will work
  3. Run with 0.5.4, will fail

Expected behavior

The test case should not fail.

Desktop (please complete the following information):

  • OS: macOS Catalina
  • Version: 10.15.5

0.6.0 throws ReflectionTypeLoadException | Could not load file or assembly Microsoft.VisualStudio.Coverage.CoreLib.Net Version=16.9.0.0

0.6.0 seems to have introduced an exception when using the latest version of Microsoft.NET.Test.Sdk (16.9.1). There is no exception when Microsoft.NET.Test.Sdk is downgraded to 16.8.3 or when Snapshooter is downgraded to 0.5.8.

I created a sample repo that reproduces the issue: https://github.com/gojanpaolo/snapshooter-0.6.0-visualstudio-coverage-bug

Below are the results for Snapshooter 0.5.8 and 0.6.0 by running dotnet test (issue also happens when tests are ran in Visual Studio)

0.5.8

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 20 ms - snapshooter-0.5.8.dll (net5.0)

0.6.0

  Failed snapshooter_0_6_0.Tests.Test [21 ms]
  Error Message:
   System.Reflection.ReflectionTypeLoadException : Unable to load one or more of the requested types.
Could not load file or assembly 'Microsoft.VisualStudio.Coverage.CoreLib.Net, Version=16.9.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
  Stack Trace:
     at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeModule.GetTypes()
   at System.Reflection.Assembly.GetTypes()
   at Snapshooter.Core.Serialization.GlobalSnapshotSettingsResolver.<>c.<GetConfiguration>b__0_0(Assembly s)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Snapshooter.Core.Serialization.GlobalSnapshotSettingsResolver.GetConfiguration()
   at Snapshooter.Core.Serialization.SnapshotSerializer.GetSettings(ISnapshotSettingsResolver snapshotSettingsResolver)
   at Snapshooter.Core.Serialization.SnapshotSerializer..ctor(ISnapshotSettingsResolver settingsResolver)
   at Snapshooter.Xunit.Snapshot.get_Snapshooter()
   at Snapshooter.Xunit.Snapshot.Match(Object currentResult, Func`2 matchOptions)
   at Snapshooter.Xunit.SnapshotExtension.MatchSnapshot(Object currentResult, Func`2 matchOptions)
   at snapshooter_0_6_0.Tests.Test() in C:\@dev\snapshooter-0.6.0-visualstudio-coverage-bug\snapshooter-0.6.0\Tests.cs:line 8

Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 21 ms - snapshooter-0.6.0.dll (net5.0)

Desktop (please complete the following information):

  • OS: Windows 10 Pro
  • Version 1909

Snapshot.Match() does not work with Visual Studio Live Unit Testing

Describe the bug
When using snapshooter with VS.NET Live Unit Testing following message is written to output

Message: 
    Snapshooter.Exceptions.SnapshotTestException : Could not resolve directory for SnapShot
Stack Trace: 
    LiveUnitTestingDirectoryResolver.CheckForSession(SnapshotFullName snapshotFullName)
    XunitSnapshotFullNameReader.ReadSnapshotFullName()
    SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension)
    Snapshooter.ResolveSnapshotFullName(String snapshotName, SnapshotNameExtension snapshotNameExtension)
    Snapshot.FullName()
    Snapshot.Match(Object currentResult, Func`2 matchOptions)
    SnapshotExtension.MatchSnapshot(Object currentResult, Func`2 matchOptions)
    SchemaTests.Ensure_Schema_IsCorrect()
--- End of stack trace from previous location where exception was thrown ---

To Reproduce
Steps to reproduce the behavior:

  1. Write an unit test with snapshot comparison
  2. Enable live unit testing
  3. See error

Expected behavior
I would expect the test to complete.

Desktop (please complete the following information):

  • OS: Win10
  • VS.NET Version 16.7.2
  • Snapshooter.Xunit 0.5.6

Running snapshot test in release mode fails

Describe the bug
When running a snapshot test in the release configuration it fails with the following error:

Test.Tests.SnapshooterTest.MultiplySnapshotTest

Snapshooter.Exceptions.SnapshotTestException : The snapshot full name could not be evaluated. This error can occur, if you use the snapshot match within a async test helper child method. To solve this issue, use the Snapshot.FullName directly in the unit test to get the snapshot name, then reach this name to your Snapshot.Match method.
   at Snapshooter.Xunit.XunitSnapshotFullNameReader.ReadSnapshotFullName() in /home/vsts/work/1/s/src/Snapshooter.Xunit/XunitSnapshotFullNameReader.cs:line 54
   at Snapshooter.SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension) in /home/vsts/work/1/s/src/Snapshooter/SnapshotFullNameResolver.cs:line 69
   at Snapshooter.Xunit.Snapshot.Match(Object currentResult, Func`2 matchOptions) in /home/vsts/work/1/s/src/Snapshooter.Xunit/Snapshot.cs:line 145

To Reproduce

  1. Create this class and it's corresponding test:
public class Multiplier
{
    public float MultiplyByThree(float input)
    {
        return input * 3;
    }
}
public class SnapshooterTest
{
    [Fact]
    public void MultiplySnapshotTest()
    {
        var multiplier = new Multiplier();
        float result = multiplier.MultiplyByThree(5);
        
        Snapshot.Match(result);
    }
}
  1. Run the test (debug configuration)
  2. Verify your snapshot
  3. Run the test again (release configuration)

Expected behavior
The test should pass both with the debug and the release configuration.

Actual behavior
The test only passed in debug mode.

Environment:

  • OS: macOs Mojave
  • Package Versions:
    • Snapshooter: 0.4.5
    • xUnit: 2.4.1

snapshooter FullNameReader exception with xunit

Describe the bug

Message: 
    System.ArgumentNullException : Value cannot be null. (Parameter 'source')
  Stack Trace: 
    ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument)
    Enumerable.Any[TSource](IEnumerable`1 source)
    XunitSnapshotFullNameReader.IsTheoryTestMethod(MemberInfo method) line 87
    XunitSnapshotFullNameReader.IsXunitTestMethod(MemberInfo method) line 75
    XunitSnapshotFullNameReader.ReadSnapshotFullName() line 32
    SnapshotFullNameResolver.ResolveSnapshotFullName(String snapshotName, String snapshotNameExtension) line 69
    Snapshooter.ResolveSnapshotFullName(String snapshotName, SnapshotNameExtension snapshotNameExtension) line 84
    Snapshot.FullName() line 252
    Snapshot.Match(Object currentResult, Func`2 matchOptions) line 145
    Snapshot.Match[T](T currentResult, Func`2 matchOptions) line 29
    CallSite.Target(Closure , CallSite , Type , Object )
    UpdateDelegates.UpdateAndExecuteVoid2[T0,T1](CallSite site, T0 arg0, T1 arg1)
    PlatformIntegration.Validate_GraphQL_Introspection() line 158
    --- End of stack trace from previous location where exception was thrown ---

To Reproduce
we get a lot the error above with xunit tests in Visual Studio 2019 (16.4) and .Net Core 3.1

it looks like the issue is here:

return method?.GetCustomAttributes(typeof(FactAttribute)).Any() ?? false;

GetCustomAttributes might be null and Any() throws an exception then

Sign assembly

To reference this library in a signed assembly it has to be also signed.

Diff output instead of Assert.Equals

Is your feature request related to a problem? Please describe.
I would like to see a complete diff between actual and expected snapshot. Currently Assert.Equals is showing only the first position where the string is different.

Describe the solution you'd like
Using an known package DiffPlex we can print the entire snapshot diff in case when they are different.

Snapshot includes Subject of FluentAssertions

Describe the bug
If you use Snapshooter with FluentAssertions the snapshot includes properties which are not part of my object under test.

To Reproduce

  1. Write a test like
[Fact]
public void CreatePerson_VerifyPersonBySnapshot_SuccessfulVerified()
{
    // arrange
    var serviceClient = new ServiceClient();

    // act
    TestPerson person = serviceClient.CreatePerson("David", "Mustermann");

    // assert
    person.Should().MatchSnapshot();
}
  1. Check the created snapshot
{
  "Subject": {
    "Firstname": "David",
    "Lastname": "Mustermann"
  }
}
  1. realize that the snapshot includes the Subject Property of FluentAssertions.

Expected behavior
Snapshooter should remove this property("Subject") to make the snapshot look as if it was made without using FluentAssertions.

{
  "Firstname": "David",
  "Lastname": "Mustermann"
}

Packageversions used:

  • Snapshooter.Xunit 0.4.5

Multiple asserts in one assert context do not work

results.MatchSnapshot(matchOptions =>
            {
                return matchOptions.Assert(fieldOption =>
                {
                    Assert.Equal(sessionId, fieldOption.Field<string>("[0].Data.onPublishSchema.sessionId"));
                    Assert.Equal(sessionId, fieldOption.Field<string>("[1].Data.onPublishSchema.sessionId"));

                    /*
                    Assert.Collection(
                        fieldOption.Fields<string>("[*].Data.onPublishSchema.sessionId"),
                        t => Assert.Equal(sessionId, t),
                        t => Assert.Equal(sessionId, t));*/
                });
            });

Snapshot.Match not detected as Assert

Describe the bug
Im using Snapshot.Match(object) to assert a test. But I get the warning S2699: Tests should include assertions
. Do I have to ignore that warning or is there a reason why it not detected?

To Reproduce
Steps to reproduce the behavior:

  1. Create a Testclass
  2. Add a test with only a Match.Snapshot()
  3. Warning apears depending on your analyzer settings

Expected behavior
No warning since Snapshot.Match is an assertion for me

Desktop (please complete the following information):

  • OS: Windows 10
  • Version 20H2

Doesn't work with xunit TheoryAttribute tests

Describe the bug
Doesn't work with xunit TheoryAttribute tests

To Reproduce
Steps to reproduce the behavior:
Create a test like this...

        [Theory]
        [InlineData("a")]
        [InlineData("b")]
        [InlineData("c")]
        [InlineData("d")]
        public void DummyTest(string data)
        {
            Snapshot.Match(data);
        }

Expected behavior
A snapshot should be created for each variation of input data.

Additional context
This is a pretty tricky scenario to handle. The inline data could be pretty much anything.

The latest version no longer supports .NET Core 2.1

Describe the bug
I'm using Snapshooter for some tests in a project that targets .NET Core 2.1 and 3.1. In the latest 0.5.7 release of Snapshooter.Xunit .NET Core 2.1 is no longer supported.

This change was made in Snapshooter for NUnit tests. (#110). The target framework was changed from netstandard2.0 to netstandard2.1 which isn't supported when targeting netcoreapp2.1.

It was probably an intentional change , but I wanted to check as it was just a patch version bump.

To Reproduce
Steps to reproduce the behavior:
Failing tests can be viewed here CycloneDX/cyclonedx-dotnet#247
You can also clone that repo locally if needed. dotnet test will run tests for 2.1 and 3.1. Although you need .NET Core 2.1 and 3.1 installed to run the tests. Although there is a devcontainer configuration. Repro by updating the Snapshooter.Xunit package references.

Expected behavior
Continued support for .NET standard 2.0 or major version bumps for breaking changes (with the caveat that right now this is still on 0.x.x versioning, so I can understand breaking changes coming in like this until a 1.0.0 release).

Desktop (please complete the following information):

  • OS: Linux
  • Version Ubuntu 20.04

Additional context
N/A

schema/structure snapshot

Is your feature request related to a problem? Please describe.
quite often dealing with highly dynamic data, especially JSON data, it's quite complicated to track them with snapshooter as we would need to ignore a lot of fields or write specific logic to validate data. a way to compare structure instead of content would come a long way.

Describe the solution you'd like
the solution could use something like Newtonsoft.Json.Schema and snapshot the schema instead of the content.

Describe alternatives you've considered
Today we are using Newtonsoft.Json.Schema and do all the work around snapshotting the schema by hand. While this works, the usability of snapshooter would come in handy when properly integrated.

Support for MsTest

Hey folks, does this library support MsTest framework. On the website I see that, support should come soon, but I am not sure if that information is up to date. If not are there any plans to do it?
image

Thanks!

Snapshot file name is not in format InvariantCulture

The snapshot file name is culture info sensitive. This means the file names of the snapshots can be different on systems with different culture info configured (de, en, ch....). This should not be, because the name of the snapshot file must be always the same, not depending on the configured system.

I propose to have the snapshot name always in the culture info "invariant" (CultureInfo.InvariantCulture). Therefore the name of the snapshot is everywhere the same.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.