Giter Site home page Giter Site logo

json-snapshot.github.io's Introduction

Purpose of Snapshot Testing

Snapshots help figuring out whether the output of the modules covered by tests is changed, without doing tons of asserts!

When is it usefull?

It's usefull for deterministic tests. That is, running the same tests multiple times on a component that has not changed should produce the same results every time. You're responsible for making sure your generated snapshots do not include platform specific or other non-deterministic data.

A Json Snapshot test does not assert Java types. You can continue doing that with any other testing framework.

Based on facebook's Jest framework

GitHub Repository

How to install using Maven

Add to your pom.xml dependencies section:

<dependency>
    <groupId>io.github.json-snapshot</groupId>
    <artifactId>json-snapshot</artifactId>
    <version>1.0.17</version>
</dependency>

Usage

package com.example;

import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.runners.MockitoJUnitRunner;

import java.util.Arrays;
import java.util.List;

import static io.github.jsonSnapshot.SnapshotMatcher.*;
import static io.github.jsonSnapshot.SnapshotUtils.*;
import io.github.jsonSnapshot.SnapshotCaptor;

@RunWith(MockitoJUnitRunner.class)
public class ExampleTest {

    @Mock
    private FakeObject fakeObject;


    @BeforeClass
    public static void beforeAll() {
        start();
    }
    
    @AfterClass
    public static void afterAll() {
        validateSnapshots();
    }
    
    @Test // Snapshot any object
    public void shouldShowSnapshotExample() {
        expect("<any type of object>").toMatchSnapshot();
    }

    @Test // Snapshot arguments passed to mocked object (from Mockito library)
    public void shouldExtractArgsFromMethod() {
        fakeObject.fakeMethod("test1", 1L, Arrays.asList("listTest1"));
        fakeObject.fakeMethod("test2", 2L, Arrays.asList("listTest1", "listTest2"));

        expect(extractArgs(fakeObject, "fakeMethod", new SnapshotCaptor(String.class), new SnapshotCaptor(Long.class), new SnapshotCaptor(List.class)))
                .toMatchSnapshot();
    }
    
    @Test // Snapshot arguments passed to mocked object support ignore of fields
    public void shouldExtractArgsFromFakeMethodWithComplexObject() {
        FakeObject fake = new FakeObject();
        fake.setId("idMock");
        fake.setName("nameMock");

        //With Ignore
        fakeObject.fakeMethodWithComplexObject(fake);
        Object fakeMethodWithComplexObjectWithIgnore = extractArgs(
                fakeObject, "fakeMethodWithComplexObject", 
                new SnapshotCaptor(Object.class, FakeObject.class, "name"));

        Mockito.reset(fakeObject);

        // Without Ignore of fields
        fakeObject.fakeMethodWithComplexObject(fake);
        Object fakeMethodWithComplexObjectWithoutIgnore = extractArgs(
                fakeObject, "fakeMethodWithComplexObject", 
                new SnapshotCaptor(Object.class, FakeObject.class));

        expect(fakeMethodWithComplexObjectWithIgnore, fakeMethodWithComplexObjectWithoutIgnore).toMatchSnapshot();
    }
    
    class FakeObject {
        
        private String id;

        private Integer value;

        private String name;

        void fakeMethod(String fakeName, Long fakeNumber, List<String> fakeList) {

        }
        
        void fakeMethodWithComplexObject(Object fakeObj) {
        
        }
        
        void setId(String id) {
            this.id = id;
        }
        
        void setName(String name) {
            this.name = name;
        }
    }
}

When the test runs for the first time, the framework will create a snapshot file named ExampleTest.snap alongside with your test class. It should look like this:

com.example.ExampleTest.shouldShowSnapshotExample=[
    "<any type of object>"
]


com.example.ExampleTest.shouldExtractArgsFromMethod=[
  {
    "FakeObject.fakeMethod": [
      {
        "arg0": "test1",
        "arg1": 1,
        "arg2": [
          "listTest1"
        ]
      },
      {
        "arg0": "test2",
        "arg1": 2,
        "arg2": [
          "listTest1",
          "listTest2"
        ]
      }
    ]
  }
]


com.example.ExampleTest.shouldExtractArgsFromFakeMethodWithComplexObject=[
  {
    "FakeObject.fakeMethodWithComplexObject": [
      {
        "arg0": {
          "id": "idMock"
        }
      }
    ]
  },
  {
    "FakeObject.fakeMethodWithComplexObject": [
      {
        "arg0": {
          "id": "idMock",
          "name": "nameMock"
        }
      }
    ]
  }
]

Whenever it runs again, the expect method argument will be automatically validated with the .snap file. That is why you should commit every .snap file created.

Inheritance

Test classes inheritance becames usefull with snapshot testing due to the fact that the assertions are variable following snasphots, instead of code. To make usage of this benefit you should be aware of the following:

Start SnapshotMatcher on child classes only:

package com.example;

import org.junit.AfterClass;
import org.junit.BeforeClass;

import static io.github.jsonSnapshot.SnapshotMatcher.start;
import static io.github.jsonSnapshot.SnapshotMatcher.validateSnapshots;

public class SnapshotChildClassTest extends SnapshotSuperClassTest {

    @BeforeClass
    public static void beforeAll() {
        start();
    }

    @AfterClass
    public static void afterAll() {
        validateSnapshots();
    }
    
    @Override
    public String getName() {
        return "anyName";
    }
}

Super classes can have @Test defined, but you should make the class abstract.

package com.example;

import org.junit.Test;

import static io.github.jsonSnapshot.SnapshotMatcher.expect;

public abstract class SnapshotSuperClassTest {

    public abstract String getName();

    @Test
    public void shouldMatchSnapshotOne() {
        expect(getName()).toMatchSnapshot();
    }

}

json-snapshot.github.io's People

Contributors

andrebonna avatar dependabot[bot] avatar grimsa avatar maguro avatar metas-ts avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

json-snapshot.github.io's Issues

Documentation example does not work

Trying to use the example code of Readme file that uses inner class FakeObject inside the test class throw this exception:

io.github.jsonSnapshot.SnapshotMatchException: com.example.elastic.ExampleTest$FakeObject$MockitoMock$352055545.fakeMethod(java.lang.String, java.lang.Long, java.util.List)


at io.github.jsonSnapshot.SnapshotUtils.process(SnapshotUtils.java:80)
at io.github.jsonSnapshot.SnapshotUtils.extractArgs(SnapshotUtils.java:28)
at com.example.elastic.ExampleTest.shouldExtractArgsFromMethod(ExampleTest.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.mockito.internal.runners.DefaultInternalRunner$1.run(DefaultInternalRunner.java:79)
at org.mockito.internal.runners.DefaultInternalRunner.run(DefaultInternalRunner.java:85)
at org.mockito.internal.runners.StrictRunner.run(StrictRunner.java:39)
at org.mockito.junit.MockitoJUnitRunner.run(MockitoJUnitRunner.java:163)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

If I create a different file for this class without using inner approach works fine (the same as in test folder of your project). Is there any point of my first test using your library? Thank very much for your support.

Ignore order of fields in snapshot.

The snapshot is serialized as JSON. JSON does not guarantee ordering of object fields.

RFC7159

An object is an unordered collection of zero or more name/value pairs

However, the snapshot test fails if the order of fields in the serialized JSON has changed.

Snapshot.java#L42

if (!rawSnapshot.trim().equals(currentObject.trim()))

Getting class not found error at Start Method of snapshot class

Hi team,
Was trying to use this library and while calling Start() methog in @before hook for cucumber, im getting this error.
Pfa the pic
IMG_20210602_191322
Com/fasterxml/jackson/core/util/JacksonFeature class not found.

Have jackson libraries in POM of version 2.9.8 for jackson-core, Jackson-annotation and jackson-data.

Is there anything that im doing it wrong

[JUnit5] Support non-public test methods

JUnit5 supports package-private test methods, and IntelliJ IDEA suggests to reduce visibility where possible:
image

This makes json-snapshot fail with the following exception:

io.github.jsonSnapshot.SnapshotMatchException: Please annotate your test method with @Test and make it without any parameters!

	at io.github.jsonSnapshot.SnapshotMatcher.getMethod(SnapshotMatcher.java:156)
	at io.github.jsonSnapshot.SnapshotMatcher.expect(SnapshotMatcher.java:90)

[Windows] Comparison with stored snapshot fails due to line endings

Preconditions:

  • Use default json-snapshot configuration

Steps:

  1. Run a test to create snapshot
  2. Run a test again

Expected: test passess
Actual: test fails

Symptom: in Snapshot#toMatchSnapshot rawSnapshot contains lines separated by \n, while in currentObject lines are separated by \r\n.

This generates a failure with every line being different by just the \r symbol.

Root cause: SnapshotFile constructor uses hardcoded \n (line 33)
Possible workaround: use com.fasterxml.jackson.core.util.DefaultIndenter.SYS_LF instead (both there and in SPLIT_STRING).

is this project alive?

Hi,

is this project still alive?
I've noticed some outstanding PRs and no new releases.

Any help needed?

Thanks

Add support for meta JUnit annotations

  private static boolean hasTestAnnotation(Method method) {
    return method.isAnnotationPresent(Test.class)
        || method.isAnnotationPresent(BeforeClass.class)
        || method.isAnnotationPresent(org.junit.jupiter.api.Test.class)
        || method.isAnnotationPresent(BeforeAll.class);
  }

does not allow for meta annotations annotated with@Test.

Does this work with TestNG?

I'm using TestNG instead of JUnit and I'm encountering some errors:

Gradle suite > Gradle test > com.snapshottest.demo.DemoApplicationTests > beforeAll FAILED
    java.lang.IllegalArgumentException at DemoApplicationTests.java:20
        Caused by: java.lang.ClassNotFoundException at DemoApplicationTests.java:20

Sample test class below:

package com.snapshottest.demo;

import io.github.jsonSnapshot.SnapshotMatcher;
import io.restassured.http.ContentType;
import io.restassured.RestAssured;
import io.restassured.parsing.Parser;
import static io.restassured.RestAssured.given;

import org.testng.annotations.Test;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.AfterClass;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@SpringBootTest(classes = SpringRunner.class)
public class DemoApplicationTests {
  @BeforeClass
  public static void beforeAll() {
    SnapshotMatcher.start();
  }

  @AfterClass
  public static void afterAll() {
    SnapshotMatcher.validateSnapshots();
  }

	@Test
	public void contextLoads() {
  }

  @Test 
  public void dummyApiHas200() {
    RestAssured.defaultParser = Parser.JSON;

    String response = given()
      .contentType(ContentType.JSON)
      .when()
      .get("http://dummy.restapiexample.com/api/v1/employees")
      .then()
      .statusCode(200)
      .extract()
      .response()
      .asString();
    
    SnapshotMatcher.expect(response).toMatchSnapshot();
  }
}

[JUnit 5] Add support for nested tests

Nested test docs: https://junit.org/junit5/docs/current/user-guide/#writing-tests-nested

Currently snapshot matching within nested test classes is not supported.

Example:

class Demo {
    @BeforeAll
    static void beforeAll() {
        SnapshotMatcher.start();
    }

    @AfterAll
    static void afterAll() {
        SnapshotMatcher.validateSnapshots();
    }

    @Test
    void one() {
        expect("one").toMatchSnapshot();       // succeeds
    }

    @Nested
    class NestedTest {
        @Test
        void two() {
            expect("two").toMatchSnapshot();         // fails
        }

        @Nested
        class DeeplyNestedTest {
            @Test
            void three() {
                expect("three").toMatchSnapshot();       // fails
            }
        }
    }
}

Stack trace:

io.github.jsonSnapshot.SnapshotMatchException: Could not find method two on class class java.lang.Object
Please annotate your test method with @Test and make it without any parameters!

	at io.github.jsonSnapshot.SnapshotMatcher.lambda$getMethod$4(SnapshotMatcher.java:174)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at io.github.jsonSnapshot.SnapshotMatcher.getMethod(SnapshotMatcher.java:172)
	at io.github.jsonSnapshot.SnapshotMatcher.lambda$getMethod$3(SnapshotMatcher.java:171)
	at java.util.Optional.map(Optional.java:215)
	at io.github.jsonSnapshot.SnapshotMatcher.getMethod(SnapshotMatcher.java:171)
	at io.github.jsonSnapshot.SnapshotMatcher.expect(SnapshotMatcher.java:99)

Unnecessary transitive dependencies pulled in through this lib

When json-snapshot is added to a project, it pulls a number of dependencies (see output of mvn dependency:tree:

[INFO] \- io.github.json-snapshot:json-snapshot:jar:1.0.16:test
[INFO]    +- org.apache.commons:commons-lang3:jar:3.7:test
[INFO]    +- org.mockito:mockito-junit-jupiter:jar:2.23.0:test
[INFO]    +- org.junit.platform:junit-platform-runner:jar:1.2.0:test
[INFO]    |  +- org.junit.platform:junit-platform-launcher:jar:1.2.0:test
[INFO]    |  \- org.junit.platform:junit-platform-suite-api:jar:1.2.0:test
[INFO]    \- org.junit.vintage:junit-vintage-engine:jar:5.1.0:test

Most of them should not be needed on the test classpath of the project using this lib.
E.g. I want to use JUnit 5 in my project and NOT have JUnit 4 on classpath (to avoid accidentally mixing up annotations from different versions). This is currently not possible, as if I exclude JUnit 4 from json-snapshot dependency, then io.github.jsonSnapshot.SnapshotMatcher#hasTestAnnotation fails.

The check which classes are available on classpath should be string-based (i.e. if the annotation class is not available, json-snapshot should simply not look for that on test methods).

Ignore certain snapshot fields.

Allow snapshot testing of objects with varying content.
E.g. the object may contain a randomly generated id or created_at timestamps.

#7 contains a simple implementation that allows ignoring nested fields. It only works if the object under test is a nested java.util.Map.

Snapshot of methods with parameters

Currently json-snapshot only works with test methods without parameters. Change reflection parser to snapshot test methods with any number of parameters.

Please release latest json ignore order changes

Latest release version is 1.0.17, after that there are commits for json order ignore changes. Please release new version with these commits. Also, add JSONAssertConfig.java class to use JSON AssertMatching Strategy for comparison.

Warning about unused snapshot doesn't need to show all the contents

Context:

  • I have a class with multiple snapshots but often times, I only want to run a single test when I'm troubleshooting an issue or working on new functionality.
  • I've possibly mixed in non-snapshot tests with my snapshot tests.

Right now, when you have unused snapshots, it displays the entire contents of that unused snapshot. I think just printing a a list of all those unused methods is good enough.

Snapshot failing even though nothing changed

  {
    "businessContactSelected": false,
    "machineConfigHasChanged": false,
    "machineHeaderHasChanged": false,
    "cancellable": false,
    "editable": false,
    "selected": false,
    "machineSelected": false,
    "isFeatured": false,
    "isBranch": false,
    "failedApplySelections": false,
    "invalidConfiguration": false,
    "purchaseOrderNumber": "",
    "sapOrderNumber": "",
    "isIsgAgreementAccepted": false,
    "isBranchStock": false,
    "isAttachment": false,
    "referenceComment": "",
    "linkType": "",
    "featured": false,
    "branch": false,
    "branchStock": false
  }
]

Missing content at line 21:
  ["    "branch": false,"]

Extra content at line 23:
  ["    "branch": false,"]

How is this possible that test is failing?

Test method lookup fails when method has parameters

This reproduces the problem:

class TestX {

    @BeforeAll
    static void beforeAll() {
        SnapshotMatcher.start();
    }

    @AfterAll
    static void afterAll() {
        SnapshotMatcher.validateSnapshots();
    }

    @Test
    void testName() {
        testBasedOnArgs("a");
    }

    private void testBasedOnArgs(String arg) {
        expect(arg).toMatchSnapshot();
    }
}

Root cause: method name is taken from StackTraceElement and then used in lookup using Class#getMethod() or Class#getDeclaredMethod(). However, if parameter types are not provided, method will not be found.

Solution could be to get all methods via Class#getDeclaredMethods() and then match method by name.

Lib can not be used with Jackson 2.10.0 or later

Jackson 2.10 was released 2019-09-26.
It contains a fix for FasterXML/jackson-core#502, which makes snapshot building fail with the following exception:

io.github.jsonSnapshot.SnapshotMatchException: Failed `createInstance()`: io.github.jsonSnapshot.SnapshotMatcher$1 does not override method; it has to

	at io.github.jsonSnapshot.SnapshotMatcher.lambda$defaultJsonFunction$0(SnapshotMatcher.java:116)
	at io.github.jsonSnapshot.Snapshot.takeSnapshot(Snapshot.java:87)
	at io.github.jsonSnapshot.Snapshot.toMatchSnapshot(Snapshot.java:43)

The cause of this is io.github.jsonSnapshot.SnapshotMatcher#buildDefaultPrettyPrinter where DefaultPrettyPrinter is extended, but now required createInstance method is not overridden.

This makes serializing JSON fail with the following stack trace:

java.lang.IllegalStateException: Failed `createInstance()`: io.github.jsonSnapshot.SnapshotMatcher$1 does not override method; it has to
	at com.fasterxml.jackson.core.util.DefaultPrettyPrinter.createInstance(DefaultPrettyPrinter.java:256)
	at com.fasterxml.jackson.core.util.DefaultPrettyPrinter.createInstance(DefaultPrettyPrinter.java:15)
	at com.fasterxml.jackson.databind.ObjectWriter$GeneratorSettings.initialize(ObjectWriter.java:1299)
	at com.fasterxml.jackson.databind.ObjectWriter._configureGenerator(ObjectWriter.java:1174)
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1129)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:1005)
	at io.github.jsonSnapshot.SnapshotMatcher.lambda$defaultJsonFunction$0(SnapshotMatcher.java:114)

Feature request: Add JUnit 5 extension that invokes start/validateSnapshots

Currently the following block has to be duplicated in every test.

    @BeforeAll
    static void beforeAll() {
        SnapshotMatcher.start();
    }

    @AfterAll
    static void afterAll() {
        SnapshotMatcher.validateSnapshots();
    }

It should be possible to add a simple JUnit 5 extension that would handle this.

Note for implementation:
JUnit 5 compile dependency should be optional - i.e. projects using JUnit 4 or an entirely different testing framework should not get transitive JUnit 5 jars.

Avoid EOL problems when loading snapshot file

I had problems with the SPLIT_STRING constant in SnapshotFile.
I believe it related to the way i configured git's EOL-Settings.
At any rate, sometimes my ".snap" files end up on my disk with r\n as their EOL string.
If this happened, the `SPLIT_STRING="\n\n\n" is not spliting the file's different snapshots properly.

Change extension `.snap`

I use intellij Idea and in there the .snap extension is already allocated, and i wouldn't like to change the default association.
Is there a way to change the extension from .snap to .json-snapshot or something else?

In this way i would just associate in idea the new extension as a json file type and have the highlighting work ootb for my whole team.

Support parameterised tests

Hi,

When using JUnit's Parameterized test runner, snapshot testing can not be used, as only one snapshot per test method is permitted due to the way that the snapshot is resolved on subsequent runs.

I've tried a small change to support passing qualifiers through to the matcher:

private String someParameter;

public ATestClass(String someParameter) throws IOException {
	this.someParameter = someParameter;
}

@Test
public void test() throws Exception {

	Object actual = ...

	expect(actual).toMatchSnapshot(someParameter);
}

and

public void toMatchSnapshot(Object... qualifiers) {

This seems to work pretty well. The .snap ends up as you would expect:

ATestClass.test(theValueOfSomeParameter)=[

I'm happy to raise a PR if this would be of interest.

Thanks,
-Dan

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.