sskorol/test-data-supplier

TestNG DataProvider on steroids

test-data-supplier's Issues

Resolve outdated dependencies

AssertJ core requires updating.

Downgrade Reflection package

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.9.3
TestNG	7.4.0
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

Tests that use DS should fail due to DS internals.

What is the actual behavior?

ronmamo/reflections#277

What is the motivation / use case for changing the behavior?

Broken release.

Add ability to turn off SPI

Hi. We would like to have an ability to disable SPI in test-data-supplier.
Motivation: In our project we already have SPI with AnnotationTransformer. Adding test-data-supplier will break our tests because TestNG doesn't allow more than one annotation transformer enabled.

Possible solution: I spotted an interesting feature in allure-testng library. It allows to disable SPI by specifying 'spi-off' classifier in dependencies.
https://github.com/allure-framework/allure-java/blob/master/allure-testng/build.gradle.kts

It would be great if test-data-supplier has something similar. We love this library but due to SPI conflict we are forced to use old version without SPI: 1.4.0

Note: We understand that without SPI test-data-supplier's annotation transformer would require manual handling but we're ok with that.

Java 9 support - illegal reflective access issue due to joor

While this issue is due to joor, I'd like to make it general for test-data-supplier to support JDK9.

Even if you won't be able to tackle it (for any reasons at all) I'd still want to thank you for creating this. :-)

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.7.0
TestNG	6.14.3
Build tool	Maven 3.5
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

No JDK9-related warnings.

What is the actual behavior?

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.joor.Reflect (file:/home/tammo/.m2/repository/org/jooq/joor-java-8/0.9.8/joor-java-8-0.9.8.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class)
WARNING: Please consider reporting this to the maintainers of org.joor.Reflect
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

What is the motivation / use case for changing the behavior?

JDK11 is just around the corner. JUnit 5 becomes an alternative for my beloved TestNG. :-) We can't have that!

Let me know if you'd like me to provide any details on, e.g., how I got the issue.

[FR] Add basic YAML processing support

Hi,

Aftrer #62 and #63 it would be grate to add YAML processing support

UseCase:

1. List of objects is stored in the following YAML format:

---
 username: admin
 password: admin
---
 username: sskorol
 password: password
---
 username: guest
 password: '123'

The reason for usage "---" (start sign of YAML document) as object delimiter is complex objects with several nested structures.
Here is the Jakson parser exapmple:

fun getMapper(): ObjectMapper {
   return ObjectMapper(YAMLFactory()).registerModule(KotlinModule())
           .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false)
}
...
val result: ArrayList<T> = arrayListOf()
val data = YamlConverter().getMapper().readerFor(T::class.java)
                .readValues<T>(it)
while (data.hasNextValue()) {
            result.add(data.nextValue())
}

Add DS analysis on subclass level

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.4.1
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

See TestNG-1691

What is the actual behavior?

TestNG / DS ignores subclass level annotations.

What is the motivation / use case for changing the behavior?

See TestNG-1691

Other information

Most likely it requires additional libs involvement for subtypes scanning in runtime.

Changelog support

Add a gradle plugin for generating changelogs.

IntelliJ Idea 2019.3 compatability

How to use test data supplier with IntelliJ Idea 2019.3?

Add basic CSV / JSON processing support

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.5.5
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

User should be able to read data from CSV / JSON files / urls into Java entities with minimal effort.

What is the actual behavior?

There're no native CSV / JSON processors.

What is the motivation / use case for changing the behavior?

Simplify users' scenarios for accessing data.

Other information

Create an utility class for getting CSV / JSON data via apache poi and gson libraries.

Update README with integration details

Add more info about gradle / maven configuration.

Dependencies update

New vavr version has been released.

IntelliJ Idea 2020 compatability

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.7.0
TestNG	7.0.0
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

AspectJ update

Migrate to 1.8.11 version.

Update dependencies

Current dependencies are outdated. Apart from that, there were some TestNG issues fixed in the latest release. It's better to stay up to date.

Custom names support

Add an ability to specify custom DataSupplier names.

Rft transformer

Split transformation logic between model and utility classes.

Add indices support

To make DataSupplier similar to DataProvider, it's required to implement indices[] arg support.
However, the same functionality could be achieved with streams.

Add Javadocs

All Javadocs are suppressed at moment. Need to add more details for existing API.

TestNG 7.2.0 is missing as dependency for data supplier

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.9.2
TestNG	7.4.0
Build tool	[email protected]

Is the issue reproducible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

To compile project.

What is the actual behavior?

At the moment test-data-supplier is depending on testNG 7.2.0 which is missing from mvn repository: https://mvnrepository.com/artifact/org.testng/testng Maybe to upgrade dependencies for test-data-supplier for testNG from 7.2.0 to 7.4.0.

Optimize args mess

Meta data args should be optimized to increase readability.

Java 9 support

See related discussion: https://stackoverflow.com/questions/45673466/testng-tests-execution-against-jdk-9-module-causes-inaccessibleobjectexception

Add GPG signing support

For publishing into Maven repo, artifacts must be signed.

Update Gradle version

Gradle 4.1 migration.

Rft: readability improvements

Split / rename methods and variables to make code more obvious and readable.

Parameters in DataSupplier ?

I'm submitting

• bug report
• feature request
• question

Content

Can we pass parameters in @DataSupplier ? sth like this

@DataSupplier
public Stream foo(String param1, int param2){
//return sth
}

@test(dataProvider = "foo", dataProviderParams = "stringA;5")
public void testFoo(Object bar) {
//test code here
}

Update lib loading strategy to use SPI

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.4.5
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

User is not bothered with manual listener configuration.

What is the actual behavior?

Listener should be manually added to build file or xml.

What is the motivation / use case for changing the behavior?

Simplify installation flow.

Other information

Add org.testng.ITestNGListener SPI.

Supplier doesn't initialize on Java 11

Updated jdk from 10 to 11.
Updated test-data-supplier to 1.8.2
Updated aspectj to 1.9.2

After launching testng test the follwing error occurs

AspectJ Internal Error: unable to add stackmap attributes. Unsupported class file major version 55
java.util.ServiceConfigurationError: org.testng.ITestNGListener: io.github.sskorol.core.DataProviderTransformer Unable to get public no-arg constructor
	at java.base/java.util.ServiceLoader.fail(ServiceLoader.java:581)
	at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:672)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1232)
	at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1264)
	at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1299)
	at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1384)
	at org.testng.TestNG.addServiceLoaderListeners(TestNG.java:967)
	at org.testng.TestNG.initializeConfiguration(TestNG.java:906)
	at org.testng.TestNG.initializeEverything(TestNG.java:996)
	at org.testng.TestNG.run(TestNG.java:1009)
	at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:73)
	at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)
Caused by: java.lang.VerifyError: Expecting a stackmap frame at branch target 45
Exception Details:
  Location:
    io/github/sskorol/core/DataProviderTransformer.supplyParallelData(Lorg/testng/ITestContext;Lorg/testng/ITestNGMethod;)Ljava/util/Iterator; @19: invokestatic
  Reason:
    Expected stackmap frame at this location.
  Bytecode:
    0000000: 2b3a 042c 3a05 b200 9c2a 2a19 0419 05b8
    0000010: 008c 4eb8 0092 2db6 0096 2a2b 2cb6 0002
    0000020: b600 03b9 0004 0100 3a07 a700 0f3a 06b8
    0000030: 0092 2db6 0099 1906 bfb8 0092 2db6 0099
    0000040: 1907 b0                                
  Exception Handler Table:
    bci [19, 45] => handler: 45

	at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
	at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3138)
	at java.base/java.lang.Class.getConstructor0(Class.java:3343)
	at java.base/java.lang.Class.getConstructor(Class.java:2152)
	at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:659)
	at java.base/java.util.ServiceLoader$1.run(ServiceLoader.java:656)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.util.ServiceLoader.getConstructor(ServiceLoader.java:667)
	... 10 more

Rft: optimize code and migrate to Sonar

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.6.0
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

All violations are fixed.

What is the actual behavior?

There're some code points which require optimization and decomposition.

What is the motivation / use case for changing the behavior?

Clean code.

Other information

Replace gradle quality plugin with Sonar Cloud.
Fix all the potential violations.

Update README with useful links

Add links to external maven / gradle projects with usage examples.

CI and coverage support

• add jacoco plugin;
• add travis config;
• increase coverage;

test execution fails in case @Test annotation is assigned on class level

Hi,

test-data-supplier:1.2.0
gradle 4.2, kotlin 1.1.51, testng 6.11

build.gradle

tasks.withType(Test) {
    systemProperties = System.properties
    useTestNG {
        listeners << 'io.github.sskorol.core.DataProviderTransformer'
    }
    dependsOn prepareTestData
    testLogging.showStandardStreams = true
    outputs.upToDateWhen { false }
}

code:

@Test <<<<
class AddCustomer : BaseTest() {
    @DataSupplier
    fun addCustomersTestData(): List<CustomerInfo?> {
        return YamlConverter().fromYaml<CustomerInfo>("/testData/customer/requestData/addCustomers.yml")
                .map { it.addRandomPartToName() }
    }
    @Test(dataProvider = "addCustomersTestData")
    fun addCustomer(custInfoRequest: CustomerInfo) {
        val customerInfoResponse =
                RequestBuilder("Customer", "add_customer")
                        .sendPost(sessionID, custInfoRequest)
                        .extractBody<AddUpdateCustomerResponse>()
        Assertions.assertThat(customerInfoResponse.iCustomer).isPositive();
    }
}

STR:

1. Run gradle test

Exception

org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 1.
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
	at com.sun.proxy.$Proxy1.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:146)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:128)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
	at io.github.sskorol.core.DataProviderTransformer.transform(DataProviderTransformer.java:36)
	at org.testng.internal.annotations.JDK15AnnotationFinder.transform(JDK15AnnotationFinder.java:155)
	at org.testng.internal.annotations.JDK15AnnotationFinder.findAnnotation(JDK15AnnotationFinder.java:231)
	at org.testng.internal.annotations.JDK15AnnotationFinder.findAnnotation(JDK15AnnotationFinder.java:204)
	at org.testng.TestNG.createCommandLineSuitesForClasses(TestNG.java:575)
	at org.testng.TestNG.initializeCommandLineSuites(TestNG.java:878)
	at org.testng.TestNG.run(TestNG.java:1104)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:129)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:88)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	... 25 more

Please check.Thank you.

AspectJ reference is missing in README

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.7.0
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproducible on a runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

The installation part of README must include aspectj reference, as it's required for all the service loaders that are handled by internal aspect implementation.

What is the actual behavior?

Missing aspectj in README leads to confusion, as users' SPIs are not being called.

What is the motivation / use case for changing the behavior?

Improve UX.

Fix Sonar violations

Some helper classes have issues with generics. Need to adjust signatures to pass the check.

Tuples type and parallel feature support

Add a new Tuple return type support from vavr library.
Add runInParallel flag support, as an alternative to DataProvider's parallel option.

Custom IAnnotationTransformer support

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.5.0
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

Users should be able to apply their own IAnnotationTransformer implementation.

What is the actual behavior?

Currently TestNG allows adding only a single implementation. As test-data-supplier uses transformer for its internal staff, a library prevents users from adding their own instances.

What is the motivation / use case for changing the behavior?

There shouldn't be any restrictions on test-data-supplier side.

Other information

Should be implemented via SPI.

Update docs

Add IDEA plugin links and description.

Downgrade to Gradle 4.5.1

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.5.0
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

User is able to release a new version.

What is the actual behavior?

Gradle throws caching exception while releasing.

What is the motivation / use case for changing the behavior?

Release a new version.

Other information

Downgrade Gradle to 4.5.1 version.

Jacoco xml report

Add missing xml report generation for online checks.

Add Factory processing with missing DP class

Will be implemented as soon as testng-team/testng#1647 is merged and released.
Requires DataProviderTransformer modification.

Is support separate @DataSupplier and @Test into different Java class?

Is support separate @DataSupplier and @Test into different Java class?

DataSupplier listener

Add a common DataSupplier interceptor for tracking before / after / around events.
Should be useful for collecting additional meta data and reporting tools integration.

Migrate to TestNG 7.0.0

I'm submitting

• bug report
• feature request

Other information

Update outdated dependencies to provide a better Java 11 integration. Migrate to TestNG 7.0.0.

@DataSupplier doesn't work with @Factory

public class SimpleTest{

    int i;

    @DataSupplier
    public Object[] ints(){
        return new Object[] {1, 2, 3};
    }

    public SimpleTest(){}

    @Factory(dataProvider = "ints")
    public SimpleTest(int i){
    this.i = i;
    }

    @Test
    public void simpleTest() {
        System.out.println("Simple Test Method " + i);
    }
}

Output:

Method public SimpleTest(int) requires a @DataProvider named : ints
	at org.testng.internal.Parameters.findDataProvider(Parameters.java:336)
	at org.testng.internal.Parameters.handleParameters(Parameters.java:498)
	at org.testng.internal.FactoryMethod.invoke(FactoryMethod.java:71)
	at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:158)
	at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:40)
	at org.testng.TestRunner.initMethods(TestRunner.java:401)
	at org.testng.TestRunner.init(TestRunner.java:250)
	at org.testng.TestRunner.init(TestRunner.java:220)
	at org.testng.TestRunner.<init>(TestRunner.java:161)
	at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:578)
	at org.testng.SuiteRunner.init(SuiteRunner.java:185)
	at org.testng.SuiteRunner.<init>(SuiteRunner.java:131)
	at org.testng.TestNG.createSuiteRunner(TestNG.java:1383)
	at org.testng.TestNG.createSuiteRunners(TestNG.java:1363)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1217)
	at org.testng.TestNG.runSuites(TestNG.java:1144)
	at org.testng.TestNG.run(TestNG.java:1115)
	at org.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:72)
	at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:123)

If we change @DataSupplier to @dataProvider everything works fine

Fix potential crash related to Reflections API update

Latest Reflections API causes runtime failure that is visible only while real lib usage. It should be downgraded.

Apart from that, it might be useful to provide TestNGMethod structure to the DataSupplierMetaData.

Changelog plugin seems abandoned. So it should be removed as well.

Add meta data injection support

Currently it's not possible to inject TestNG meta data (ITestContext / Method) into DataSupplier signature. Should be fixed on DataProviderTransformer level.

DataSupplier parrallel execution

Does this plugin support parrallel execution like @dataProvider(parallel = true) ?

Revise data retrieval mechanism

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.6.5
TestNG	6.14.3
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

Json, Yaml, Csv readers should be generic with an ability to specify a custom source type in runtime.

What is the actual behavior?

Readers are static. There's no way to specify custom source in runtime. Only via Source annotation.

What is the motivation / use case for changing the behavior?

Provide more flexible data reading mechanism.

Other information

Use a mix of Strategy and Builder pattern with an ability to set custom data source.

Update TestNG and other deps

I'm submitting

• bug report
• feature request

Environment

Test Data Supplier	1.4.0
TestNG	6.13.1
Build tool	[email protected]
IDE	[email protected]

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

What is the expected behavior?

Factories without explicitly set DP class, should implicitly set corresponding meta-data.

What is the actual behavior?

This feature is missing in TestNG 6.13.1. However, it's fixed in 6.14.2.

What is the motivation / use case for changing the behavior?

Add better Factory support.

Other information

See details: testng-team/testng#1647

Flattering collections and new return types

• Add Map / Entry to the supported list of return types.
• Sometimes it's required to perform a Java-like flatMap operation to extract collection values directly into test's signature.

    @DataSupplier(flatMap = true)
    public Map<Integer, String> getInternallyExtractedMapData() {
        return EntryStream.of(asList("user3", "user4")).toMap();
    }

    @DataSupplier(flatMap = true)
    public StreamEx getInternallyExtractedTupleData() {
        final List<String> list1 = asList("data1", "data2");
        final List<String> list2 = asList("data3", "data4");
        return IntStreamEx.range(0, Math.min(list1.size(), list2.size()))
                          .boxed()
                          .map(i -> Tuple.of(list1.get(i), list2.get(i)));
    }

• Map should be transformed into list of Entry.

Twitter publishing support

Make an auto tweet on release.