roookeee/datus

datus enables you to define a conversion process between two data structures in a fluent functional API

Heya

I am just some random dude that cares quite a bit about how to develop great and useable software (which is pretty hard!). Mostly interested in Java / Kotlin, but C# isn't that far off :)

Feel free to contact me about anything!

datus's Issues

Examine having multiple inputs for generating one output object

There are numerous scenarios where an output object is constructed out of multiple input objects. datus does not support this concept as it has severe implications:

• Can't auto generate convert(Collection<In1>, Collection<In2>) as it does not correlate the two collections, we would need a map then, which seems cumbersome
• Both mutable and immutable API would have to be copy pasted as I don't see a way to do it otherwise

Maybe datus should define Tuple2...X types, these would easily fix this problem by unifying multiple input types into one, though the user has to create them by hand. Great care should be taken when deciding this as I don't want to implement too many concepts that are not directly related to data-mapping.

Apply PECS where it is missing

As a library datus should follow PECS. Most parts of datus are already PECS compliant but some functions that were added just before releasing 1.0 don't comply yet (or they have just slipped my attention).

I will categorize this issue as semver-minor because applying PECS is backwards compatible and not being able to use functions/predicates/etc. that would work with PECS is more of a bug than a new feature.

• Mapper interface

Use of mutation testing in datus - Help needed

Hello there!

My name is Ana. I noted that you use the mutation testing tool in the project.
I am a postdoctoral researcher at the University of Seville (Spain), and my colleagues and I are studying how mutation testing tools are used in practice. With this aim in mind, we have analysed over 3,500 public GitHub repositories using mutation testing tools, including yours! This work has recently been published in a journal paper available at https://link.springer.com/content/pdf/10.1007/s10664-022-10177-8.pdf.

To complete this study, we are asking for your help to understand better how mutation testing is used in practice, please! We would be extremely grateful if you could contribute to this study by answering a brief survey of 21 simple questions (no more than 6 minutes). This is the link to the questionnaire https://forms.gle/FvXNrimWAsJYC1zB9.

Drop me an e-mail if you have any questions or comments ([email protected]). Thank you very much in advance!!

Improve datus documentation

This tickets serves as the central point for anything related to datus documentation that could be improved.

• What is missing in datus documentation?
• How could datus strength be shown in a better way?
• Is there a need for a more complex sample project?
• Are there parts that are confusing and/or badly worded?

Let me know which parts you would like to see improved :)

Feedback: Add a shortcut for .from(Function.identity())

A user of datus has noted that .from(Function.identity() feels clumsy when a mapping step has to combine multiple getters of the input type into one setter / parameter of the output type.

We should consider adding a shortcut. Naming seems to be the hardest part of this task as the implementation is pretty straightforward.

Allow changing the type in then() and orElse() after a given()-call

Add content to the nearly empty USAGE.md

The USAGE.md file is pretty bare and an in depth explanation of the different constructs of datus is required to ease the usage of datus.

What is PerformanceBenchmarkTest exactly supposed to do?

I am a little confused about PerformanceBenchmarkTest.

• .jvmArgs("-server") is an ancient setting, that is simply ignored in the JVM.

• Second, what is the point of having a single @Benchmark? You are supposed to benchmark against something else, I guess.

• It seems your testedConversionShouldBeCorrect, just ensures that the conversion is correct, but that is something JMH already offers via @TearDown: https://hg.openjdk.java.net/code-tools/jmh/file/b6f87aa2a687/jmh-samples/src/main/java/org/openjdk/jmh/samples/JMHSample_05_StateFixtures.java

Anyway, these are nitpicks, mainly. The idea is that I hoped this test would have some samples against some popular frameworks. I have seen https://github.com/arey/java-object-mapper-benchmark, though. It's just that your results rely on those tests and I am not sure about their correctness (without understanding them, those are meaningless numbers: look how the benchmark presents that "direct" mapping is slower than "map-struct"). I wonder if I should provide a PR against some data mappings libraries inside (or a separate project/child from datus)?

Support generic types as input and output types

Currently the Datus class expects two Class<T> parameters to infer the input and output types. These cannot be passed if e.g. T is a List<T> as List<T>.class cannot be created / obtained.

We should add an overload to Datus.forTypes() without any parameters that leaves the type definition to the caller via Datus.<List<Integer>, List<String>>forTypes() (it is ugly indeed, but it works, don't think there is a better way).

Thanks to #33 for noticing this issue ! :)

Clarify Mapper.convertToMap behaviour on key collisions

The JavaDoc does not state what happens when there is a key-collision for a given input collection. We should clarify that the last mapped value is retained in the result map and any other behaviour should be implemented via the conversionStream function which offers all other alternatives through the Collectors.toMap collector

Consider renaming when

When is a reserved keyword in Kotlin which forces Kotlin users to use the when function of the builder APIs with backticks - annoying.

Reevaluate terminal operations in the immutable API

Goal of this issue is to document any other approaches to the immutable API which offer a better usablity than the current implementation.
The following parts of the immutable API come to mind:

• ImmutableConstructionStep map vs mapTo
• ImmutableConstructionStep when has to be followed by a .to(Function.identity())

I am open for suggestions but releasing 1.0 does not necessitate a new immutable API.

EDIT: solution was found, see below

Add tests for other JVM languages

Currently datus cannot be used in Kotlin as outlined here - Kotlins compiler can't infer the generic type parameters as accurately as Java.

We should add some tests for Scala and Kotlin (once the inference is more capable) to ensure datus really works with these languages.

Add an orElse variant that uses a null value directly

When using orElse(null) the java compiler complains about an ambiguous method call (supplier variant and value variant are equally specific) and some user might prefer orElse(null) over orElse(Function.identity()) when doing a null check.

We should add another orElse variant that uses a null value directly, e.g. orElseNull() - naming should be straight forward.

Conditionally apply a transformation

Suppose I have two DTOs:

public class In {

    public String getAge() {
        return age;
    }

    public void setAge(String age) {
        this.age = age;
    }

    private String age;
}

and

public class Out {

    public Integer getAge() {
        return age;
    }

    public void setAge(Integer age) {
        this.age = age;
    }

    private Integer age;
}

And I want to conditionally apply a transformation: I want to map String age only if it is made from digits, otherwise simple don't map it.

The only way I found to do this is:

        In in = new In();
        in.setAge("no");

        Out out = Datus.forTypes(In.class, Out.class)
                       .mutable(Out::new)
                       .from(In::getAge).given(Pattern.compile("\\d+").asPredicate(), Function.identity())
                       .orElseNull().map(x -> x == null ? null : Integer.parseInt(x)).into(Out::setAge)
                       .build()
                       .convert(in);

        Assertions.assertNotNull(out);
        Assertions.assertNull(out.getAge());

which looks weird, at best. Did I miss some documentation and there is a better way? If no, I think I might submit a PR with my idea to handle this; unless you already have something like this in motion.

Apply the PECS pattern to all function of datus

See https://stackoverflow.com/questions/50313830/why-many-java-stream-interface-methods-use-lower-bounded-wildcard-in-parameters for more information

Improve datus performance by reducing megamorhpic call sites

datus makes extensive use of lambda-chaining (.map-steps) and wrapping other lambdas (e.g. to support implicit nullsafeness). The resulting wrapper-functions share their byte-code / method-body with all calls to these functions (multiple .map-steps) which makes them mega-morphic and thus un-inlineable.

This affects the following places:

• ìnto` in the mutable API
• map in both APIs
• nullsafe() in both APIs

I have asked on SO on how to solve this and Holger supplied a solution which improves datus performance in the benchmark suite from roughly 8 million op/s to 23 million op/s.

The following points need to be considered while implementing this feature:

• either add an automatic-detection if re-loading a class via a new classloader is possible (e.g not in SubstrateVM) or add a feature toggle to disable / enable the feature
• Reference Holger's answer
• make sure that the resulting helper classes are not accessible from outside of datus (we don't want to leak such a functionality as it's not the core functionality of datus)
• profile what kind of impact these new classloader instantiations have

Gauging interest: Kotlin compatability / port?

Just leaving this as an issue to see if anyone is interested in a Kotlin port / compatability variant of Datus - leave a comment / reaction.

source to destination conversion.

Hi,
What if i want to convert a property from source byte to destination string this future is currently supported by map struct.

Add missing JavaDoc to ConstructorParameterBinding (class level)

Don't know how it slipped through, lets fix it ! :)

Add a nullsafe mode

As much as datus is about explicitness: traversing nested objects which can be null is quite cumbersome as it stands.

We have to find a way to avoid something like this:

.from(Order::getCustomer)
    .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
    .given(Objects::nonNull, Address::getCity).orElseNull()
    .into(OrderDTO::setBillingCity)

I personally would like to veto any variant that would make every mapping step of a mapping definition nullsafe - it's too implicit.

Implement a sample project that shows all of datus features

The existing documentation (readme.md & usage.md) may be enough but I myself like to play around with code (that compiles out of the box) instead of reading about a library. Let's add a simple example project that highlights datus features.

note: as this won't affect any release related code I don't know how to handle semantic versioning here - as it stands I won't do a new release after finishing the sample project.

• implement the sample project
• reference it from the readme / usage files

Extend the immutable API to support up to 12 constructor arguments

This seems necessary as I have witnessed plenty of complex objects that far exceeded 8 constructor parameters

Add a helper class for defining mappers for recursive data structures

I reconsidered adding this functionality to datus as there are quite a few scenarios wherin simple data structures are recursively included in themselfs (parent child relations like categories or products containing more subproduct variants).

There is an utility class outlined in https://github.com/roookeee/datus/blob/master/USAGE.md#advanced-usage--faq to fix this problem. Implementing the given class once (+ unit tests) is far superior to letting users handle it themselves.

Let's add it :)

Examine if its possible to define mappers for self-recursive data-structures

As its not possible to reference the currently being built mapper it is impossible to convert self-recursive data-structures (e.g. trees, double-linked-lists).

Concept: add the ability to map to multiple outputs

Say you have a Customer object as a field inside your Order object and want to extract both the cutomers name and adress to two separate fields of the output object. With datus you have to do getCustomer() twice which is cluttered and quite cumbersome.

See this benchmark suite with the following manual code:

public final class ManualMapper implements OrderMapper {

    @Override
    public OrderDTO map(Order source) {
        OrderDTO target = new OrderDTO();
        Customer customer = source.getCustomer();
        if (customer != null) {
            target.setCustomerName(customer.getName());
            if (customer.getBillingAddress() != null) {
                target.setBillingCity(customer.getBillingAddress().getCity());
                target.setBillingStreetAddress(customer.getBillingAddress().getStreet());
            }
            if (customer.getShippingAddress() != null) {
                target.setShippingCity(customer.getShippingAddress().getCity());
                target.setShippingStreetAddress(customer.getShippingAddress().getStreet());
            }
        }
        if (source.getProducts() != null) {
            List<ProductDTO> targetProducts = new ArrayList<ProductDTO>(source.getProducts().size());
            for (Product product : source.getProducts()) {
                targetProducts.add(new ProductDTO(product.getName()));
            }
            target.setProducts(targetProducts);
        }
        return target;
    }

}

which currently looks like this:

    private static final Mapper<Product, ProductDTO> productMapper = Datus.forTypes(Product.class, ProductDTO.class)
            .immutable((String name) -> new ProductDTO(name))
            .from(Product::getName).to(ConstructorParameter::bind)
            .build();


    private static final Mapper<Order, OrderDTO> orderMapper = Datus.forTypes(Order.class, OrderDTO.class)
            .mutable(OrderDTO::new)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getName).orElseNull()
                .into(OrderDTO::setCustomerName)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
                .given(Objects::nonNull, Address::getCity).orElseNull()
                .into(OrderDTO::setBillingCity)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getBillingAddress).orElseNull()
                .given(Objects::nonNull, Address::getStreet).orElseNull()
                .into(OrderDTO::setBillingStreetAddress)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getShippingAddress).orElseNull()
                .given(Objects::nonNull, Address::getCity).orElseNull()
                .into(OrderDTO::setShippingCity)
            .from(Order::getCustomer)
                .given(Objects::nonNull, Customer::getShippingAddress).orElseNull()
                .given(Objects::nonNull, Address::getStreet).orElseNull()
                .into(OrderDTO::setShippingStreetAddress)
            .from(Order::getProducts)
                .given(Objects::nonNull, productMapper::convert).orElseNull()
                .into(OrderDTO::setProducts)
            .build();

We have to do something about this and null checking is a problem too :/

It's not really an issue...

I just wanted to tell you that compared to some libraries (even my code, but I am not even ready for a release yet, so I might improve more), your code does a pretty good job. Here are my results run against github actions with latest JMH: https://github.com/wind57/0xMapper/pull/8/checks?check_run_id=820811700

You need to download the artifacts to see them.

You can close this issue or somehow reference to those, if you want.

Improve unit-test readability + coverage

Some of the unit-tests are rather odd and should be reconsidered / cleaned up to be more understandable

datus is still actively maintained, there aren't any feature requests or bug reports though :)

datus is subjectively feature complete but feature request / maintenance is still done whenever needed

Do we need map all the fields ?

It can be painful to map all the fields. Why do we need to map them even when the names are the same?

Publish datus to maven central and add its dependency definition into the README.md

see title

Add a JMH testsuite

datus was just integrated here but the main repository could use some fancy images + benchmark statistics itself. The benchmark should focus on other mapping frameworks that don't use code generation but we should definitely mention those that do and how they compare to datus perf

Exchange the explicit mapper classes through lambdas

datus currently implements the Mapper<A,B> in two distinct classes for the mutable and immutable API. This is unnecessary as the Mapper<A,B> interface is a functional interface. As the mapper implementation classes are package private this is a non-breaking change that reduces the overall code size of datus while also opening up for future JVM optimizations in regards to lambdas. Some local JMH testing also showed minor performance benefits when using lambdas.

Add a document with an in-depth comparison to other mapping frameworks

<self explanatory>
Could take some time to be succinct and as unbiased as possible