yahoo/elide

Elide is a Java library that lets you stand up a GraphQL/JSON-API web service with minimal effort.

Elide

Opinionated APIs for web & mobile applications.

Read this in other languages: 中文.

Table of Contents

• Background
• Documentation
• Install
• Usage
• Security
• Contribute
• License

Background

Elide is a Java library that lets you setup model driven GraphQL or JSON API web service with minimal effort. Elide supports two variants of APIs:

1. A CRUD (Create, Read, Update, Delete) API for reading and manipulating models.
2. An analytic API for aggregating measures over zero or more model attributes.

Elide supports a number of features:

Security Comes Standard

Control access to fields and entities through a declarative, intuitive permission syntax.

Mobile Friendly APIs

JSON-API & GraphQL lets developers fetch entire object graphs in a single round trip. Only requested elements of the data model are returned. Our opinionated approach for mutations addresses common application scenarios:

• Create a new object and add it to an existing collection in the same operation.
• Create a set of related, composite objects (a subgraph) and connect it to an existing, persisted graph.
• Differentiate between deleting an object vs disassociating an object from a relationship (but not deleting it).
• Change the composition of a relationship to something different.
• Reference a newly created object inside other mutation operations.

Filtering, sorting, pagination, and text search are supported out of the box.

Atomicity For Complex Writes

Elide supports multiple data model mutations in a single request in either JSON-API or GraphQL. Create objects, add them to relationships, modify or delete together in a single atomic request.

Analytic Query Support

Elide supports analytic queries against models crafted with its powerful semantic layer. Elide APIs work natively with Yavin to visualize, explore, and report on your data.

Schema Introspection

Explore, understand, and compose queries against your Elide API through generated Swagger documentation or GraphQL schema.

Customize

Customize the behavior of data model operations with computed attributes, data validation annotations, and request lifecycle hooks.

Storage Agnostic

Elide is agnostic to your particular persistence strategy. Use an ORM or provide your own implementation of a data store.

Documentation

More information about Elide can be found at elide.io.

Install

To try out an Elide example service, check out this Spring boot example project.

Alternatively, use elide-standalone which allows you to quickly setup a local instance of Elide running inside an embedded Jetty application.

Usage

For CRUD APIs

The simplest way to use Elide is by leveraging JPA to map your Elide models to persistence:

The models should represent the domain model of your web service:

@Entity
public class Book {

    @Id
    private Integer id;

    private String title;

    @ManyToMany(mappedBy = "books")
    private Set<Author> authors;
}

Add Elide annotations to both expose your models through the web service and define security policies for access:

@Entity
@Include(rootLevel = true)
@ReadPermission("Everyone")
@CreatePermission("Admin OR Publisher")
@DeletePermission("None")
@UpdatePermission("None")
public class Book {

    @Id
    private Integer id;

    @UpdatePermission("Admin OR Publisher")
    private String title;

    @ManyToMany(mappedBy = "books")
    private Set<Author> authors;
}

Add Lifecycle hooks to your models to embed custom business logic that execute inline with CRUD operations through the web service:

@Entity
@Include(rootLevel = true)
@ReadPermission("Everyone")
@CreatePermission("Admin OR Publisher")
@DeletePermission("None")
@UpdatePermission("None")
@LifeCycleHookBinding(operation = UPDATE, hook = BookCreationHook.class, phase = PRECOMMIT)
public class Book {

    @Id
    private Integer id;

    @UpdatePermission("Admin OR Publisher")
    private String title;

    @ManyToMany(mappedBy = "books")
    private Set<Author> authors;
}

public class BookCreationHook implements LifeCycleHook<Book> {
    @Override
    public void execute(LifeCycleHookBinding.Operation operation,
                        LifeCycleHookBinding.TransactionPhase phase,
                        Book book,
                        RequestScope requestScope,
                        Optional<ChangeSpec> changes) {
       //Do something
    }
}

Map expressions to security functions or predicates that get pushed to the persistence layer:

    @SecurityCheck("Admin")
    public static class IsAdminUser extends UserCheck {
        @Override
        public boolean ok(User user) {
            return isUserInRole(user, UserRole.admin);
        }
    }

To expose and query these models, follow the steps documented in the getting started guide.

For example API calls, look at:

1. JSON-API
2. GraphQL

For Analytic APIs

Analytic models including tables, measures, dimensions, and joins can be created either as POJOs or with a friendly HJSON configuration language:

{
  tables: [
    {
      name: Orders
      table: order_details
      measures: [
        {
          name: orderTotal
          type: DECIMAL
          definition: 'SUM({{$order_total}})'
        }
      ]
      dimensions: [
        {
          name: orderId
          type: TEXT
          definition: '{{$order_id}}'
        }
      ]
    }
  ]
}

More information on configuring or querying analytic models can be found here.

Security

Security is documented in depth here.

Contribute

Please refer to the contributing.md file for information about how to get involved. We welcome issues, questions, and pull requests.

If you are contributing to Elide using an IDE, such as IntelliJ, make sure to install the Lombok plugin.

Community chat is now on discord. Join by clicking here. Legacy discussion is archived on spectrum.

License

This project is licensed under the terms of the Apache 2.0 open source license. Please refer to LICENSE for the full terms.

Articles

Intro to Elide video

Create a JSON API REST Service With Spring Boot and Elide

Custom Security With a Spring Boot/Elide Json API Server

Logging Into a Spring Boot/Elide JSON API Server

Securing a JSON API REST Service With Spring Boot and Elide

Creating Entities in a Spring Boot/Elide JSON API Server

Updating and Deleting with a Spring Boot/Elide JSON API Server

elide's Issues

Create compile scope module implementing only elide-annotations

elide-core module pulls in dependencies not required when compiling annotated beans. Creating a separate elide-annotations module for compile scope would be much better.

Wonky exception type

When Elide fails to assign an ID it responds with 403 Forbidden, this feels misleading. I think elide should respond with 400 Bad Request instead.

Filtering on collection broken

Example query with filter: sample.com/service/v1/user/12345/memberships?filter[membership.company.id]=54321&token=myTokenString

Draft: File Upload Design

I'm trying to draft the design of supporting file upload with metadata in Elide.

Motivation

Use cases(1, 2) are common of building a REST JSON API that allows for files to be uploaded with json documents describing the metadata.

An example is in an E-Commerce site, sellers may want to upload images for their products, and specify which products the images are related to.

Approaches

Base64 Way

The client encodes the files to be upload with Base64, and embed the Base64 string as a field in the JSON document.

The server receives the JSON document with Base64 data embedded, and applies additional processing logic (decoding from Base64, image resizing, saving to local filesystem, etc.)

• Advantages:
  • JSON-API spec-compliant.
  • All in one HTTP call.
• Disadvantage:
  • Base64 encoded data size is 25% larger than the original data.
  • Additional processing time introduced by Base64 encoding/decoding.

Multipart Way

The client sends a multipart/form-data or a multipart/mixed HTTP request instead of application/vnd.api+json. Have a part in the payload as the metadata part with Content-Type: application/vnd.api+json. The rest of the payload can hold the file data.

The server has to have endpoints configured to recognize multipart/form-data or multipart/mixed request type and parse the payload. This could be troublesome if it's a sub-resource (POST /product/1/images).

An example multipart request looks like:

POST /upload HTTP/1.1
Accept: application/vnd.api+json
Content-Type: multipart/form-data;boundary=Boundary_1_208866101_1460195430845
User-Agent: Jersey/2.22.1 (HttpUrlConnection 1.8.0_60)
MIME-Version: 1.0
Host: localhost:8080
Connection: keep-alive
Content-Length: 287451

--Boundary_1_208866101_1460195430845
Content-Type: application/vnd.api+json
Content-Disposition: form-data; name="data"

{
  "data": {
    "type": "image",
    "attributes": {
      "isThumbnail": true
    }, 
    "relationships": {
      "product": {
        "data": {
          "type": "product",
          "id": "1"
        }
      }
    }
  }
}
--Boundary_1_208866101_1460195430845--
Content-Type: application/octet-stream
Content-Disposition: form-data; name="files"

...some binary data...
--Boundary_1_208866101_1460195430845

• Advantages:
  • All in one HTTP call.
• Disadvantages:
  • Non-resumeable uploads

Multi-phase(or Network Transaction) way

Inspired by Twitter API ,Youtube API, and this post. The idea is to sort of transactionize the process of file upload using in multiple HTTP calls. The procedure goes as follows:

1. The client sends an initial JSON-API HTTP request, to provide the metadata of the files to be upload as well as a X-Upload-Content-Length header field to indicate the total accounted size of the content.
2. The server receives the initial call, processes the metadata(save to database, add to job queue, etc.), marks the metadata record as unfinished, assigns a transaction id, comes up with a timeout, and responds to client with a Location header which contains a tailored URL to the file upload endpoint
3. The client receives the response, and sends the file to the file upload endpoint.
4. The server receives the file upload request and processes it. Response will be the same as the response to the initial call.
5. The client sends an finialize request, to mark that file upload is complete.
6. The server receives the finalize request, and close the network transaction
7. If the upload processes still haven't finished after the timeout, the server close the network transaction, marks the metadata record as aborted. Responds to further upload requests with reject messages.

• Advantages:
  • Can decouple the pure JSON-API endpoints and file upload endpoints. Some application may have dedicated servers with optimized file systems, or use third-party file upload services.
  • Can extend to implement resumeable upload. The client and server can agree on the max size in one HTTP request, and the client splits the file and sends separately.
  • Server can reject or close the file upload process in cases of oversize or timeout.
• Disadvantages:
  • Multiple HTTP calls.

Handling Large Files (Chunking Mode)

In all case described above, the client can send the request with Transfer-Encoding: chunked to enable streaming mode.

TODO:

Come up with an Elide change proposal of implementing some or all of the scenarios above.

Null pointer exception

Elide returns a null pointer exception when the "path" parameter is missing from the body of a patch request

Explore Supporting Arbitrary Object Instantiation

We should explore our options to instantiate checks and classes through a user-specified way. In particular, this would allow users to use injection for creation of things such as checks. As far as objects are concerned, users can already do this by overriding the data store createObject() directly.

Anyway, this merits some deeper thought.

ID should not be required on POST

http://jsonapi.org/format/
Exception: The id member is not required when the resource object originates at the client and represents a new resource to be created on the server.

Avoid Excessive save() Calls

We should avoid our excessive calls to save(). In particular, non-modified objects should be avoided whenever possible but we should also investigate other situations where this may occur.

Patch Extension handling attribute updates

Refer to "Updating attributes" section at http://jsonapi.org/extensions/jsonpatch/
Currently elide expects the "value" parameter to contain data in the form of JsonApiDocument, but we should also allow individual attribute updates by accepting simple string, int, boolean, etc in the "value" parameter as per the JsonApi spec.

Should update Readme to highlight articles

https://dzone.com/articles/create-a-json-api-rest-service-with-spring-boot-an
https://dzone.com/articles/custom-security-with-a-spring-bootelide-json-api-s

Need better logging of security failures

That we log the set of checks failed is good, but we should log what field we were checking (and for all the first check that failed) when the failure occurred, not just what object we were looking at.

Use name() instead of toString() for when reading enum values

Add support for @CreatePermission and @DeletePermission on Fields

We need support for adding @CreatePermission and @DeletePermission to fields. In particular, these would only be valid for relationships. Runtime checks would likely need to be in place to ensure that these annotations are not specified on attributes as it has no semantic meaning.

Fix up hardcoding of "id"

We hardcode the id field to the string id in the PersistentResource. This is in the setId() and getRelation(..., String id) methods. We should add a method to the EntityDictionary which can return the name of the id field and update it accordingly.

Update on Create Standard Check

Create a standard check included with elide-core that allows an "update on create" permission. That is, the object is allowed to be created by a user but then never again modified. Since this is a common pattern (i.e. ToS, etc.) it fits to have it included in the core out-of-box.

Gitter channel

https://gitter.im/yahoo/elide

Commonize JSR 303 Validation Code

Need to write the JSR 303 validation code in a nice package that can be used where needed.

'lets', not 'let's'

Sorry; I hate to be the one to call out a grammar issue but when it's in the product's description it's pretty noticeable-- it should be "Elide is a Java library that lets you stand...", not "...that let's you stand".

Support filtering by relationship fields

Currently, you can not execute a GET request and filter entities by its relationship fields.

Example:

Given Book and User entities and Book has an author field that is a one-to-one to relationship to User.

I would like to request all Books where author is = 55

or

myAwesomeElideInstance.com/books?filter[books.author]=55

Return 200 instead of 204 on update

There has been expressed interest in returning a proper 200 OK code (with proper response body) upon updating instead of a 204 No Content. As far as I can tell, there are no technical limitations to us doing this at the moment. This is likely an artifact of an artificial limitation we encountered during our initial development.

That said, this should comply with the JSON API spec: http://jsonapi.org/format/#crud-updating-responses-200

Return less revealing status codes

I would expect the following behavior:

• when querying a root collection I have access to I should always receive 403 Forbidden (and never 404 Not Found) so that I cannot discover the collection size.
• when querying a nested collection I should always receive 404 Not Found (and never 403 Forbidden) when I do not have access so that I cannot discover:
  1. what collections exist.
  2. the potential contents of collections I know about.

I am currently receiving both error codes for each scenario.

Enable Redis as a new datastore

Look at @JsonProperty for entityBinding names

It came up to me that I have a couple of boolean fields in a model with name like isDeleted.
I could annotate the field/getter with @Column(name = "isDeleted") so that the database column it created will be isDeleted instead of deleted.

But in the json request/response I get deleted as the name of the field. Intuitively, I tried to annotate the fields/getters with @JsonProperty("isDeleted"), but that doesn't work.

I inspected that inside of EntityBinding the name of the field strictly depends on the name of field or method. Would it be helpful to support looking at @JsonProperty annotation for binding field name?

Investigate performance of Patch Extension on ID's

As from this PR, we do not filter for id in patch extension: #114. We need to investigate the performance impacts of this and consider alternatives. In particular, there are issues with handling UUID's in this context.

Instantiate Expression Checks at Boot

Given that our expression language takes strings now, there is a particular pain point in verifying that there were no typos. We should instantiate (and cache) instances of every check for all beans that we have found at startup. If something cannot be instantiated, we should fail-fast and exit.

NPE on expressions starting with LPAREN

Discovered an NPE while evaluating checks beginning with a left parenthesis.

POST to To-One not returning relationships

I noticed that POSTing to To-One relationships does not return the parent reverse relationship (on bi-directional) to the user. When running a GET, this does work properly and the relationship exists. Need to investigate further and ensure this doesn't occur for other types of POST as well.

Pagination Support

More detail when I have a chance soon.

Need to explore developing an @Limit annotation and supporting general pagination to avoid large queries from overloading the DB.

There are work-arounds to this, but this is the proper solution in many cases.

NOTE: This may require some modifications to the DataStoreTransaction interface. As a result, we should consider this modification before the initial release of Elide 2.0.

Run Integration Tests with elide-dbmanager-hibernate5

Currently the integration tests only run with the elide-dbmanager-hibernate3. To ensure the quality and stability of elide-dbmanager-hibernate5, we should run the integration tests with elide-dbmanager-hibernate5 as well.

Document parameterized interface quirk

Type erasure borks EntityBinding. Can fix with targetEntity

Support filtering by id

Currently, you can not execute a GET request and filter entities by id. This case is especially useful when performing an filter with an in operator. For example, requesting comments where id is in [1,5,7,8].

Fix examples to use non-deprecated methods

Some methods/constructors used in the examples are deprecated in the new releases. We should update the examples to use latest recommended methods

Move datastores to Builder interface

HSQLdb explicitly doesn't support SCROLL_FORWARD_ONLY and this won't be the only backend-dependent issue we find. Builders give us a way to add those other customization options without writing crazy constructors and deprecating lots of stuff all the time.

Provide a way to disable inheriting permissions from a super class.

@include should have an attribute to disable permission inheritance from its super class. While permissions can be overridden on a field by field basis, there is no way to start with clean slate.

Filtering on java.sql.Timestamp throwing InvalidValueException

We're throwing an InvalidValueException when receiving a Long on a java.sql.Timestamp object. I suspect this is due to poor coercion. What we likely should do first for a given type is try to find a constructor that accepts this type (i.e. if direct coercion is not possible) and use this. Will require some more investigation first to confirm this issue/solution.

The tests need love

We need:

• A test plan
• A review of current test coverage
• A plan for filling in the gaps uncovered by our review

Defaulting Permission for Object

We should be able to have a default permission (i.e. default deny all) and selectively update fields within that object with additional permissions.

Currently, the object takes precedence and ignores fields. Field security annotations should be first-class citizens.

filter requires entity namespace, but spine does not

Given an entity called User with an attribute called email, you can build a filter in Spine by doing something like this...

var query = Query(resourceType: User.self)
query.whereAttribute("email", equalTo: email)

This generates a URL like this...

https://myapi.herokuapp.com/user?filter%5Bemail%[email protected]

However Elide is expecting...

https://myapi.herokuapp.com/user?filter%5Buser.email%[email protected]

Spine has checks around the attribute being specified and won't allow you to specify user.email. There isn't really a great workaround besides specifying the field corresponding to email as user.email. I believe this will break trying to write a User. Although I think I could do some manipulation of the fields at runtime to work around this as well. It's a hack though.

This could be a bug in Spine, but it also seems reasonable that this should work. I haven't really considered how this would affect relationships which I assume is the reason it works this way. I didn't look at the jsonapi spec either.

Proposal for security expressions

The Problem

Permissions encapsulate checks in any and all annotation parameters. More complex checks are built writing custom Java code. Writing checks inside Java classes breaks the aspect oriented nature of the security annotations and also makes security brittle.If the same Java code is used in multiple places, it may have unintended consequences when that check logic is changed.

Instead of using Java to compose more complex checks, it would be simpler if:

1. Checks could be written to only do a single thing.
2. Check composition could be expressed inside the permission annotations.

Java is very restrictive about how annotation parameters can be expressed:

The return type of a method declared in an annotation type must be one
of the following, or a compile-time error occurs:

• A primitive type
• String
• Class or an invocation of Class (§4.5)
• An enum type
• An annotation type
• An array type whose component type is one of the preceding types
  (§10.1).

Suggested Solution

Permissions should have a new annotation parameter called 'expression' that lets a developer construct a boolean expression containing:

1. Unary operator not
2. Binary operators and and or
3. Parenthesis
4. Aliases for check classes

For example, the following would now be possible:

@ReadPermission(expression = "NOT UserIsLocked AND (AdminAtCompanyCheck OR SuperUserCheck) AND IsLimitedAccessCheck")

The entity dictionary will compile these expressions into check expression trees at entity bind time. A new entity dictionary constructor will be parameterized to include:

• A parameter of type Map<String, Check> containing the string alias(es) for each check class.

Fix overly aggressive shared check

Our shareable permission checking is too aggressive. If we have already accessed a non-new object through a proper hierarchy (i.e. it should have lineage), then we should not throw a forbidden exception.

For instance, adding a relationship in a post to /user/1/friends that points back to user with id 1 should theoretically succeed assuming all other permissions are correct (even if not shareable).

Spring Integration

Does this play nicely with spring framework? Is there a sample project somewhere using spring?

Logger should be named AuditLogger

The documentation says it's called AuditLogger but the source name is Logger. Also, Logger is a really bad name for this interface.

Problems Building Project on Windows 10

I think that there might be some problems with regards to the pom.xml file that causes the build process to fail on Windows 10. I've managed to build the project sucessfully on Ubuntu 14.04, but have had no luck so far on Windows.

The command run was: mvn verify -Dmaven.test.skip=true -e

[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Elide Data Store: Hibernate 5 2.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Elide: Parent Pom .................................. SUCCESS [  2.703 s]
[INFO] Elide: Core ........................................ SUCCESS [ 42.211 s]
[INFO] Elide Data Store: Parent Pom ....................... SUCCESS [  0.406 s]
[INFO] Elide Data Store: Hibernate Common ................. SUCCESS [  4.112 s]
[INFO] Elide: Integration Test Library .................... SUCCESS [  2.280 s]
[INFO] Elide Data Store: Hibernate 5 ...................... FAILURE [  0.156 s]
[INFO] Elide Data Store: Hibernate 3 ...................... SKIPPED
[INFO] Elide Data Store: In-Memory ........................ SKIPPED
[INFO] Elide Data Store: Multiplex Library ................ SKIPPED
[INFO] Elide Example: Parent Pom .......................... SKIPPED
[INFO] Elide Example: Persistence API with MySQL .......... SKIPPED
[INFO] Elide Example: Hibernate 3 API with MySQL .......... SKIPPED
[INFO] Elide Example: Persistence API with Security ....... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 52.191 s
[INFO] Finished at: 2016-02-26T18:15:44-06:00
[INFO] Final Memory: 49M/511M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project elide-datastore-hibernate5: Could not resolve dependencies for project com.yahoo.elide:elide-datastore-hibernate5:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT, com.yahoo.elide:elide-integration-tests:jar:tests:2.0.0-SNAPSHOT: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal on project elide-datastore-hibernate5: Could not resolve dependencies for project com.yahoo.elide:elide-datastore-hibernate5:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT, com.yahoo.elide:elide-integration-tests:jar:tests:2.0.0-SNAPSHOT: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:221)
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)
        at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:245)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:199)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project com.yahoo.elide:elide-datastore-hibernate5:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT, com.yahoo.elide:elide-integration-tests:jar:tests:2.0.0-SNAPSHOT: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT
        at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:211)
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)
        ... 23 more
Caused by: org.eclipse.aether.resolution.DependencyResolutionException: The following artifacts could not be resolved: com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT, com.yahoo.elide:elide-integration-tests:jar:tests:2.0.0-SNAPSHOT: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:384)
        at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:205)
        ... 24 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: The following artifacts could not be resolved: com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT, com.yahoo.elide:elide-integration-tests:jar:tests:2.0.0-SNAPSHOT: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:367)
        ... 25 more
Caused by: org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact com.yahoo.elide:elide-datastore-hibernate:jar:tests:2.0.0-SNAPSHOT
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:434)
        ... 27 more

Introducing Business Logic

From the documentation I can't see how you can customise operations. For example, when a user is created, maybe an email is sent to the manager and another message is sent to a legacy system via JMS. Is this sort of thing supported or in the pipeline?

Don't leak data from /relationships

Currently no permissions are evaluated if the contents of a relationship don't change.

For example:
A.foo = [B, C, D]
and I
POST /A/relationships/foo [B, C]
I will always get 204 even when I don't have permission to modify A#foo

Integrated Criteria Checks

Will come back to this ticket with a fleshed out proposal when I have a chance.

In short, we would like to explore creating a backend-agnostic criteria check to improve query performance arbitrarily.

NOTE: This may require modification to the DateStoreTransaction interface and should be designed/considered before the release of Elide 2.0.

NullPointerException on missing @Include annotation

Received a NullPointerException when sending this:

{
  "data": {
    "type": "comment",
    "attributes": {
      "text": "This event is awesome!"
    }, 
    "relationships": {
        "user" : {
            "data": {
                "type": "user",
                "id": "1"
            }
        }
    }
  }
}

Reason was that User.class did not have an @include annotation. Would be great to have a nicer error as it was unclear.

Revamp Security Backend

Below is the proposal for revamping the Elide security backend. It will require breaking changes to the public-facing API and-- consequently-- will be rolled up into Elide 2.0. This is a working document. That is, I will keep this up-to-date with developments on progression plans.

Current Limitations

Due to the current design of the security system, Elide faces problematic security limitations. These problems are not security vulnerabilities, but they disallow expression of certain security constraints. Moreover, they can also lead to confusion which is a show-stopper for effective security. That is, the best security rules are simple and clear making them less error-prone; the system executing these rules should also be clear in nature.

1. Fine-Grained Update Control. At the moment, it is not feasible to specify controls on Updates based on type of modification. For instance, if a user should be able to add objects to a list but disallowed to remove any, this cannot be effectively expressed in the current system. As it stands, we broadly check "update" permissions. In our case, an update of empty list effectively removes all elements within a list.
2. Check Execution Reasoning. In our current model, authorization check execution flow is determined dynamically at runtime based on a multitude of factors. In other words, the current system is trying to be "smart." Effectively, this makes reasoning about security checks far more difficult based upon when they would/should be executed.
3. Inability to control short-circuiting. It is common practice in programming to perform a couple of cheap checks before an expensive one. If any of these cheaper checks fail, then you can avoid computing the expensive check. In our current system, the lack of control on check execution does not guarantee that such a thing can be done repeatedly.
4. UserCheck's are tightly-coupled with security. The notion of a UserCheck in practice is not strictly for security, but more of an optimization instead. As a result, their tight-coupling to the security system makes them confusing to work with and limits the expressiveness of the system.

Solution

Below I have outlined the current state of our proposed solution. This solution comes from discussions with multiple sources and I suspect will further evolve through continued discourse before it is fully implemented. Again, I will keep this section up-to-date with the latest information as it's decided.

1. Fine-grained modification support. We will update the Check interface to support Changesets for Update requests. In particular, the Changeset object (pseudo-code outlined below) will become a parameter describing the proposed modification to an object. This will allow Check writers to verify the type of data modifications permitted. This solves the problem of Fine-Grained Update Control mentioned above.
2. Operation vs. Commit Execution. We will introduce two new concepts known as operation checks and commit checks. In particular, operation checks are intended to be run immediately upon request. That is, finalized data is not required to fully validate the security of this data. On the other hand, commit checks are expected to run only immediately before a transaction commit to the datastore. These checks often require the data to be fully manipulated and are run on the exact set of data which is about to be stored in the datastore. Providing checks at these granularities solve both issues of check execution reasoning and the ability to control short-circuiting since the developer now has direct control over the check execution points (i.e. operation checks can be executed to short-circuit expensive commit checks, for instance).
3. Separating UserCheck. We will elevate the UserCheck interface to a first-class citizen. Surfacing a new annotation and providing independent control from the general object security will make things less confusing in the future.

Proposed Check Interface Update

Below is the primary check interface:

public interface Check<T> {
  /**
   * This method will be called on operation-based checks since object data is not
   * necessarily finalized at this point
   */
  public boolean ok(RequestScope rs, Optional<Changeset<F>> cs);

  /**
   * This method will be called on commit-based checks since all data is finalized.
   */
  public boolean ok(T obj, RequestScope rs, Optional<Changeset<F>> cs);
}

However, since all permission types don't necessarily commitChecks (i.e. ReadPermission and DeletePermission, for instance), the commitChecks will be run as operationChecks in these scenarios joined by an and clause.

Refactoring of Security Code

These changes will include larger internal refactoring of the security. To simplify the logic and shared-state, I am suggesting a simpler PermissionManager. A PermissionManager instance will be bound to a particular RequestScope. Below is the proposed interface for calling into the PermissionManager

public class PermissionManager {

  public void checkPermissions(Class<? extends Check>[] checks, boolean isAny, PersistentResource<?> resource);

  public void checkPermissions(Class<? extends Check>[] checks, boolean isAny, PersistentResource<?> resource, ChangeSpec<?> changeSpec);

  public void executeCommitChecks();
}

Separation of Concern for User Checks

The current concept of UserCheck is intended for optimization (i.e. short-circuit DB load based on user properties) rather than security. As a result, we should separate these concepts. I propose that the UserCheck interface remain the same (except, perhaps, no longer making it a direct descendant of the Check interface) but we move it to another package (perhaps an optimization package or similar?). This may call for a new annotation. My proposal is the @UserPermission annotation. This will be very similar to existing checks, however, UserPermission will first be evaluated. If it does not evaluate to true, nothing else is checked and access it omitted to the full object. The @UserPermission annotation will apply strictly at the entity-level and there will not be field-level support for this annotation.

Pseudo-code for Changeset object

public class Changeset<T> {
  public String fieldName; // The modified field on the object
  public T added; // The serialized form of the updated field value
  public T removed; // The serialized form of the original field value
}

FAQ

1. Will this break my existing code? Yes. In particular, we suspect it will affect all custom permission checks as we intended to update the interface. That being said, we will be performing a major version bump when this feature is completed.
2. Am I insecure on Elide 1.x? In short: not that we are aware of. While this change affects the security backend of Elide, it is not prompted by any inherent vulnerabilities in our current approach with Elide 1.x. As a result, your current services should not be adversely affected. However, it is expected that all future updates will be applied to future versions of Elide and we suggest upgrading to stay current.

Parameterized Security Expressions

There has been an ask for parameterized security expressions. This is worth further investigating and discussing.

POST to ToOne relationship with existing element fails

a POST to a ToOne relationship where an element already exists fails. It drops into RecordTerminalState rather than CollectionTerminalState.

We expect the behavior of this would be a full replacement of the existing ToOne relationship with the new one.