ebean-orm/ebean

Ebean ORM

ebean's Issues

Transaction rollback using try-with-resources

If only Transaction interface extends Closeable interface we can use try-with-resources syntax to handle transaction rollback.

The close() method should only call end().

Update now working when implemented custom type

Hi,

At the beginning - It is really hard to find some documentation how to convert some database types to custom classes.

I'm trying to create javaFX8 bean (observable properties) to work with Ebean.
So bean looks like this:

@Entity
@Table(name="something")
public class Something {
    @Id
    private SimpleIntegerProperty id;
    private SimpleStringProperty name;

    public int getId() {
        return id.get();
    }

    public SimpleIntegerProperty idProperty() {
        return id;
    }

    public void setId(int id) {
        this.id.set(id);
    }

    public String getName() {
        return name.get();
    }

    public SimpleStringProperty nameProperty() {
        return name;
    }

    public void setName(String name) {
        this.name.set(name);
    }
}

I have created to ScalarTypeConverter classes like this:

public class SimpleIntegerPropertyConverter implements ScalarTypeConverter<SimpleIntegerProperty, Integer> {
    @Override
    public SimpleIntegerProperty getNullValue() {
        return null;
    }

    @Override
    public SimpleIntegerProperty wrapValue(Integer integer) {
        return new SimpleIntegerProperty(integer);
    }

    @Override
    public Integer unwrapValue(SimpleIntegerProperty simpleIntegerProperty) {
        return simpleIntegerProperty.getValue();
    }
}

public class SimpleStringPropertyConverter implements ScalarTypeConverter<SimpleStringProperty, String> {
    @Override
    public SimpleStringProperty getNullValue() {
        return null;
    }

    @Override
    public SimpleStringProperty wrapValue(String s) {
        return new SimpleStringProperty(s);
    }

    @Override
    public String unwrapValue(SimpleStringProperty simpleStringProperty) {
        return simpleStringProperty.getValue();
    }
}

This works when I'm reading from database. But save, update is not working. I found that old values is always null. Maybe I'm doing something wrong or this is bug?

You can find maven project on github where I'm testing that
https://github.com/nonameplum/appFx

Bug - @PrivateOwned makes Ebean try to remove non persistent objects from DB and throws OptimisticLockException

The ebean throws OptimisticLockException when I recover my bean from DB, add in it a new private owned child, remove this new child, and then save the parent object.

The beans and mappings are:

public class Parent{

@Version
private Integer version;

@PrivateOwned
@OneToMany(mappedBy = "parent", cascade = CascadeType.ALL)
private List<Child> children;
...

}

public class Child{

@Version
private Integer version;

@ManyToOne
private Parent parent;
... 

}

The example code could be:
public void test(){

Parent parent = Ebean.find(Parent.class, 1);
Child child = new Child();

parent.getChildren.add(child);

... for some reason, child gets unnecessary ...

parent.getChildren.remove(child);
Ebean.save(parent); // this throws OptimisticLockException

}

I googled a lot these days for some workaround to this issue since my implementation for avoiding the exception and keeping the deal with concurrency is very very expensive.

It would be great having Ebean keeping the orphan removal of PrivateOwned annotation along with some mechanism that avoids trying to delete non persistent children.

My suggestion is that ebean ignore children with null id and version when removing orphans. Alternatively, the removeOrphans of @OneToMany could be enabled.

Best regards.

Ícaro Muniz

@Transactional enhancement not rolling back an uncaught RuntimeException

Query.setId should not return all results for null input Object

If I pass a null userId to this method call, it generates no WHERE clause in the generated SQL, and pulls back ALL records, and mashalls them onto the heap.

EbeanServer.find(User.class).setAutofetch(true).where().idEq(userId).findUnique();

Of course, I have added a null check in our code, but I would expect that you would want a null check here, as the side effects are not good with large tables, and Objects that have other large tables in their hierarchy.

Invalidating L2 Cache on deletion for referenced OneToMany relations not working

When deleting a cached entity which is referenced by a OneToMany relation, an silently catched exception is thrown and also the entity which is containing the OneToMany relations isn't updated correctly.

Cause:
BeanDescriptor.cacheDelete() is calling BeanPropertyAssocOne.cacheDelete() with the wrong parameter.
It is using the DeleteRequest itself as parameter instead of deleteRequest.getBean().

Use property autofetch.garbageCollectionOnShutdown to control GC on shutdown behavior

This defaults to false. Set this to true if you want to trigger GC on shutdown. You would do this if you want AutoFetch profiling information to be collected on shutdown.

Persistence Context Cache not cleared when deleting a collection of entities in a running transaction

It seems like the Persistence Context Cache is not cleared correctly when several entities are deleted at the same time.

Here is some pseudo code to show how to reproduce the bug:

server.beginTransaction();
MyEntity entity1 = new MyEntity("entity1", "oldValue");
MyEntity entity2 = new MyEntity("entity2", "oldValue");
server.save(entity1);
server.save(entity2);
entity1.setValue("newValue");
server.update(Arrays.asList(entity1, entity2));
MyEntity found1 = server.find(MyEntity.class, "entity1");
MyEntity found2 = server.find(MyEntity.class, "entity2");
assertThat(found1.getProfileId(), is("newValue"));
assertThat(found2.getProfileId(), is("oldValue"));
server.delete(MyEntity.class, Arrays.asList(entity1.getId(), entity2.getId()));
assertThat(repository.find((MyEntity.class, "entity1"), nullValue());  <-- THIS ASSERT FAILS
assertThat(repository.find((MyEntity.class, "entity2"), nullValue());
server.rollbackTransaction();

By adding the following code after the delete operation the test will pass

    Transaction transaction = server.currentTransaction();
    server.delete(beanType, ids, transaction);
    if(transaction instanceof SpiTransaction){
        SpiTransaction spiTransaction = (SpiTransaction) transaction;
        for(Object id : ids){
            spiTransaction.getPersistenceContext().clear(beanType, id);
        }
    }

I am running org.avaje.ebeanorm:avaje-ebeanorm:3.2.3

Up to recently I run ebean 2.. and had the same problem there. But there I also had the same problem for single delete operations and update operations as well. Seems like that thouse errors are gone now though.

JPA2 - @OneToMany or @ManyToOne not supported inside @Embeddables

I tried to migrate a system from Hibernate to Ebean and this was one of the biggest problem!.

When setting a property back to it's original value ... it resets the dirty state on that property.

Customer customer = ...
// the name is Fred

setName("Jack");
// name property is now dirty

// setting name back to original value
// the name property should not longer be considered dirty
setName("Fred");

Memory Leak in Shutdown Hook

When you start Ebean (version 2.8), it registers a shutdown hook, but when you stop Ebean, it doesn't unregister the hook. This causes a Classloader memory leak (A situation where the WebappClassLoader can't be garbage collected because a higher-level object has a reference to an object loaded by the classloader we want to free).

It would be nice if a 'real' fix could be included in Ebean's ShutdownManager. I searched the latest version of the code and I see the problem still exists.

I have created this nasty Reflections-based hack to remove the shutdown hook for now, I hope this code is useful to other people worried about this problem.

@Data @Slf4j
public class EbeanShutdownBean implements DisposableBean {

    private static final String EBEANS_SHUTDOWNHOOK_CLASSNAME = "com.avaje.ebeaninternal.server.lib.DaemonScheduleThreadPool.ShutdownHook";

    public EbeanShutdownBean() {
        log.info("Created Ebean Shutdown Bean... will just lurk until shutdown");
    }

    @Override
    public void destroy() throws Exception {
        log.info("Shutting down Ebeans");
        ShutdownManager.shutdown();

        Class clazz = Class.forName("java.lang.ApplicationShutdownHooks");
        Field field = clazz.getDeclaredField("hooks");
        field.setAccessible(true);
        Map<Thread, Thread> hooks = (Map<Thread, Thread>) field.get(null);

        Thread targetHook = null;
        for (Thread hook : hooks.keySet()) {
            final Class<? extends Thread> threadClass = hook.getClass();
            final String threadClassName = threadClass.getCanonicalName();
            log.debug("Found shutdown hook " + threadClassName);
            if (threadClassName.equals(EBEANS_SHUTDOWNHOOK_CLASSNAME)) {
                targetHook = hook;
            }
        }

        if (targetHook != null) {
            log.info("Deregistering hook " + EBEANS_SHUTDOWNHOOK_CLASSNAME);
            Runtime.getRuntime().removeShutdownHook(targetHook);
        }
    }

}

Many relations are deleted when accessing many field in BeanPersistAdapter on stateless updates

When using stateless updates with the flag "updateNullValues" = false and the many field is set to null, all relations will be deleted when it has been accessed within the "preUpdate" event of the BeanPersistAdapter.

see pull request #23 for unit tests

Protected constructor not supported

It is common in Hibernate that we add a protected constructor to force user to only constructor with parameters... EBean does not support

enum.toString() is used rather than enum.name() to persist (and JSON parse/format)

Rob note: This issue has been copied from the API repo over to here.

I am using Ebean (v 3.1) as packaged in Play! Framework 2.1.0. Perhaps this has been fixed in head, but I can't find the references in the source. Perhaps this is even the desired functionality, but it seems to be a bug to me.

An Enum of the form:

public enum Enumeration {
    NAME("Human readable name."),
    ...

    public String humanFriendlyName;

    private Enumeration(String humanReadableName) {
        this.humanFriendlyName = humanReadableName;
    }

    public String toString() { return humanFriendlyName; }
}

Used as a data field in a class:

@Entity
public class ShowEnum {
    ...
    @Enumerated(EnumType.STRING)
    public Enumeration enumeration;
    ...
    public ShowEnum() {
        enumeration = Enumeration.NAME;
    }

    public ShowEnum saveShowEnum(){
        ShowEnum showEnum = new ShowEnum();
        showEnum.save();
        return showEnum;
    }
}

The Enumeration field will be persisted in the database as a charvar with "Human readable name." as it's value. Subsequent lookups on the ShowEnum instance will then fail with "IllegalArgumentException: No enum constant ...Enumeration.Human readable name."

A workaround is to add @EnumMapping or @EnumValue annotations to the enum:

@EnumMapping(NAME="NAME", ...)
public enum Enumeration {
    ...
    }

public enum Enumeration {
    @EnumValue("NAME")
    NAME("Human readable name."),
    ...
    }

The workaround is verbose, and it would seem to me to be more intuitive to map the system representation of the enum as opposed to the toString() return value.

Feedback is certainly appreciated. Even pointing out the package/class/method where I could look at the root cause (or some document on the system architecture which would let me figure it out for myself) would be appreciated. My only ORM experience to date is using Ebean within the Play! Framework, but I may be able to propose a solution and submit a PR myself.

Hash values for QueryPlan collide

I'm using Play 2.2.1, which includes ebeanorm-3.2.2 which I tested against and found (what I believe) this bug.

I am getting PersistenceExceptions thrown when creating a large set (hundreds) of similar queries in a row. It seems to depend on the data I'm passing in, which unfortunately, I cannot disclose. The queries look very similar, but have varying numbers of parameters in each section. I traced this through the ebean source code (which I repackaged and replaced the jar in my Play repo), and it lead me to believe that the hashes were colliding. I printed the computed hashes of each query, and they did indeed match. It seems that because they hash the same, the same QueryPlan object it being used for both, even though they have differing numbers of bound parameters.

Both below errors hash to the value: 1336939366

Before I knew all of this, I attempted to explain what I was seeing over on stackoverflow if you'd like to take a look at that as well.
http://stackoverflow.com/questions/20501591/ebean-dynamic-query-prepared-statements-mismatched-parameter-count-error

Here are the two stack traces:

play.api.Application$$anon$1: Execution exception[[PersistenceException: Query threw SQLException:No value specified for parameter 19 
Bind values:[SENT,1290,8988,13032,13052,1290,96,99,100,101,102,103,104,105,106,107,108,222] 
Query was:
select distinct t0.contact_id c0, t0.contact_uuid c1, t0.bounce c2 from contact t0 join email_record u1 on u1.contact_id = t0.contact_id  join contact_tag u2z_ on u2z_.contact_id = t0.contact_id  join tag u2 on u2.tag_id = u2z_.tag_id  where u1.status = ?  and t0.unit_id = ?  and u2.tag_id in (?,?)  and t0.unit_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )  and t0.campaign_id in (?,?,?,?,?,?,?,?,?,?,?,?)  

]]
    at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
    at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
    at scala.Option.map(Option.scala:145) [scala-library.jar:na]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: javax.persistence.PersistenceException: Query threw SQLException:No value specified for parameter 19 
Bind values:[SENT,1290,8988,13032,13052,1290,96,99,100,101,102,103,104,105,106,107,108,222] 
Query was:
select distinct t0.contact_id c0, t0.contact_uuid c1, t0.bounce c2 from contact t0 join email_record u1 on u1.contact_id = t0.contact_id  join contact_tag u2z_ on u2z_.contact_id = t0.contact_id  join tag u2 on u2.tag_id = u2z_.tag_id  where u1.status = ?  and t0.unit_id = ?  and u2.tag_id in (?,?)  and t0.unit_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )  and t0.campaign_id in (?,?,?,?,?,?,?,?,?,?,?,?)  


    at com.avaje.ebeaninternal.server.query.CQuery.createPersistenceException(CQuery.java:798) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.CQuery.createPersistenceException(CQuery.java:775) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.CQueryEngine.findMany(CQueryEngine.java:205) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.DefaultOrmQueryEngine.findMany(DefaultOrmQueryEngine.java:81) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.core.OrmQueryRequest.findList(OrmQueryRequest.java:265) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.core.DefaultServer.findList(DefaultServer.java:1469) ~[avaje-ebeanorm.jar:na]
Caused by: java.sql.SQLException: No value specified for parameter 19
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.PreparedStatement.checkAllParametersSet(PreparedStatement.java:2612) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:2587) ~[mysql-connector-java-5.1.27.jar:na]

And the second:

play.api.Application$$anon$1: Execution exception[[PersistenceException: Error with property[19] dt[4]data[1464][java.lang.Integer]]]
    at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
    at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
    at scala.Option.map(Option.scala:145) [scala-library.jar:na]
    at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: javax.persistence.PersistenceException: Error with property[19] dt[4]data[1464][java.lang.Integer]
    at com.avaje.ebeaninternal.server.persist.Binder.bindSimpleData(Binder.java:327) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.persist.Binder.bindObject(Binder.java:216) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.persist.Binder.bindObject(Binder.java:173) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.CQueryPredicates.bind(CQueryPredicates.java:152) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.CQuery.prepareBindExecuteQuery(CQuery.java:382) ~[avaje-ebeanorm.jar:na]
    at com.avaje.ebeaninternal.server.query.CQueryEngine.findMany(CQueryEngine.java:165) ~[avaje-ebeanorm.jar:na]
Caused by: java.sql.SQLException: Parameter index out of range (19 > number of parameters, which is 18).
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.PreparedStatement.checkBounds(PreparedStatement.java:3814) ~[mysql-connector-java-5.1.27.jar:na]
    at com.mysql.jdbc.PreparedStatement.setInternal(PreparedStatement.java:3796) ~[mysql-connector-java-5.1.27.jar:na]

This is affecting our production environment so we need a fix ASAP, so I'm going to attempt to fix this in the ebean source, and if I can, I'll try to submit a patch to this ticket. However, I am very unfamiliar with this code, so if someone has the time I'd really appreciate the help.

I can provide more data on request.
Additionally, I submitted this bug on what I assume to be the old bug system before seeing the dates on the issues there, http://www.avaje.org/bug.html.

Using findIterate with MySQL streams the entire result set at once

Using findIterate streams the entire result set. If it's particularly large, this could cause a Java OutOfMemoryError when calling findIterate.

According to the MySQL JDBC documentation, it seems like the statement needs to be constructed like this for streaming to work correctly:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
              java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);

MySQL JDBC doc: http://dev.mysql.com/doc/refman/5.5/en/connector-j-reference-implementation-notes.html
Google Group Topic: https://groups.google.com/forum/#!topic/ebean/PaMDPEetbJI

Allow programmatic shutdown of individual EbeanServers

In Play Framework (among many other scenarios), we encourage the use of in memory HSQL databases in testing, so they can be created fresh in each test thrown away at the end. This also requires creating a new EbeanServer for each test. We're now seeing that when you have 500 or more unit tests, the JVM takes a long time to shutdown, I'm seeing around 90 second shutdown times. The time it takes to shutdown increases exponentially, at 1000 tests it's 5 or 6 minutes. Debugging shows that all this time is spent in the Ebean shutdown hook.

Since Ebean doesn't provide a way to shutdown an individual ebean server, the shutdown of all of these servers must occur during the shutdown of the JVM, and the exponential time increase I would guess must be to do with straining on resources when you have that many ebean servers, both in memory and other resources, for example, using a BoneCP connection pool, anything that holds a reference to it will keep its associated thread alive, and since all the ebean servers stay live until the JVM shuts down, BoneCP can't be garbage collected and so it's associated thread stays live.

Ebean should provide a mechanism to shutdown an individual EbeanServer, at least for the purposes of unit testing in this fashion.

Query cache doesn't work

Query cache doesn't work, this simple statement invokes SQL query every time:

for (int i = 0; i < 5; i++) {
  ebeanServer.createQuery(Number.class).setUseQueryCache(true).findList();
}

I found method getFromQueryCache() in class OrmQueryRequest and it looks strange, it always returns null:

  /**
   * Try to get the query result from the query cache.
   */
  public BeanCollection<T> getFromQueryCache() {

    if (!query.isUseQueryCache()) {
      return null;
    }

    if (query.getType() == null) {
      // the query plan and bind values must be the same
      cacheKey = Integer.valueOf(query.queryHash());

    } else {
      // additionally the return type (List/Set/Map) must be the same
      cacheKey = Integer.valueOf(31 * query.queryHash() + query.getType().hashCode());
    }

    // TODO: Sort out returning BeanCollection from L2 cache
    return null;
  }

Bad ddl generated for sqlite when having more than one primary key column

Generating DDL for a simple entity (Listing 1) as the following generates a PersistenceException (Listing 2). SQLite does support primary keys with more than one column, but they have to be defined as a table constraint. I would expect ebean to generate something as in Listing 3.

Listing 1

@Entity
public class SimpleEntity {
    @Id public int firstIdField;
    @Id public int secondIdField;
}

Listing 2

javax.persistence.PersistenceException: Error: Error executing stmt[create table simple_entity ( first_id_field integer primary key, second_id_field integer primary key) ] error[[SQLITE_ERROR] SQL error or missing database (table "simple_entity" has more than one primary key)]

Listing 3

CREATE TABLE simple_entity ( 
    first_id_field integer, 
    second_id_field integer,
    PRIMARY KEY (first_id_field, second_id_field)
) 

Query self-referenced table returning reference object in the result

I figured out that this issue belongs to this module rather than where I originally posted - https://github.com/rbygrave/avaje-ebeanorm/issues/1 - so I closed that one and created this one here.

Created from google group https://groups.google.com/d/topic/ebean/55L7_s_yFJo/discussion

I come in to the a problem when trying to query against a self-referenced table recently. There is a similar post here https://groups.google.com/d/topic/play-framework/Bi0x40OvHPM/discussion in play-framework group.

Problem Description
Query against a table containing self-referenced FK returns a list containing some reference object, thus any further usage of that reference object will invoke another query (lazy load) against the database. If I use Ebean.createJsonContext().toJsonString to conver the result to JSON, the reference object will only be something like {"id": 1}.

I created a sample project on github https://github.com/peter-fu/ebean-self-ref-sample demonstrating the issue.

Best Regards,
Peter

Strange behaviour for findPagingList(pageSize)

I have a pager which list a table that has 12615 elements 10 (pageSize) at a time, the headers order the thing.

The problem is: if I keep clicking the header of a page (like 25 times for example), the admin of mysql for "show full processlist" it will start showing "little calls", but they will not stop after some minutes, it seems that the prefetch is not only requesting the next page... but perhaps the whole table 10 at a time or perhaps some random number of times (perhaps some threads there are running on to each other causing a "whole table lookup" or something similar).

Also as stated it won't create extra request if I do .findPagingList(pageSize).setFetchAhead(false).

I guess ebean is <revision name="3.1.1" status="release" pubdate="20121008215015" resolver="local" artresolver="local" homepage="http://www.avaje.org" downloaded="false" searched="false" default="false" conf="compile, master(*), runtime, compile(*), runtime(*), master" position="20"> from project/target/resolution-cache/reports/default-default-d75936-compile-internal.xml

Stateless recursive updates for entities with version field will create Inserts instead of Updates

We wanted to add a version field to all of our entities, while adding this i have discovered that on recursive stateless updates (many relations with CascadeType.ALL) will result in an Insert instead of an Update.
And so a PersistenceException will be thrown with "primary key violation".

We are using Ebean together with Play Framework Java, and there are mostly used stateless updates.

Folling behavior i have discovert:

• The many relations have set the ID correctly, but the version column is set to null
• In DefaultPersister.saveAssocManyDetails() the Insert for the recusirve Update is created cause BeanDescriptor.isStatelessUpdate() is returning false

BeanDescriptor.isStatelessUpdate() is checking for the version column with higher priority, but in my opinion only a check on the ID field would make sense, especially for weak references.

Am i expecting a wrong behavior?
I will create a unit test for reproducing this behavior.

Persistence Context Cache not cleared when updating an entity in a running transaction

This is acctually the same bug as this #46 but for the server.update method. I am using version 3.2.5.

Expr does not have a betweenProperties(...) method.

There is a betweenProperties(...) on ExpressionList but not on Expr.

At the moment you have to use fluid style to use betweenProperties like:

List list = Ebean.find(MyEntityBean.class)
.where()
.disjunction()
.betweenProperties("startDate", "endDate", date1)
.betweenProperties("startDate", "endDate", date2)
.endJunction()
.findList();

Wrong database configuration for SQLite

When using ebean with a SQLite database, the generated ddl scripts seem to be incorrect.
The generated ddl looks something like:

create table call_note (
id                   bigint AUTOINCREMENT primary key,
...
);

However, according to the SQLite Documentation, autoincrement columns can only be defined as integer. Additionally, the autoincrement keyword needs to be placed after primary key.

I was able to workaround this issue by creating my own version of the com.avaje.ebean.config.dbplatform.SQLitePlatform class and placing it in the classpath before the ebean libraries. It's a quick hack which sets the autoincrement keyword to an empty string and redefines the bigint type to integer:

package com.avaje.ebean.config.dbplatform;

import java.sql.Types;

import javax.sql.DataSource;

import com.avaje.ebean.BackgroundExecutor;

public class SQLitePlatform extends DatabasePlatform {
    
    static {
        System.err.println("\n\n!!! Custom SQLitePlatform class for ebean ORM loaded !!!!\n\n");
    }

    public SQLitePlatform() {
        super();
        this.name = "sqlite";

        this.dbIdentity.setIdType(IdType.IDENTITY);
        this.dbIdentity.setSupportsGetGeneratedKeys(false);
        this.dbIdentity
                .setSelectLastInsertedIdTemplate("select last_insert_rowid()");
        this.openQuote = "\"";
        this.closeQuote = "\"";

        this.booleanDbType = Types.INTEGER;

        dbTypeMap.put(Types.BIT, new DbType("int default 0"));
        dbTypeMap.put(Types.BOOLEAN, new DbType("int default 0"));
        dbTypeMap.put(Types.BIGINT, new DbType("integer"));

        dbDdlSyntax.setInlinePrimaryKeyConstraint(true);
        dbDdlSyntax.setIdentity("");
        dbDdlSyntax.setDisableReferentialIntegrity("PRAGMA foreign_keys = OFF");
        dbDdlSyntax.setEnableReferentialIntegrity("PRAGMA foreign_keys = ON");
    }

    /**
     * Return null in case there is a sequence annotation.
     */
    @Override
    public IdGenerator createSequenceIdGenerator(BackgroundExecutor be,
            DataSource ds, String seqName, int batchSize) {

        return null;
    }

}

Stateless Updates throws Exception when executing without changes (updateNullValues = false)

IMHO i would prefer that simply no update would be executed, like in normal Updates.

Unit tests are provied in Pull Request #23

BUG - findRowCount sql error when using a @Formula property in the where clause

This issue is as per https://groups.google.com/forum/#!topic/ebean/i5WTLw3syBw

Originally logged as:
I have following class:

@entity
@table(name = "clients1")
public class Client1 extends Model {
@id
public int id;

@Formula(select = "inv.total_amount", join = "left join (select client_id, sum(amount) as total_amount from invoices group by client_id) as inv on inv.client_id = ${ta}.id")
public int totalOrdersAmount;

public static Finder<String, Client1> find = new Finder<String, Client1>(String.class, Client1.class);

public static Page<Client1> getClientsWithOrders(int page) {
    return Client1.find
            .where().gt("totalOrdersAmount", 0)
            .findPagingList(Clients.PAGE_SIZE)
            .getPage(page);
}

}

It has totalOrdersAmount field. Also getClientsWithOrders uses this field in where clause. Now if I do

Page page1 = Client1.getClientsWithOrders(0);
page1.getDisplayXtoYofZ(" ", " ");

It will throw

[PersistenceException: java.util.concurrent.ExecutionException: javax.persistence.PersistenceException: Query threw SQLException:Unknown column 'inv.total_amount' in 'where clause' Bind values:[0] Query was: select count(*) from clients1 t0 where inv.total_amount > ? ]

Looks like getDisplayXtoYofZ (and some others) doesn't recognize join in @formula annotation. Any thoughts how to work around this problem?

Ebean loads @OneToMany associations using OUTER JOIN

A similar issue was documented in the old bug tracking system (http://www.avaje.org/bugdetail-410.html), and discussed in the google group (https://groups.google.com/forum/#!topic/ebean/V6foc1K38Lk).

Excerpt from the discussion:

I have noticed that Ebean always load the associations using OUTER JOINS. I've tried with @OnetoOne, @OneToMany, using a FetchConfig, or FetchType.LAZY, and still it always use an OUTER JOIN.

For example, I have a User model which has many books (object and db code at the end). If I load the user like this:

User user = User.find.fetch("books", new FetchConfig().query() ).where("id = 2").findUnique();

The first query loads the user:

select t0.id c0, t0.first_name c1, t0.last_name c2 
from users t0 
where t0.id = 2

The second query uses an outer join :

select t0.id c0
        , t1.id c1, t1.name c2, t1.user_id c3 
from users t0
left outer join books t1 on t1.user_id = t0.id
where t0.id = 2   
order by t0.id

However, I would expect the second query to be something like :

select t1.id c1, t1.name c2, t1.user_id c3 
from books t1
where t1.user_id = 2

Question 1. The user_id is already known by the first query, so why the join is required?
Question 2. Is there some way around this?

Code for models and SQL :

@Entity
@Table(name="users")
public class User
{
    @Id
    public int id;
    public String firstName;
    public String lastName;

    @OneToMany(mappedBy = "user")
    public List<Book> books;

    public static Finder<Integer,User> find = new Finder<Integer,User>( Integer.class, User.class );
}

@Entity
@Table(name="books")
public class Book
{
    @Id
    public int id;
    public String name;

    @ManyToOne
    public User user;

    public static Finder<Integer,Book> find = new Finder<Integer,Book>( Integer.class, Book.class );
}

And the SQL code is :
CREATE TABLE `users` (
  `id` int(11) NOT NULL,
  `first_name` varchar(45) DEFAULT NULL,
  `last_name` varchar(45) DEFAULT NULL,
  PRIMARY KEY (`id`)
);
INSERT INTO users VALUES(1,"Joe","Bloh");
INSERT INTO users VALUES(2,"Maria","Luis");
CREATE  TABLE `books` (
  `id` INT NOT NULL ,
  `user_id` INT NOT NULL ,
  `name` VARCHAR(45) NULL ,
  PRIMARY KEY (`id`)
);
INSERT INTO books VALUES(1,1,"Joe's first book");
INSERT INTO books VALUES(2,1,"Joe's second book");
INSERT INTO books VALUES(3,2,"Maria's first book");
INSERT INTO books VALUES(4,2,"Maria's second book");

disjunction expression should not produce inner join - left outer join instead

This issue is known as BUG 402 (see http://www.avaje.org/bugdetail-402.html).

The following query produces the incorrect SQL result:

// using disjunction()
query.where()
  .disjunction()
    .eq("id", 1)
    .eq("children.id", 2)
  .endJunction();

// or using or()
query.where()
  .or(Expr.eq("id", 1),
      Expr.eq("children.id", 2));

The generated SQL:

select distinct [a bunch of stuff]
from x t0
join x u1 on u1.parentid = t0.id
where ( t0.id = ? or u1.id = ? )

The object used for the query:

public class X { 
  @Id
  public int id;
  @OneToMany(mappedBy="parent")
  public List<X> children;
  @ManyToOne
  public X parent;
}

There are several topics on google groups about this issue.
The only way to circumvent this bug is to use RawSql.

Support for JSON unmarshalling of @XmlRootElement beans.

Support for unmarshalling of @XmlRootElement beans that contain @entity beans.

NPE when updating model with Embedded bean

I am using Play framework 2.1 RC1 / Java, w/ ebean 3.1.2.

Please refer to http://stackoverflow.com/questions/12881576/ebean-2-7-3-play-framework-2-0-4-optimisticlockexception-nullpointerexcept

BUG: Deleted bean not removed from PersistenceContext

Peter found a bug just now where in a single transaction if you fetch a bean (loads it into the persistenceContext) and delete the bean and then fetch it again ... the second fetch returns the bean out of the persistenceContext - even though you just deleted it.

The required fix is that when beans are deleted that they are removed from the PersistenceContext.

AbstractBeanCollection loses ModifyListenMode after loading Entity from L2 Cache

As told in https://groups.google.com/forum/?fromgroups=#!topic/ebean/xUNVrkC261g

When you load a Entity, which contains referenced Entities with a @@manytomany or a @@OneToMany annotation, changes on this collections aren't tracked down and no update is generated and executed on a .save() or .update().

I think the cause is that the AbstractBeanCollection.modifyListenMode is Null after loading the Entity from the L2 Cache.

Embedded Entities with Autofetch Exception

I've run into an exception when using embedded entities with autofetch enabled with both 2.7.7 and 3.2.2.

The definition looks like this:

Owning entity:

@Embedded
@EmbeddedColumns(columns="leadDays=mondayProcurementLeadDays, enabled=mondayProcurementEnabled")
private ProcurementDay mondayProcurement;

Embedded entity:

@Embeddable
public class ProcurementDay {
  private int leadDays;
  private boolean enabled;
  //getters / setters
}

This seems to cause an exception as soon as autofetch starts to tune queries involving this entity, it seems autofetch wants to add the embedded entity as a join, the relevant bits of the stack trace are (Ebean 3.2.2):

Caused by: java.lang.NullPointerException
    at java.util.TreeMap.cmp(TreeMap.java:1913)
    at java.util.TreeMap.put(TreeMap.java:1982)
    at java.util.TreeSet.add(TreeSet.java:255)
    at com.avaje.ebeaninternal.server.query.SqlTreeAlias.addPropertyJoin(SqlTreeAlias.java:82)
    at com.avaje.ebeaninternal.server.query.SqlTreeAlias.addJoin(SqlTreeAlias.java:69)
    at com.avaje.ebeaninternal.server.query.SqlTreeBuilder.buildRoot(SqlTreeBuilder.java:182)
    at com.avaje.ebeaninternal.server.query.SqlTreeBuilder.build(SqlTreeBuilder.java:119)
    at com.avaje.ebeaninternal.server.query.CQueryBuilder.createSqlTree(CQueryBuilder.java:238)
    at com.avaje.ebeaninternal.server.query.CQueryBuilder.buildQuery(CQueryBuilder.java:203)
    at com.avaje.ebeaninternal.server.query.CQueryEngine.findMany(CQueryEngine.java:158)
    at com.avaje.ebeaninternal.server.query.DefaultOrmQueryEngine.findMany(DefaultOrmQueryEngine.java:77)
    at com.avaje.ebeaninternal.server.core.OrmQueryRequest.findList(OrmQueryRequest.java:265)
    at com.avaje.ebeaninternal.server.core.DefaultServer.findList(DefaultServer.java:1460)
    at com.avaje.ebeaninternal.server.querydefn.DefaultOrmQuery.findList(DefaultOrmQuery.java:885)

I'm unfamiliar with the internals of this part of the code but I think the issue lies in how SqlTreeAlias.addJoin() is handling embedded properties, it seems that the way the split() method handles embedded beans always results in sending null to addPropertyJoin() which tries to add null to a TreeSet implementation that doesn't allow null causing the exception.

Bug - models.Role cannot be cast to java.util.Map$Entry [error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveAssocManyDetails(DefaultPersister.java:877)

I have an entity called User with the following property:

@OneToMany(cascade = CascadeType.ALL)
@mapkey(name = "id")
public Map<Long, Role> roles = new HashMap<Long, Role>();

Role is also an entity. When I add a Role to the roles map and then save the User, I get the following:

models.Role cannot be cast to java.util.Map$Entry
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveAssocManyDetails(DefaultPersister.java:877)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveMany(DefaultPersister.java:799)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveAssocMany(DefaultPersister.java:704)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.update(DefaultPersister.java:442)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveEnhanced(DefaultPersister.java:343)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.saveRecurse(DefaultPersister.java:315)
[error] at com.avaje.ebeaninternal.server.persist.DefaultPersister.save(DefaultPersister.java:282)
[error] at com.avaje.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:1577)
[error] at com.avaje.ebeaninternal.server.core.DefaultServer.save(DefaultServer.java:1567)
[error] at com.avaje.ebean.Ebean.save(Ebean.java:538)

Am I missing something or is this a bug?

ManyToMany Collection cache isn't updated correctly

When updating a ManyToMany field of an already cached bean, the collection cache for this field isn't updated.

See pull request #23 for unit test

Query parsing error for SELECT something * 1.5 AS something_else

I'm getting this:

[PersistenceException: Expecting Max 3 words parsing column sites_score * 1.5 + follow_score AS score. Got [sites_score, *, 1.5, +, follow_score, AS, score]]

When using RawSql with a valid query for PostgreSQL using SELECT sites_score * 1.5 + follow_score AS score as a column.

Fix JSON support for TimeZone

Hi Guys,

I am using Timezone class in my entity to store default timezone for user, it can serialize and deserialize in DB - field value in db is ie. Europe/Berlin and this is OK.

Unfortunately ebean-jaxrs when serializing to json converts it to:

{
...
timezone: "sun.util.calendar.ZoneInfo[id="Europe/Berlin",offset=3600000,dstSavings=3600000,useDaylight=true,transitions=143,lastRule=java.util.SimpleTimeZone[id=Europe/Berlin,offset=3600000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=2,startMonth=2,startDay=-1,startDayOfWeek=1,startTime=3600000,startTimeMode=2,endMode=2,endMonth=9,endDay=-1,endDayOfWeek=1,endTime=3600000,endTimeMode=2]]"
...
}

Is it possible to provide a custom json serializer/deserializer for a specific type?

Marcin

findByIds doesn't return generic type

Hi,

https://github.com/ebean-orm/avaje-ebeanorm/blob/master/src/main/java/com/avaje/ebean/Query.java#L444

public List<Object> findIds();

Is it possible that we change that line to the following? Without this I have to iterate the idList to cast every id.

public List<I> findIds();

Thanks,
Peter

Args constructor with method invocation is causing problem

I have some classes with Args-Constructor that invokes some method from the constructor. For some reason, EBean is raising a NPE.

Bug - @AttributeOverrides annotation for attribute definition in entity does not work for embedded primary keys

Hi,

I was debugging my code for a few hours and found bug in Ebean library. I am using 3.2.2 version, but the same bug is still in repository.

I've got example of an entity definition (I simplified the example to show the problem):

@Entity
public class User extends Model {
    @Id
    @Embedded
    @AttributeOverrides({
            @AttributeOverride(name = "value", column = @Column(name = "user_id"))
    })
    public UserId userId;
}

Here is definition of UserId (simplified too):

@Embeddable
public class UserId implements Serializable {
    public String value;

    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

As you can see in the entity definition, I've put the information that user ID should be stored in "user_id" column. But it will not work when the user ID attribute is marked as a primary key.

Why? FactoryId class from Ebean generates wrapper for primary key and uses in that case BindableIdEmbedded class. Here is source code of constructor of BindableIdEmbedded:

  public BindableIdEmbedded(BeanPropertyAssocOne<?> embId, BeanDescriptor<?> desc) {
    this.embId = embId;
    this.props = embId.getTargetDescriptor().propertiesBaseScalar();
    matches = MatchedImportedProperty.build(props, desc);
  }

It constructs wrapper for embedded primary key field getting all properties from the target descriptor (only from UserId class, without matching with User entity annotations for the field). Next, constructor of InsertMeta object uses these properties to construct SQL. And uses dbColumn attribute from there, which is set to "value". It's wrong, my annotations in entity are ignored.

How to hack this bug? We can easily change the UserId class definition adding Column annotation:

@Embeddable
public class UserId implements Serializable {
    @Column(name = "user_id")
    public String value;

    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

It will work but it's hack, as I mentioned (we can use many fields of UserId type in entity, original annotations in entity works for fields without EmbeddedId or Id).

I would fix it but actually that's my second day with that library, I don't now how the library works in details. I think that probably BindableIdEmbedded constructor should be fixed (3 lines mentioned before) - probably original props from embId shouldn't be ignored there, but I'm not sure of this.

Right now I'm using mentioned hack, but if someone want to fix it and knows how, it will be nice.

Thanks,
Tomasz

Join fetching related objects that use inheritance causes wrong SQL

Consider the following entities:

// Profile.java
@Entity
public class Profile {

   @ManyToOne(cascade = CascadeType.ALL)
    public Picture picture;
    public String slug;
    ...

}

// Media.java
@Entity
@Inheritance(strategy=InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(discriminatorType=DiscriminatorType.STRING, name = "type")
public class Media {

    public String url;
    ...
}

// Picture.java
@Entity
@DiscriminatorValue("Picture")
public class Picture extends Media {
    ...
}

If you try to fetch a Profile with its Picture using the following sentence:

Profile aProfile = Ebean.find(Profile.class).fetch("picture").where().eq("slug", slug).findUnique();

It generates the following SQL:

select (list of columns...)
from profile t0
left outer join media t1 on t1.id = t0.picture_id
where t1.type = 'Picture' and t0.slug = ?

Which works OK, but fails to load a Profile when it has no related Picture (a.k.a null), which is a valid case.

The generated SQL should be instead:

select (list of columns...)
from profile t0
left outer join media t1 on t1.id = t0.picture_id and t1.type = 'Picture'
where t0.slug = ?

or well

select (list of columns...)
from profile t0
left outer join media t1 on t1.id = t0.picture_id
where (t1.type = 'Picture' or t1.type is null) and t0.slug = ?

Hope this makes sense.

Thanks!

JSR 303 Validation groups are ignored for DDL generation (of @NotNull)

Due to recent changes Ebean is now using jsr303 validation annotations to generate databases, unfortunately it ignores validation groups.

Jsr303 has the concept of groups which may be used to introduce conditional validation, Ebean seems to ignore groups and if there is a @NotNull annotation it adds not null constraint to column ignoring the group part.

Example:

@NotNull(groups = VGPerson.class)
private String firstName;

@NotNull(groups = VGPerson.class)
private String lastName;

@NotNull(groups = VGCompany.class)
private String companyName;

Here ebean should ignore @NotNull annotation.

The perfect solution would be to add a configuration parameter to ebean - which validation group should be used for schema generation (default value: javax.validation.groups.Default - this would ensure that it is more or less backwards compatible).

If more granular level of control is required devs could introduce their own group.

java.lang.IllegalStateException: getLoadBatch position[2] didn't find a bean in the list?

java.lang.IllegalStateException: getLoadBatch position[2] didn't find
a bean in the list?
at
com.avaje.ebeaninternal.server.loadcontext.DLoadWeakList.getLoadBatch(DLoadWeakList.java:
89)
at
com.avaje.ebeaninternal.server.loadcontext.DLoadWeakList.getLoadBatch(DLoadWeakList.java:
75)
at
com.avaje.ebeaninternal.server.loadcontext.DLoadBeanContext.loadBean(DLoadBeanContext.java:
160)
at com.avaje.ebean.bean.EntityBeanIntercept.loadBean(EntityBeanIntercept.java:
487)
at
com.avaje.ebean.bean.EntityBeanIntercept.preGetter(EntityBeanIntercept.java:
578)
at model.AttributeValue._ebean_get_name(AttributeValue.java:4)
at model.AttributeValue.getName(AttributeValue.java:42)

which goes in the right direction, because if you don't get the batch
the data wouldn't be loaded.

I think the batch is lost because it was a weak-reference. The
exception only happens if we limited the memory.

Jaxrs and Inheritance - BeanDescriptor bug

Having entities:

@MappedSuperclass
@Entity
@Table(name = "animals")
@Inheritance(strategy = InheritanceType.SINGLE_TABLE)
@DiscriminatorColumn(name = "species")
abstract class Animal { 
    [...]
}

@Entity
@DiscriminatorValue("CAT")
class Cat extends Animal {
    [...]
    private String name;
    [...]
}

binding this to resource method in jaxrs:

@Post
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Cat create(Cat cat) {
    [...]
}

json body sent in post does not contain species field. This should not be a bug since the type is set as the argument.

Debugging BeanDescriptor class shows that inheritInfo is able to figure out the value of the DescriminatorValue annotation.

causes an exception:

! com.avaje.ebean.text.TextException: Error reading inheritance discriminator - expected [species] but read [name]
! at com.avaje.ebeaninternal.server.deploy.BeanDescriptor.jsonRead(BeanDescriptor.java:2572)
! at com.avaje.ebeaninternal.server.deploy.BeanDescriptor.jsonReadBean(BeanDescriptor.java:2542)
! at com.avaje.ebeaninternal.server.text.json.DJsonContext.toBean(DJsonContext.java:93)
! at com.avaje.ebeaninternal.server.text.json.DJsonContext.toBean(DJsonContext.java:86)
! at com.avaje.ebeaninternal.server.text.json.DJsonContext.toObject(DJsonContext.java:171)
! at com.avaje.ebean.jaxrs.JaxrsJsonProvider.readFrom(JaxrsJsonProvider.java:152)

I believe that changing BeanDescriptor.jsonRead method to check if discriminatorValue is present and use is as default should be enough to fix it.

Documentation of ebean

Hi:

I have used Ebean before, and it seems that new version(3.2.4) is available , however the documentation is rather out of date, the reference(http://www.avaje.org/doc/ebean-userguide.pdf) is written at 2010 and focused on 2.6.0.

Then I wonder if latest documentation is available?

DB Connection pool issue (in Azure Cloud platform with aggressive network timeout). Looked like leaked connections

So there was an issue with the connection pool that surfaced when an application was run in the Azure cloud platform - more specifically this environment has relatively aggressive network timeouts on idle connections and the pool didn't detect or handle this well.

This issue was noted on topic https://groups.google.com/forum/#!topic/ebean/OtXTLnrvhXg

duplicate column names

When I try to do an insert on an entity that has a multiple column primary key (Embeddable) the generated SQL contains duplicate column names for each of the 3 columns that are part of the primary key.

Here is the class definition, the insert code, the generated SQL and error:

http://pastie.org/8550745

Here is the key definition:

http://pastie.org/8550753

At first I thought it was because I had defined the composite key as well as the fields individually so I made sure to comment them out but I get the same error.

I'm not sure if this is a bug or a misunderstanding on my part.

Thank you for any help,
Justin

Connection pool leak with findFutureRowCount(), findFutureList() and findFutureIds()

Issue found when invalid SQL is used to execute a findFutureRowCount().

Test case in master and will commit shortly.

Ebean Cascade not functional with play 2.2

Hi, I have been working with paly framework 2.2 and realized that the cascade options do not work as expected. Any reason why and help would be appreciated.