speedment/speedment

Speedment is a Stream ORM Java Toolkit and Runtime

Java Stream ORM

Speedment is an open source Java Stream ORM toolkit and runtime. The toolkit analyzes the metadata of an existing SQL database and automatically creates a Java representation of the data model. This powerful ORM enables you to create scalable and efficient Java applications using standard Java streams with no need to type SQL or use any new API.

Speedment was originally developed by researchers and engineers based in Palo Alto with the purpose to simplify and streamline the development of Java database applications by leveraging the Java Stream API.

Speedment is licensed under the business-friendly Apache 2 license. Contribution from users is encouraged. Please feel free to request new features, suggest improvements and file bug reports. Read more about contributing here.

Quick Start

Assuming you have Maven installed and a relational database available, you can start using Speedment in a minute:

• Start a New Speedment Maven Project
• Connect to Your Database

Expressing SQL as Java Streams

There is a remarkable resemblance between Java streams and SQL as summarized in the simplified table. This means there is no need for manually writing SQL-queries any more. You can remain in a pure Java world!

Example

Search in a film database for a film with a length greater than 120 minutes:

// Searches are optimized in the background!
Optional<Film> longFilm = films.stream()
    .filter(Film.LENGTH.greaterThan(120))
    .findAny();

Results in the following SQL query:

SELECT 
    `film_id`,`title`,`description`,`release_year`,
    `language_id`,`original_language_id`,`rental_duration`,`rental_rate`,
    `length`,`replacement_cost`,`rating`,`special_features`,
    `last_update` 
FROM 
     `sakila`.`film`
WHERE
    (`length` > 120)

Features

Speedment is equipped with the features listed below and more.

View Database Tables as Standard Java Streams

• Pure Java - Stream API instead of SQL eliminates the need of a query language
  
• Dynamic Joins - Ability to perform joins as Java streams on the application side
  
• Parallel Streams - Workload can automatically be divided over several threads
  

Short and Concise Type Safe Code

• Code Generation - Automatic Java representation of the latest state of your database eliminates boilerplate code and the need of manually writing Java Entity classes while minimizing the risk for bugs.
  
• Null Protection - Minimizes the risk involved with database null values by wrapping to Java Optionals
  
• Enum Integration - Mapping of String columns to Java Enums increases memory efficiency and type safety
  

Lazy Evaluation for Increased Performance

• Streams are Lazy - Content from the database is pulled as elements are needed and consumed
  
• Pipeline Introspection - Optimized performance by short circuiting of stream operations
  

Tutorials

The tutorials are divided into three sections. The basics are covered in the first section without any expected prior knowledge of Speedment. This builds a foundation of knowledge needed to fully benefit from the following tutorials.

Basics

• Tutorial 1 - Hello Speedment
• Tutorial 2 - A First Stream from Speedment

Sample applications

• Tutorial 3 - Speedment Spring Boot Integration; REST assured - it is easy
• Tutorial 4 - Speedment filters based on Json Web Tokens
• Tutorial 5 - Log errors in a database
• Tutorial 6 - Use Speedment with Java EE
• Tutorial 7 - Create Event Sourced Systems
• Tutorial 8 - Writing Your Own Extensions

Extending Speedment

• Tutorial 9 - Formatting Your Output as JSON
• Tutorial 10 - Working with BLOBS and CLOBS

Resources

• Documentation - Read the Speedment User Guide.
• JavaDocs - Latest Speedment JavaDocs.
• Examples - There are 15 detailed examples here and more can be found in the User Guide provided above.
• Gitter Chatroom - Reach out to the Speedment developers and other community members via the Gitter chatroom.
• Creating a Pull Request - Pull requests and improvement suggestions from the community are gladly accepted. Find more information here.

Requirements

Java Version

Speedment requires Java 8 or later. Make sure your IDE is configured to use JDK 8 (version 1.8.0_40 or newer).

Database Connectors

Speedment Open Source comes with support for the following databases out-of-the-box:

• MySQL
• MariaDB
• PostgreSQL
• SQLite

Enterprise database connectors include:

• AS400
• Cassandra
• DB2
• Informix
• Oracle
• Snowflake
• SQL Server

For more information, see Speedment Licensing and Pricing.

Licenses

• Speedment Open Source - This site covers the Speedment Open Source project available under the Apache 2 license.
• Speedment Stream - The same great features as Speedment OSS with support for commercial databases. Learn more at speedment.com/stream.
• Speedment HyperStream - An extension av Speedment Stream which also includes hypersonic query performance enabled by a unique in-JVM-memory management model. Learn more at speedment.com/hyperStream.

Copyright

Copyright (c) 2014-2019, Speedment, Inc. All Rights Reserved. Visit www.speedment.com for more info.

Github activity visualized

speedment's Issues

Add JavaDocs to MapStream

Introdice immutable configuration objects like UnmodifiableTable

Protects the configuration consistency

Introduce more standard implementations of TypeMapper

Boolean yes/no
Boolean true/false
Enum from String
Various Time implementation to long
Various Time implementation to int
Etc,

Platform not started if methods are called before start

Use resultSet.getInt(int ordinalColumn) instead of resultSet.getInt("columnName")

will improve performance.

Hare logo in JavaDoc

Include the Hare in JavaDoc. See Platform class for example.

Add optimized mapping similar to predicates

Currently the custom streams can optimize predicate statements like this:

Hare.stream()
    .filter(HareField.NAME.equal("Harry"))
    .findAny();

And the resulting SQL would be:

SELECT * FROM `Hare` WHERE `name` = "Harry" LIMIT 1;

It would be really usefull to also be able to optimize update operations. It could look like this:

Hare.stream()
    .filter(HareField.NAME.equal("Harry"))
    .map(HareField.NAME.set("Spire"))
    .forEach(Hare::persist);

And the resulting SQL would be:

UPDATE `Hare` SET `name` = "Spire" WHERE `name` = "Harry";

GUI support for high resolution displays

Right now, most parts of the GUI become very difficult to use on a high resolution display. JavaFX does have support for scaling up many components, but these features must be carefully used when designing the interface.

Primitive class mapping for not null fields

ApplicationMetadata - Only include metadata for generated classes

This is just a suggestion.

The com.speedment.internal.core.runtime.ApplicationMetadata#getMetadata currently contains the metadata for the complete schema.

Would it make sense to only include the metadata for the classes that are actually really generated?

Be able to set config parameters via System.properties or via a configuration file

Make HareSpeedmentApplication.build() read from java parametres and config file

config.groovy
name="myProject";
dbms {
name = "myDb";
password = "mySecretPwd283";
}

or

-Dspeedment.project.hare.db0.password=myOtherSectretPwd2354

Implement support for custom jdbcConnectorParameters

It shall be possible to set the jdbcConnectorParameters using a withJdbcConnectorParameters() method

Introduce JDBC connection pooling

No specific message in GUI if error during generation

If there is an error during generation, the user are left hanging as to what was the problem....

IndexColumn 'speedment_stat.db0.speedment_stat.usage.PRIMARY.id'
2015-07-26T12:54:32.268Z INFO  [JavaFX Application Thread] (c.s.g.c.SceneController) - Generating classes com.speedment.beacon.speedment_stat.*
2015-07-26T12:54:32.269Z INFO  [JavaFX Application Thread] (c.s.g.c.SceneController) - Target directory is src/main/java
2015-07-26T12:54:32.270Z INFO  [JavaFX Application Thread] (c.s.u.a.LoggingAdapterImpl) - JGoogleAnalytics: Tracking Asynchronously focusPoint=Speedment-2.0.0-EA2-SNAPSHOT-Generate
2015-07-26T12:54:32.446Z INFO  [JavaFX Application Thread] (c.s.g.c.SceneController) - 
.-------------: Generation failed! :--------------.
 Total time: 0.176s
 Finished at: Sun, 26 Jul 2015 14:54:32 +0200
 Files generated: 0
'-------------------------------------------------'
2015-07-26T12:54:32.446Z FATAL [JavaFX Application Thread] (c.s.g.c.SceneController) - Error! Failed to generate code.

Column names, tables and schema are not within quotes

17:45:21.040 [main] ERROR com.speedment.core.db.impl.AsynchronousQueryResultImpl - Error executing select id,key from speedment_stat.beacon_property_key
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'key from speedment_stat.beacon_property_key' at line 1

Move CollectorUtil into the API

and add JavaDoc

Examine impact if Platform is not Singleton but an instance

Streams are not closed when iterator() or spliterator() are called on Streams

finallyClose() should be removed.

ApplicationMetadata - Constant string too long

Using a really complicated/large database schema results in a constant String that is too long to compile inside com.speedment.internal.core.runtime.ApplicationMetadata#getMetadata

ProjectImpl.findTableByName() consumes 13% of all CPU capacity

When running a large number of manager.stream(), the ProjectImpl.findTableByName() consumes 13% of all CPU capacity. There is room for optimization.

Excluded tables still get generated

If you disable tables in the GUI, classes for the tables still get generated.

BeanValidation

Perhaps with an event listener for the life cycle events (persist/update/...). This could be useful for integration with other frameworks. e.g Validation and adding Bean Validation constraints
to the domain classes. JPA defines an integration with Bean Validation where validation
occurs on the different life cycle events. To make something like this happen with Speedment
one would need some sort of callback functionality.

If failed, a Bean Validation ConstraintViolationException might be thrown.

Generated code is too tied to the specific database instance

I think the generated code is a little too tied to the specific database instance, instead of just the database schema. I mean, the code is representing the schema after all, not the actual database.

When starting my application, I do not specify which server or which database to use. The only reference I can find for "127.0.0.1" is within the generated metadata, so I assume that's what it uses when it connects to the database.

I'd like to see something like this:

new SpeedmentApplication()
    .server("127.0.0.1")
    .database("SpeedmentTest")
    .withUsername("root")
    .withPassword("X")
    .start();

If you want to have multiple instances running for your system (for example for different customers) that use the same database schema, you don't want to generate code for each server/database.

Mapping is missing for java.util.Date

2015-10-13T17:25:03.925Z ERROR [JavaFX Application Thread] (c.s.i.g.c.SceneController) - Error! Failed to generate code.
java.lang.NullPointerException: The JavaTypeMapperComponent does not have a mapping for com.speedment.internal.core.config.dbms.MySqlDbmsType@3f66345c, java.util.Date
    at com.speedment.internal.core.platform.component.impl.JavaTypeMapperComponentImpl.lambda$getFromMapOrThrow$764(JavaTypeMapperComponentImpl.java:76)
    at com.speedment.internal.core.platform.component.impl.JavaTypeMapperComponentImpl$$Lambda$761/1187235233.get(Unknown Source)
    at java.util.Optional.orElseThrow(Optional.java:290)
    at com.speedment.internal.core.platform.component.impl.JavaTypeMapperComponentImpl.getFromMapOrThrow(JavaTypeMapperComponentImpl.java:76)
    at com.speedment.internal.core.platform.component.impl.JavaTypeMapperComponentImpl.apply(JavaTypeMapperComponentImpl.java:61)
    at com.speedment.internal.core.code.manager.EntityManagerImplTranslator.lambda$defaultReadEntity$785(EntityManagerImplTranslator.java:166)
    at com.speedment.internal.core.code.manager.EntityManagerImplTranslator$$Lambda$759/377777485.accept(Unknown Source)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    at java.util.stream.SortedOps$RefSortingSink$$Lambda$389/1353328890.accept(Unknown Source)
    at java.util.ArrayList.forEach(ArrayList.java:1249)
    at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
    at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.forEachOrdered(ReferencePipeline.java:423)
    at com.speedment.internal.core.code.manager.EntityManagerImplTranslator.defaultReadEntity(EntityManagerImplTranslator.java:164)
    at com.speedment.internal.core.code.manager.EntityManagerImplTranslator.make(EntityManagerImplTranslator.java:119)
    at com.speedment.internal.core.code.manager.EntityManagerImplTranslator.make(EntityManagerImplTranslator.java:58)
    at com.speedment.internal.core.code.DefaultJavaClassTranslator.get(DefaultJavaClassTranslator.java:102)
    at com.speedment.internal.core.code.DefaultJavaClassTranslator.get(DefaultJavaClassTranslator.java:63)
    at com.speedment.internal.core.code.MainGenerator$$Lambda$660/231725738.apply(Unknown Source)
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:512)
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:502)
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
    at com.speedment.internal.core.code.MainGenerator.accept(MainGenerator.java:93)
    at com.speedment.internal.gui.controller.SceneController.lambda$initialize$441(SceneController.java:221)
    at com.speedment.internal.gui.controller.SceneController$$Lambda$455/1546110104.handle(Unknown Source)
    at com.speedment.internal.gui.controller.SceneController.lambda$initialize$444(SceneController.java:262)
    at com.speedment.internal.gui.controller.SceneController$$Lambda$458/401857420.run(Unknown Source)
    at com.speedment.internal.gui.controller.ActionChoiceController.lambda$createHoverEffect$573(ActionChoiceController.java:115)
    at com.speedment.internal.gui.controller.ActionChoiceController$$Lambda$462/462700480.handle(Unknown Source)
    at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86)
    at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238)
    at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
    at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59)
    at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
    at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
    at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
    at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
    at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
    at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54)
    at javafx.event.Event.fireEvent(Event.java:198)
    at javafx.scene.Scene$MouseHandler.process(Scene.java:3758)
    at javafx.scene.Scene$MouseHandler.access$1500(Scene.java:3486)
    at javafx.scene.Scene.impl_processMouseEvent(Scene.java:1762)
    at javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2495)
    at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:350)
    at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:275)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$350(GlassViewEventHandler.java:385)
    at com.sun.javafx.tk.quantum.GlassViewEventHandler$$Lambda$306/1340072090.get(Unknown Source)
    at com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:404)
    at com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:384)
    at com.sun.glass.ui.View.handleMouseEvent(View.java:555)
    at com.sun.glass.ui.View.notifyMouse(View.java:927)
    at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at com.sun.glass.ui.win.WinApplication.lambda$null$145(WinApplication.java:101)
    at com.sun.glass.ui.win.WinApplication$$Lambda$111/1579402200.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:745)

Code generation does not support Date fields

Hi,
I'm trying to generate code from a MySQL DB but I get the following exception:

2015-10-21T18:10:54.625Z ERROR JavaFX Application Thread - Error! Failed to generate code.
java.lang.NullPointerException: The JavaTypeMapperComponent does not have a mapping for com.speedment.internal.core.config.dbms.MySqlDbmsType@76f9e4f0, java.util.Date
......

It seems caused by a field of type Date

Items in the logger output when persist fails

Items shall not be logged, since one can observe them with a logger provided to persist()

Streams are not closed properly if an Exception is thrown in a terminating operation

The finallyClose() must take a Producer, not just a value

Entity interface shall contain equals() and hashCode() support method that defines what an implementing class shall implement.

Schema names that contains dots (.) does not work

MySQL supports schema names with a dot in them from version 5.1.6 and up, but Speedment can not handle them since a class name with a dot is not allowed in Java. A solution to this might be to replace dots with underscores in the generated code and escape the schema name in SQL queries.

Implement new logger and remove Log4j dependency

Json shall not quote some types

numbers, true, false shall not be quoted. See http://json.org/

Implement a new schema(name) in AbstractRelationalDbmsHandler that just returns schemas with that name

Called in ProjectPromptController (176)

Null columns read improperly in generated code

Use rs.wasNull() to detect nulls.

Maven arcetype for Speedment

Unnecessary imports in generated application file

Generated application files have a lot of unnecessary imports. A solution for this would be to use the short name in the onInit()-method.

package com.speedment.example.mysalesinsight;

import com.speedment.example.mysalesinsight.db0.salesinfo.country.impl.CountryManagerImpl;
import com.speedment.example.mysalesinsight.db0.salesinfo.customer.impl.CustomerManagerImpl;
import com.speedment.example.mysalesinsight.db0.salesinfo.office.impl.OfficeManagerImpl;
import com.speedment.example.mysalesinsight.db0.salesinfo.product.impl.ProductManagerImpl;
import com.speedment.example.mysalesinsight.db0.salesinfo.region.impl.RegionManagerImpl;
import com.speedment.example.mysalesinsight.db0.salesinfo.salesperson.impl.SalespersonManagerImpl;
import com.speedment.internal.core.runtime.SpeedmentApplicationLifecycle;
import javax.annotation.Generated;

/**
 * A {@link
 * com.speedment.internal.core.runtime.SpeedmentApplicationLifecycle} class
 * for the {@link com.speedment.config.Project} named mysalesinsight.
 * <p>
 * This Class or Interface has been automatically generated by Speedment. Any
 * changes made to this Class or Interface will be overwritten.
 * 
 * @author Speedment
 */
@Generated("Speedment")
public class MysalesinsightApplication extends SpeedmentApplicationLifecycle<MysalesinsightApplication> {

    public MysalesinsightApplication() {
        setSpeedmentApplicationMetadata(new MysalesinsightApplicationMetadata());
    }

    @Override
    protected void onInit() {
        loadAndSetProject();
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.country.impl.CountryManagerImpl(speedment));
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.customer.impl.CustomerManagerImpl(speedment));
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.office.impl.OfficeManagerImpl(speedment));
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.product.impl.ProductManagerImpl(speedment));
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.region.impl.RegionManagerImpl(speedment));
        put(new com.speedment.example.mysalesinsight.db0.salesinfo.salesperson.impl.SalespersonManagerImpl(speedment));
        super.onInit();
    }
}

Avoiding upper case package names

I personally don't like uppercase package names, and if I'm not entirely wrong, it's not recommended to have them either.

When I design my databases, I use a convention that matches the Java-classes, like so:

CREATE TABLE `Article` (
  `articleId` int(11) NOT NULL AUTO_INCREMENT,
  `articleNumber` varchar(16) NOT NULL,
  `articleName` varchar(100) NOT NULL,
  `description` varchar(255) NOT NULL,
  PRIMARY KEY (`articleId`),
  UNIQUE KEY `articleName_UNIQUE` (`articleName`),
  UNIQUE KEY `articleNumber_UNIQUE` (`articleNumber`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

It's the same for my database names. They can contain uppercase letters.

When generating my Speedment code, my package names will get uppercase letters. It would be nice if we could avoid that. :)

XxxApplicationMetadata Methods in can only be 64k bytes.

Find another way to construct the meta data since we may run out of space for large meta data sets.

Expand/Collapse all for TreeNodes in GUI

Minor convenience improvement:

For large databases it is pretty tedious to collapse all the nodes in the tree view on the left side as they are expanded by default (e.g. to just have a quick look, quickly disable multiple tables from generation, etc,).

A right click action like e.g. "expand all" or "collapse all" would help.

The Speedment GUI is not creating the groovy source directory properly

Mail-address is saved per project

If the mail-address is stored in the user directory instead it would only have to be entered once.

Move MapStream to the API

Also, break up into two parts: interface MapStream and MapStreamImpl

Streams are not closed when streams are chained in a way that new streams are created.

e.g. stream.map() will not close the original stream

Allow String usage of Fields, UserField.ID -> "id"

Global SQL debug listener

For debug and understanding....

Map nullable columns to OptionalInt, OptionalLong etc.

Examine if getParentInterfaceMainClass can be removed

The interface Child currently requires all implementations to have the method getParentInterfaceMainClass(). It is a simple method that returns the type of the parent even if the parent is currently not set. The issue is that Child has many concrete implementations and we should therefore examine if the Child-interface can be simplified, maybe by removing the getParentInterfaceMainClass-method.

.equals and .equalsIgnoreCase in SQL does not match correctly

The predicate builders for .equals and .equalsIgnoreCase does not generate correct SQL since the = operator in SQL is case insensitive for non-binary strings. The SQL generator must therefore compare the byte values for strings if the comparing should be case sensitive.

Move JsonEncoder to api package

Possibility to configure package name inside GUI

Minor convenience improvement: configure the package name inside the GUI instead of directly in the groovy script.

Define Exception handling for Entity.stream()