pushtorefresh/storio

Reactive API for SQLiteDatabase and ContentResolver.

StorIO — modern API for SQLiteDatabase and ContentResolver

Overview:

• Powerful & Simple set of Operations: Put, Get, Delete
• API for Humans: Type Safety, Immutability & Thread-Safety
• Convenient builders with compile-time guarantees for required params. Forget about 6-7 null in queries
• Optional Type-Safe Object Mapping, if you don't want to work with Cursor and ContentValues you don't have to
• No reflection in Operations and no annotations in the core, also StorIO is not ORM
• Full control over queries, transaction and object mapping
• Every Operation over StorIO can be executed as blocking call or as io.reactivex.Flowable/io.reactivex.Single/io.reactivex.Completable/io.reactivex.Maybe
• RxJava as first class citizen, but it's not required dependency!
• Reactive: io.reactivex.Flowable from Get Operation will observe changes in StorIO (SQLite or ContentProvider) and receive updates automatically
• StorIO is replacements for SQLiteDatabase and ContentResolver APIs
• StorIO + RxJava is replacement for Loaders API
• We are working on MockStorIO (similar to MockWebServer) for easy unit testing

---

Why StorIO?

• Simple concept of just three main Operations: Put, Get, Delete -> less bugs
• Almost everything is immutable and thread-safe -> less bugs
• Builders for everything make code much, much more readable and obvious -> less bugs
• Our builders give compile time guarantees for required parameters -> less bugs
• StorIO annotated with @NonNull and @Nullable annotations -> less bugs
• Open Source -> less bugs
• Documentation, Sample app and Design tests -> less bugs
• StorIO has unit and integration tests -> less bugs
• Less bugs -> less bugs

Documentation:

• Why we made StorIO
• StorIO SQLite
• StorIO ContentResolver

Easy ways to learn how to use StorIO -> check out Documentation, Design Tests and Sample App:

• Design tests for StorIO SQLite
• Design tests for StorIO ContentResolver
• Sample App

Download:

// If you need StorIO for SQLite
implementation 'com.pushtorefresh.storio3:sqlite:3.0.1'

// If you need StorIO for ContentResolver
implementation 'com.pushtorefresh.storio3:content-resolver:3.0.1'

// Notice that RxJava is optional dependency for StorIO,
// So if you need it -> please add it manually.

You can find all releases on Maven Central.

Some examples

Get list of objects from SQLiteDatabase

List<Tweet> tweets = storIOSQLite
  .get()
  .listOfObjects(Tweet.class) // Type safety
  .withQuery(Query.builder() // Query builder
    .table("tweets")
    .where("author = ?")
    .whereArgs("artem_zin") // Varargs Object..., no more new String[] {"I", "am", "tired", "of", "this", "shit"}
    .build()) // Query is immutable — you can save it and share without worries
  .prepare() // Operation builder
  .executeAsBlocking(); // Control flow is readable from top to bottom, just like with RxJava

Put something to SQLiteDatabase

storIOSQLite
  .put() // Insert or Update
  .objects(someTweets) // Type mapping!
  .prepare()
  .executeAsBlocking();

Delete something from SQLiteDatabase

storIOSQLite
  .delete()
  .byQuery(DeleteQuery.builder()
    .table("tweets")
    .where("timestamp <= ?")
    .whereArgs(System.currentTimeMillis() - 86400) // No need to write String.valueOf()
    .build())
  .prepare()
  .executeAsBlocking();

Reactive? Single.just(true)!

Get something as io.reactivex.Flowable and receive updates!

storIOSQLite
  .get()
  .listOfObjects(Tweet.class)
  .withQuery(Query.builder()
    .table("tweets")
    .build())
  .prepare()
  .asRxFlowable(BackpressureStrategy.LATEST) // Get Result as io.reactivex.Flowable and subscribe to further updates of tables from Query!
  .observeOn(mainThread()) // All Rx operations work on Schedulers.io()
  .subscribe(tweets -> { // Please don't forget to dispose
  	  // Will be called with first result and then after each change of tables from Query
  	  // Several changes in transaction -> one notification
  	  adapter.setData(tweets);
  	}
  );

Want to work with plain Cursor, no problems

Cursor cursor = storIOSQLite
  .get()
  .cursor()
  .withQuery(Query.builder() // Or RawQuery
    .table("tweets")
    .where("who_cares = ?")
    .whereArgs("nobody")
    .build())
  .prepare()
  .executeAsBlocking();

How object mapping works?

You can set default type mappings when you build instance of StorIOSQLite or StorIOContentResolver

StorIOSQLite storIOSQLite = DefaultStorIOSQLite.builder()
  .sqliteOpenHelper(someSQLiteOpenHelper)
  .addTypeMapping(Tweet.class, SQLiteTypeMapping.<Tweet>builder()
    .putResolver(new TweetPutResolver()) // object that knows how to perform Put Operation (insert or update)
    .getResolver(new TweetGetResolver()) // object that knows how to perform Get Operation
    .deleteResolver(new TweetDeleteResolver())  // object that knows how to perform Delete Operation
    .build())
  .addTypeMapping(...)
  // other options
  .build(); // This instance of StorIOSQLite will know how to work with Tweet objects

You can override Operation Resolver per each individual Operation, it can be useful for working with SQL JOIN.

---

To save you from coding boilerplate classes we created Annotation Processor which will generate PutResolver, GetResolver and DeleteResolver at compile time, you just need to use generated classes

Notice that annotation processors are not part of the library core, you can work with StorIO without them, we just made them to save you from boilerplate.

StorIOSQLite:

dependencies {
  implementation 'com.pushtorefresh.storio3:sqlite-annotations:insert-latest-version-here'

  annotationProcessor 'com.pushtorefresh.storio3:sqlite-annotations-processor:insert-latest-version-here'
}

StorIOContentResolver:

dependencies {
  implementation 'com.pushtorefresh.storio3:content-resolver-annotations:insert-latest-version-here'

  annotationProcessor 'com.pushtorefresh.storio3:content-resolver-annotations-processor:insert-latest-version-here'
}

@StorIOSQLiteType(table = "tweets")
public class Tweet {

  // Annotated fields should have package-level visibility.
  @StorIOSQLiteColumn(name = "author")
  String author;

  @StorIOSQLiteColumn(name = "content")
  String content;

  // Please leave default constructor with package-level visibility.
  Tweet() {}
}

Kotlin:

In order to make annotation processors work with Kotlin you need to add the following to your build.gradle:

apply plugin: 'kotlin-kapt'

Then use kapt configuration instead of annotationProcessor.

@StorIOSQLiteType(table = "tweets")
data class Tweet @StorIOSQLiteCreator constructor(
       StorIOSQLiteColumn(name = "author") val author: String,
       StorIOSQLiteColumn(name = "content") val content: String)

AutoValue:

@AutoValue
@StorIOSQLiteType(table = "tweets")
public abstract class Tweet {

  // Annotated methods should have package-level or public visibility.
  @StorIOSQLiteColumn(name = "author")
  abstract String author();

  @StorIOSQLiteColumn(name = "content")
  abstract String content();

  // Parameters order depends on declaration order.
  @StorIOSQLiteCreator
  static Tweet create(String author, String content) {
    return new AutoValue_Tweet(author, content);
  }
}

Annotation Processor will generate three classes in same package as annotated class during compilation:

• TweetStorIOSQLitePutResolver
• TweetStorIOSQLiteGetResolver
• TweetStorIOSQLiteDeleteResolver

You just need to apply them:

StorIOSQLite storIOSQLite = DefaultStorIOSQLite.builder()
  .sqliteOpenHelper(someSQLiteOpenHelper)
  .addTypeMapping(Tweet.class, SQLiteTypeMapping.<Tweet>builder()
    .putResolver(new TweetStorIOSQLitePutResolver()) // object that knows how to perform Put Operation (insert or update)
    .getResolver(new TweetStorIOSQLiteGetResolver()) // object that knows how to perform Get Operation
    .deleteResolver(new TweetStorIOSQLiteDeleteResolver())  // object that knows how to perform Delete Operation
    .build())
  .addTypeMapping(...)
  // other options
  .build(); // This instance of StorIOSQLite will know how to work with Tweet objects

BTW: Here is a class with all types of fields, supported by StorIO SQLite Annotation Processor.

Few tips about Operation Resolvers:

• If your entities are immutable or they have builders or they use AutoValue/AutoParcel -> write your own Operation Resolvers
• If you want to write your own Operation Resolver -> take a look at Default Operation resolvers, they can fit your needs
• Via custom Operation Resolvers you can implement any Operation as you want -> store one object in multiple tables, use custom sql things and so on

API of StorIOContentResolver is same.

---

Versioning:

Because StorIO works with important things like User data and so on, we use Semantic Versioning 2.0.0 scheme for releases (http://semver.org).

Short example: 1.2.3 -> MAJOR.MINOR.PATCH

• MAJOR version changes when we make incompatible API changes.
• MINOR version changes when we add functionality in a backwards-compatible manner.
• PATCH version changes when we make backwards-compatible bug fixes.

Please read CHANGELOG and check what part of the version has changed, before switching to new version.

Architecture:

StorIOSQLite and StorIOContentResolver — are abstractions with default implementations: DefaultStorIOSQLite and DefaultStorIOContentResolver.

It means, that you can have your own implementation of StorIOSQLite and StorIOContentResolver with custom behavior, such as memory caching, verbose logging and so on or mock implementation for unit testing (we are working on MockStorIO).

One of the main goals of StorIO — clean API for Humans which will be easy to use and understand, that's why StorIOSQLite and StorIOContentResolver have just several methods, but we understand that sometimes you need to go under the hood and StorIO allows you to do it: StorIOSQLite.LowLevel and StorIOContentResolver.LowLevel encapsulates low-level methods, you can use them if you need, but please try to avoid it.

Queries

All Query objects are immutable, you can share them safely.

Concept of Prepared Operations

You may notice that each Operation (Get, Put, Delete) should be prepared with prepare(). StorIO has an entity called PreparedOperation, and you can use them to perform group execution of several Prepared Operations or provide PreparedOperation as a return type of your API (for example in Model layer) and client will decide how to execute it: executeAsBlocking() or asRxFlowable(). Also, Prepared Operations might be useful for ORMs based on StorIO.

You can customize behavior of every Operation via Resolvers: GetResolver, PutResolver, DeleteResolver.

Rx Support Design

Every Operation can be executed as io.reactivex.Flowable, io.reactivex.Single, io.reactivex.Completable or io.reactivex.Maybe. Get Operations will be automatically subscribed to the updates of the data. Every rx operation runs on Schedulers.io(). You can change it by defaultRxScheduler() or set it to null to execute on current thread.

3rd party additions/integrations for StorIO

• CodeGenUnderStorIO allows you generate Java classes for db entities from the db schema built in some visual editor.

---

Master branch build status:

Made with love in Pushtorefresh.com by @artem_zin, @nikitin-da and @geralt-encore

storio's Issues

Integration tests for bambooStorage.get().withQuery(rawQuery)

Add README

Add option for providing listeners for CRUD operations

It would be a great feature!

For example, you'll be able to send event about change to the event bus or do a custom post processing of change in ContentProvider

rawQuery support

Unit tests for PreparedDelete

PutResolver is bad

Create common thing for put(object) and delete(object) operations

Integrate with Travis CI

BambooStorage.getAsList() not always close cursor

BambooStorage.Internal should provide API for transactions

Currently, we need several methods:

areTransactionsSupported()
beginTransaction()
setTransactionSuccessful()
endTransaction()

Just like pure SQLiteDatabase

Will be useful for #40 put(collection) and delete(collection) operations

Make BambooStorage.buildUri() public and available for overriding

Add support for working with SQLiteDatabase directly

Many developers don't want to deal with ContentProvider, so, we can add support of direct usage BambooStorage with SQLiteDatabase

execSql support

I think it should be part of BambooStorage, we can provide blocking and Observable variants.

bambooStorage().execSql(SQLQuery).prepare()
Execute a single SQL statement that is NOT a SELECT/INSERT/UPDATE/DELETE.

And SQLQuery + SQLQueryBuilder, which will be same as RawQuery

@nikitin-da any comments?

Add overload for put().objects(T... objects)

Create BambooStorageSQLiteDatabase implementation

Annotation processing for generating entities parsers and serializers

Should be nice

Better naming for type of `get()` operation results

get().asCursor() -> get().cursor()

get().asListOfObjects() -> get().listOfObjects()

Update RxJava to 1.0.8

Normal methods names for IBambooStorableItem

I realised, that current IBambooStorableItem methods names are not good, so, they will be renamed

Maybe we should exclude support.v4 from dependencies

Currently it used just for NonNull and Nullable annotations, so maybe it's better to forget about this annotations and remove support.v4

Option for transaction enable/disable for put(collection) and delete(collection) operations

Via transactions (#41)

Like this:

bambooStorage()
            .delete()
            .objects(users)
            .mapFunc(User.MAP_TO_DELETE_QUERY)
            .inTransaction(true)
            .prepare()
            .createObservable();

Add Observable variants for put, putAll, delete and deleteAll operations

Documentation!

There is no JavaDoc at all...

get().numberOfResults()

It should be part of get() operation, like get().cursor() or get().listOfObjects() -> get().numberOfResults()

In fact, it's get().cursor() with returning cursor.getCount() and auto closing, but will be better to use COUNT from SQL for this

Unit tests for PreparedGet

Unit tests for PreparedPut

Request builder?

What about request builder instead of tons of methods with overloads

Add BambooStorage.addAll() method

Refactor add() update() addOrUpdate() to one method save() or put()

Add support for RxJava

Check Observable support for Non-blocking IO, should be useful and interesting

Rename operation builder part "mapFunc()" to -> "withMapFunc()"

For better readability

Overload put().objects()

Support two overloads

bambooStorage.put().objects(@NonNull T... objects)
bambooStorage.put().objects(@NonNull Iterable<T> objects)

More integration tests for bambooStorage.query()

Reactive streams for bambooStorage.get()

We should provide an option for using observable created for get() operation as reactive stream:

For example:

Observable<List<User>> observableUsers = bambooStorage
  .get()
  .listOfObjects(User.class)
  .withMapFunc(User.MAP_FROM_CURSOR)
  .withQuery(Users.QUERY_ALL)
  .prepare()
  .createObservableStream();

observableUsers.subscribe(new Observer<List<Users>>() {
  @Override public void onNext(List<User> users) {
      // will be called after each change in users table
  }

  // ...
});

API for get operation should be more similar to put operation

Or vice versa.

Personally, I don't like difference:

List<User> users = bambooStorage()
                .get()
                .query(new QueryBuilder()
                        .table("users")
                        .where("email = ?")
                        .whereArgs("[email protected]")
                        .build())
                .resultAsObjects(User.MAP_FROM_CURSOR) // <-- this 
                .executeAsBlocking();

Observable<SinglePutResult> observablePutResult = bambooStorage()
                .put()
                .object(user) // <-- and this
                .mapFunc(User.MAP_TO_CONTENT_VALUES)
                .putResolver(User.PUT_RESOLVER)
                .prepare() // <-- and this
                .createObservable();

API Design for delete operation

Prepare Design of API for delete operation

BambooStorage should set internal id from cursor

put().contentValues(array) can reuse put().contentValues(Iterable<ContentValues>)

With Arrays.asList()

put() operation for collection of ContentValues

Refactor put() operation

Remove putResolver()

Use _id

Upload to maven central

BambooStorage should allow user to set default transformer for Observables

For example, I want to make all queries on Schedulers.io() and observe them on AndroidSchedulers.mainThread()

It can be done via compose() operator, here is useful article from @dlew http://blog.danlew.net/2015/03/02/dont-break-the-chain/

Possible syntax:

Transformer<Observable, Observable> transformer = new Transformer<>() {
  @Override Observable transform(Observable observable) {
    return observable
                 .subscribeOn(Schedulers.io())
                 .observeOn(AndroidSchedulers.mainThread());
  }
};

BambooStorage bambooStorage = new BSSQLiteDatabase.Builder()
  .db(sqliteDatabase)
  .observableTransformer(transformer)
  .build();

Rename design tests for consistency

All design tests for operations should be named like this: "operationDesignTest"

Support for custom internal id field name

"@ContentPathForContentResolver" annotation can be used for this

Add countOf operation

Use case:

User just want to get count of some query results without managing cursor or wasting memory and time on collection creation and entities parsing

Should provide blocking and observable variants

Rename onAnyCRUDOperation() to onAnyModifyOperation()

Add Sample app

Add checks for all Builder classes

Implement put() operation for collection of objects

Will be great to use transactions for this

Add license

Rename bambooStorage#getInternal()

Because it's very similar to get() operation and can confuse user.

One of the variants: just internal()