lwhite1/daboodb

A simple, pure java document store

daboodb's Issues

Improve configuration settings management and startup logic

This involves at least 2 things.

First, we have to make sure we have an approach to loading configuration files that works well at startup regardless of whether we're in Embedded or ClientServer mode.

Second, we need a way to make changes easily so the test framework can exercise every implementation of a given interface, and not just the one set by default in the configuration properties files.

Settings properties files cannot be found at runtime when daboo used from a different project

java.io.FileNotFoundException: src/main/resources/server.properties (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at org.daboodb.daboo.server.ServerSettings.loadServerSettings(ServerSettings.java:154)
at org.daboodb.daboo.server.ServerSettings.getInstance(ServerSettings.java:40)
at org.daboodb.daboo.server.Database.init(Database.java:199)
at org.daboodb.daboo.server.Database.writeLog(Database.java:213)
at org.daboodb.daboo.server.Database.handleRequest(Database.java:61)
at org.daboodb.daboo.server.DirectCommServer.handleRequest(DirectCommServer.java:26)
at org.daboodb.daboo.client.DirectCommClient.sendRequest(DirectCommClient.java:19)
at org.daboodb.daboo.client.DbClient.write(DbClient.java:91)
at org.daboodb.performance.LoadRunner1.main(LoadRunner1.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Exception in thread "main" org.daboodb.daboo.shared.exceptions.RuntimeRequestException: A Throwable was caught handling a WRITE request
at org.daboodb.daboo.server.Database.handleRequest(Database.java:79)
at org.daboodb.daboo.server.DirectCommServer.handleRequest(DirectCommServer.java:26)
at org.daboodb.daboo.client.DirectCommClient.sendRequest(DirectCommClient.java:19)
at org.daboodb.daboo.client.DbClient.write(DbClient.java:91)
at org.daboodb.performance.LoadRunner1.main(LoadRunner1.java:30)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: org.daboodb.daboo.shared.exceptions.StartupException: Unable to load server settings as Java properties
at org.daboodb.daboo.server.ServerSettings.loadServerSettings(ServerSettings.java:159)
at org.daboodb.daboo.server.ServerSettings.getInstance(ServerSettings.java:40)
at org.daboodb.daboo.server.Database.init(Database.java:199)
at org.daboodb.daboo.server.Database.writeLog(Database.java:213)
at org.daboodb.daboo.server.Database.handleRequest(Database.java:61)
... 9 more
Caused by: java.io.FileNotFoundException: src/main/resources/server.properties (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at org.daboodb.daboo.server.ServerSettings.loadServerSettings(ServerSettings.java:154)
... 13 more

Implement optimistic locking

make it optional. Embedded mode won't normally need it.

Add support for non-lexicographical comparators

should support:

• integers
• doubles
• longs

Review and fix all exception handling

Eliminate separate DocumentContents interface

I think we could do without the DocumentContent interface and have docs implement Document directly, or create a DocumentWrapper class. We could also provide an AbstractDocumentWrapper that made this explicit.

My sense is that this would be a usability improvement for devs, but that needs to be evaluated.

Add compression plugin for lz4 pure java

Implemented

Implement JMH-based performance testing

Implement hash-map backed storage options

Provide a way to configure the system for better performance and scaling when range searches are not required.

Implement logging

Using HFT chronicle? Link: https://github.com/OpenHFT/Chronicle-Logger

In any case we should use fast logging with a binary format. Presumably, that format would be protobufs.

There is a protobuf file already in dabu for requests, at /src/main/resources/proto/Request.proto
We'd need another one for logging.

The generated java files currently go in: src/main/java/org/dabudb/dabu/generated/protobufs/

We're using the latest 3.0 Beta version of the protobuf compiler: 3.0.0-beta-2.

Implement performance and scalability testing using YCSB

Implement performance logging

We should track the length of time that transactions take,

Insufficient test coverage

Coverage should be ~90% (excluding generated protobuf code) for version 1.0

Implement flat transactions

All reads and writes in a batch should be transactional. (Turn off for better performance?)

Eliminate Todos for V. 1.0

self explanatory.

Review implementation and use of MessagePipe

Should we go back to having serialization be explicit and providing an implementation that works with protobufs? Current implementation seems not well thought out