palantir / atlasdb Goto Github PK

View Code? Open in Web Editor NEW

43.0 43.0 6.0 112.27 MB

Transactional Distributed Database Layer

Home Page: https://palantir.github.io/atlasdb/

License: Apache License 2.0

Java 99.15% Shell 0.22% Groovy 0.39% Clojure 0.11% Python 0.06% Dockerfile 0.02% FreeMarker 0.05%

octo-correct-managed transactions

atlasdb's People

Contributors

Stargazers

Watchers

Forkers

slotermeyer tancyx annielzy shawndanger

atlasdb's Issues

Split Cassandra-shading project into a separate repo

Postgres kvs uses an in-memory timestamp service

This won't work even for a single node postgres because timestamps need to be monotonic across restarts.

NamespaceMappingKeyValueService does not implement getMetadataForTable correctly

According to the docs, this should return an empty byte array if the table in question does not exist. Instead, it throws an exception.

Cassandra Quorum Deletes

We should definitely leave default delete consistency to ALL, but I don't think we should necessarily be preventing people from using cassandra's features here. People should be able to run deletes at quorum consistency with the understanding they are then responsible for proper maintenance (i.e. repairing) of their cluster.

Make Checkstyle stricter

Dependency on com.palantir.patches.sourceforge:trove3:3.0.3-p5 prevents jcenter-only dep resolution

Should make this a proper publish workflow or shade our own project to make it so jcenter only resolutions are possible.

Test issue

edit

Have seen (3 times now) AtlasDB in HA mode get into a 'failed' state where it won't commit transactions

Hey guys,
We've been running our ETE tests with AtlasDB in HA mode with 3 nodes, all backed by Cassandra.

3 times now (in ~2 weeks), I've seen AtlasDB get itself into a really bad state - we get into a state where every transaction fails due to TransactionConflictExceptions.

We take out a lock before performing our transactions (talk to @joelea or check our source for details) in order to maintain a linearised commit log, and we take out a write lock on each write request (copied from the Gatekeeper code). TransactionConflictExceptions should never happen, because only one transaction should be executing at any one time.

Turning everything off and on again appears to resolve the issue - it doesn't appear to resolve itself within about 10 minutes (so not without human intervention).

The stack traces we see are like

salt1.palantir.dev         | ERROR [2016-04-07 08:05:07,136] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 61aab62e693f6878
salt1.palantir.dev         | ! com.palantir.atlasdb.transaction.api.TransactionConflictException: Transaction Conflict after 35 ms for table: salt.blobs with start timestamp: 1013585
salt1.palantir.dev         | ! Another transaction wrote and committed between our start and end ts. It is possible we are a long running transaction. Cells:
salt1.palantir.dev         | ! [
salt1.palantir.dev         | !  CellConflict [cell=Cell [rowName={"bucket": "ri.salt.test.test.saltbucket", "key": "ri.salt.test.key.testKey"}, columnName=t], theirStart=2010006, theirCommit=2010007],
salt1.palantir.dev         | !  CellConflict [cell=Cell [rowName={"bucket": "ri.salt.test.test.saltbucket", "key": "ri.salt.test.key.testKey"}, columnName=s], theirStart=2010006, theirCommit=2010007],
salt1.palantir.dev         | !  CellConflict [cell=Cell [rowName={"bucket": "ri.salt.test.test.saltbucket", "key": "ri.salt.test.key.testKey"}, columnName=b], theirStart=2010006, theirCommit=2010007],
salt1.palantir.dev         | !  CellConflict [cell=Cell [rowName={"bucket": "ri.salt.test.test.saltbucket", "key": "ri.salt.test.key.testKey"}, columnName=v], theirStart=2010006, theirCommit=2010007],
salt1.palantir.dev         | !  CellConflict [cell=Cell [rowName={"bucket": "ri.salt.test.test.saltbucket", "key": "ri.salt.test.key.testKey"}, columnName=c], theirStart=2010006, theirCommit=2010007],
salt1.palantir.dev         | ! ]
salt1.palantir.dev         | !
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.api.TransactionConflictException.create(TransactionConflictException.java:111) ~[atlasdb-api-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.throwIfWriteAlreadyCommitted(SnapshotTransaction.java:1287) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.throwIfConflictOnCommit(SnapshotTransaction.java:1261) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.commitWrites(SnapshotTransaction.java:1141) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.commit(SnapshotTransaction.java:1102) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.commit(SnapshotTransaction.java:1063) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.ForwardingTransaction.commit(ForwardingTransaction.java:84) ~[atlasdb-client-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.AbstractTransactionManager.runTaskThrowOnConflict(AbstractTransactionManager.java:70) ~[atlasdb-client-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager.finishRunTaskWithLockThrowOnConflict(SnapshotTransactionManager.java:155) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SerializableTransactionManager.finishRunTaskWithLockThrowOnConflict(SerializableTransactionManager.java:30) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager.runTaskWithLocksThrowOnConflict(SnapshotTransactionManager.java:121) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.SerializableTransactionManager.runTaskWithLocksThrowOnConflict(SerializableTransactionManager.java:30) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.AbstractLockAwareTransactionManager.runTaskWithLocksWithRetry(AbstractLockAwareTransactionManager.java:64) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.atlasdb.transaction.impl.AbstractLockAwareTransactionManager.runTaskWithLocksWithRetry(AbstractLockAwareTransactionManager.java:99) ~[atlasdb-impl-shared-0.3.34.jar:na]
salt1.palantir.dev         | ! at com.palantir.salt.store.GloballyLockedTransactionManager.runTaskWithRetry(GloballyLockedTransactionManager.java:39) ~[salt-0.8.0-29-g53abf46.dirty.jar:na]
salt1.palantir.dev         | ! at com.palantir.salt.store.Inserter.insert(Inserter.java:78) ~[salt-0.8.0-29-g53abf46.dirty.jar:na]
salt1.palantir.dev         | ! at com.palantir.salt.store.StoreService.createOrUpdateBlob(StoreService.java:85) ~[salt-0.8.0-29-g53abf46.dirty.jar:na]
salt1.palantir.dev         | ! at com.palantir.salt.SaltResource.createOrUpdateObject(SaltResource.java:83) ~[salt-0.8.0-29-g53abf46.dirty.jar:na]
salt1.palantir.dev         | ! at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source) ~[na:na]
salt1.palantir.dev         | ! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_72]
salt1.palantir.dev         | ! at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_72]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:160) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:308) ~[jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:291) [jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1140) [jersey-server-2.19.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:471) [jersey-container-servlet-core-2.22.1.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:425) [jersey-container-servlet-core-2.22.1.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:383) [jersey-container-servlet-core-2.22.1.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:336) [jersey-container-servlet-core-2.22.1.jar:na]
salt1.palantir.dev         | ! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:223) [jersey-container-servlet-core-2.22.1.jar:na]
salt1.palantir.dev         | ! at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49) [dropwizard-jetty-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83) [jetty-servlets-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300) [jetty-servlets-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at io.dropwizard.jetty.BiDiGzipFilter.doFilter(BiDiGzipFilter.java:132) [dropwizard-jetty-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:29) [dropwizard-servlets-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:43) [dropwizard-jersey-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:38) [dropwizard-jersey-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at com.palantir.dropwizard.websecurity.filters.AppSecurityFilter.doFilter(AppSecurityFilter.java:62) [dropwizard-web-security-0.2.0.jar:na]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:240) [metrics-jetty9-3.1.2.jar:3.1.2]
salt1.palantir.dev         | ! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:51) [dropwizard-jetty-0.9.2.jar:0.9.2]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:95) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:159) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.Server.handle(Server.java:499) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) [jetty-server-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [jetty-io-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) [jetty-util-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) [jetty-util-9.2.13.v20150730.jar:9.2.13.v20150730]
salt1.palantir.dev         | ! at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]

running AtlasDB

Sorry for submitting this question under Issues, I could not find a more suitable place.

AtlasDB seems like a very nice project. Could you please provide some basic commands on how to run AtlasDB? I figured out that I needed Gradle first, I installed it and issued 'gradle' successfully, as well as 'gradle build'. What is the next step? How do I actually run one of the projects now?

For instance, in the Wiki I see a section called "Running a standalone atlas server that talks json/rest" and it shows this line: 'AtlasDbServer server src/dist/atlasdb_standalone.yml'. But how do you run that line? When I issue it (from a top directory like /usr/local/atlasdb/), I get 'bash: AtlasDbServer: command not found'. I did play a little with the java command and by supplying a classpath, as well as with some other options, but not much luck.

Thanks in advance.

Make ClientSplitLockService a Dynamic Proxy

This will be easier by first making LockClient serializable and removing the split between LockService.lockAnonymously() and LockService.lockWithClient()

Cassandra error when you delete all columns in a row in table with dynamic columns

If you add a row to a table with dynamic columns and then delete all the columns in it before commiting, then a future get on the row throws the following NPE:

ERROR [2016-02-09 23:56:23,230] com.palantir.remoting.http.server.JsonExceptionMapper: null
! org.apache.cassandra.thrift.InvalidRequestException: null
! at org.apache.cassandra.thrift.Cassandra$get_range_slices_result$get_range_slices_resultStandardScheme.read(Cassandra.java:17600) ~[cassandra-thrift-2.1.7.jar:2.1.7]
! at org.apache.cassandra.thrift.Cassandra$get_range_slices_result$get_range_slices_resultStandardScheme.read(Cassandra.java:17567) ~[cassandra-thrift-2.1.7.jar:2.1.7]
! at org.apache.cassandra.thrift.Cassandra$get_range_slices_result.read(Cassandra.java:17493) ~[cassandra-thrift-2.1.7.jar:2.1.7]
! at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) ~[libthrift-0.9.2.jar:0.9.2]
! at org.apache.cassandra.thrift.Cassandra$Client.recv_get_range_slices(Cassandra.java:811) ~[cassandra-thrift-2.1.7.jar:2.1.7]
! at org.apache.cassandra.thrift.Cassandra$Client.get_range_slices(Cassandra.java:795) ~[cassandra-thrift-2.1.7.jar:2.1.7]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraKeyValueService$7$1.apply(CassandraKeyValueService.java:853) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraKeyValueService$7$1.apply(CassandraKeyValueService.java:826) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer.runWithGoodResource(CassandraClientPoolingContainer.java:117) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraClientPoolingContainer.runWithPooledResource(CassandraClientPoolingContainer.java:97) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.common.pooling.ForwardingPoolingContainer.runWithPooledResource(ForwardingPoolingContainer.java:30) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.pooling.ForwardingPoolingContainer.runWithPooledResource(ForwardingPoolingContainer.java:30) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.RetriablePoolingContainer.runWithPooledResource(RetriablePoolingContainer.java:56) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraKeyValueService$7.getPage(CassandraKeyValueService.java:826) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraKeyValueService$7.getFirstPage(CassandraKeyValueService.java:817) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.atlasdb.keyvalue.cassandra.CassandraKeyValueService$7.getFirstPage(CassandraKeyValueService.java:814) ~[atlasdb-cassandra-0.3.23.jar:na]
! at com.palantir.util.paging.AbstractPagingIterable$PagingIterator.computeNext(AbstractPagingIterable.java:60) ~[atlasdb-commons-0.3.23.jar:na]
! ... 105 common frames omitted
! Causing: com.palantir.common.exception.PalantirRuntimeException: null
! at com.palantir.common.base.Throwables.createPalantirRuntimeException(Throwables.java:100) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.Throwables.createPalantirRuntimeException(Throwables.java:92) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.Throwables.throwUncheckedException(Throwables.java:88) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.util.paging.AbstractPagingIterable$PagingIterator.computeNext(AbstractPagingIterable.java:62) ~[atlasdb-commons-0.3.23.jar:na]
! at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-18.0.jar:na]
! at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-18.0.jar:na]
! at com.palantir.common.base.ClosableIterators$1.hasNext(ClosableIterators.java:32) ~[atlasdb-commons-0.3.23.jar:na]
! at com.google.common.collect.Iterators$9.hasNext(Iterators.java:935) ~[guava-18.0.jar:na]
! at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:268) ~[guava-18.0.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction$BatchSizeIncreasingRangeIterator.getBatch(SnapshotTransaction.java:762) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction$5.computeNext(SnapshotTransaction.java:666) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction$5.computeNext(SnapshotTransaction.java:663) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-18.0.jar:na]
! at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-18.0.jar:na]
! at com.google.common.collect.Iterators$5.hasNext(Iterators.java:547) ~[guava-18.0.jar:na]
! at com.palantir.common.base.ClosableIterators$1.hasNext(ClosableIterators.java:32) ~[atlasdb-commons-0.3.23.jar:na]
! at com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43) ~[guava-18.0.jar:na]
! at com.google.common.collect.Iterators$PeekingImpl.hasNext(Iterators.java:1149) ~[guava-18.0.jar:na]
! at com.palantir.common.collect.IteratorUtils$4.computeNext(IteratorUtils.java:171) ~[atlasdb-commons-0.3.23.jar:na]
! at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-18.0.jar:na]
! at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-18.0.jar:na]
! at com.google.common.collect.TransformedIterator.hasNext(TransformedIterator.java:43) ~[guava-18.0.jar:na]
! at com.google.common.collect.Iterators$7.computeNext(Iterators.java:650) ~[guava-18.0.jar:na]
! at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143) ~[guava-18.0.jar:na]
! at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138) ~[guava-18.0.jar:na]
! at com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43) ~[guava-18.0.jar:na]
! at com.google.common.collect.ForwardingIterator.hasNext(ForwardingIterator.java:43) ~[guava-18.0.jar:na]
! at com.google.common.collect.Iterators$6.hasNext(Iterators.java:617) ~[guava-18.0.jar:na]
! at com.palantir.common.base.BatchingVisitableFromIterable.batchAcceptSizeHint(BatchingVisitableFromIterable.java:64) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.AbstractBatchingVisitable.batchAccept(AbstractBatchingVisitable.java:39) ~[atlasdb-commons-0.3.23.jar:na]
! at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) ~[na:na]
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_31]
! at java.lang.reflect.Method.invoke(Method.java:483) ~[na:1.8.0_31]
! at com.palantir.common.proxy.SingleCallProxy.invoke(SingleCallProxy.java:48) ~[atlasdb-commons-0.3.23.jar:na]
! at com.sun.proxy.$Proxy73.batchAccept(Unknown Source) ~[na:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.getBatchingVisitableFromIterator(SnapshotTransaction.java:602) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction.access$700(SnapshotTransaction.java:136) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransaction$2.batchAcceptSizeHint(SnapshotTransaction.java:579) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.common.base.AbstractBatchingVisitable.batchAccept(AbstractBatchingVisitable.java:43) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.BatchingVisitables$12.batchAcceptSizeHint(BatchingVisitables.java:293) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.AbstractBatchingVisitable.batchAccept(AbstractBatchingVisitable.java:43) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.BatchingVisitables$17.batchAccept(BatchingVisitables.java:489) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.BatchingVisitableView.forEach(BatchingVisitableView.java:115) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.common.base.BatchingVisitableView.forEach(BatchingVisitableView.java:127) ~[atlasdb-commons-0.3.23.jar:na]
! at com.palantir.foundry.catalog.store.atlas.AtlasCatalogStore.lambda$visitAllFiles$67(AtlasCatalogStore.java:651) ~[foundry-catalog-1.0.0-beta10-dev-65-g17901ab.dirty.jar:na]
! at com.palantir.foundry.catalog.store.atlas.AtlasCatalogStore$$Lambda$30/1836991353.execute(Unknown Source) ~[na:na]
! at com.palantir.atlasdb.transaction.impl.AbstractTransactionManager.runTaskThrowOnConflict(AbstractTransactionManager.java:68) ~[atlasdb-client-0.3.23.jar:na]
! at com.palantir.atlasdb.transaction.impl.SnapshotTransactionManager.runTaskReadOnly(SnapshotTransactionManager.java:207) ~[atlasdb-impl-shared-0.3.23.jar:na]
! at com.palantir.foundry.catalog.store.atlas.AtlasCatalogStore.visitAllFiles(AtlasCatalogStore.java:617) ~[foundry-catalog-1.0.0-beta10-dev-65-g17901ab.dirty.jar:na]```

@rjullman

Default Intellij Configuration Mistakes

By default intellij configs created with ./gradlew intellij mess up organized imports.

Most notably:
"Class count to use import with ''" is low (should be as large as possible)
"Names count to use static import with ''" is low
"Packages to Use Import with '*'" has entries

LMDB as KeyValueService

Have you tried LMDB as a KeyValueService? Would you be interested in such a contribution?

https://github.com/deephacks/lmdbjni

Thoughts?

Merge atlas-spi and atlas-config into a single project

Key value services don't agree on what 'all table names' means

Dynamic columns sort order

Hi,
As it stands, dynamic columns are unordered when retrieved through the schema API, although they are presumably ordered in the (Cassandra) backend. RowResult still maintains a SortedMap for columns, but this gets flattened into a HashMap in schema-generated code. Can we export the sort order all the way?
R.

Remove Unused Code

Enable @Ignore-d Tests in AbstractSweeperTest

This will probably require refactoring the BackgroundSweeper to allow the test to control table priority selection.

Combine AtlasDB lock factories into a single factory class

Re #57

Use Immutables for Metadata and Definition

Recombine leveldb and cassandra tests now that we ignore tests on Travis

Don't think these need to be split into separate projects anymore now that we ignore tests on Travis (which kills both of those tests due to total system load).

Namespace logging uses Validate.isTrue incorrectly

Namespace#create uses

        Validate.isTrue(isNamespaceValid(name), "'%s' contains invalid characters, only letters or numbers allowed.", name);

, but printf-style string replacement is not supported by Validate#isTrue

Test Cassandra KVS with Docker instead of in-memory Cassandra process

Fails to store empty blob

With a schema like this:

schema.addTableDefinition("blobs", new TableDefinition() { {
                rowName();
                rowComponent("id", ValueType.STRING);

                columns();
                column("blob", "b", ValueType.BLOB);
            }
        });

Storage and retrieval works fine in general, but if you store an empty byte array (i.e. length zero), then retrieval will fail to find anything. It will give an Optional.empty when you look up that row.

I tried to force Atlas to be aware of empty blobs as such:

schema.addTableDefinition("blobs", new TableDefinition() { {
                rowName();
                rowComponent("id", ValueType.STRING);

                columns();
                column("blob", "b", ValueType.BLOB);
                column("timestamp", "t", ValueType.VAR_LONG);
            }
        });

Then the row would be found (not Optional.absent), but the blob itself would be null.

Fix lock service test build flakes

seeing these on Travis (where I guess timing is less predictable?):

com.palantir.lock.AllLockTests > com.palantir.lock.impl.ClientAwareLockTest.testInterruptibleTryLock FAILED
    java.util.concurrent.TimeoutException at ClientAwareLockTest.java:320

Have a Strict KVS Specification

Currently C* KVS is not even testing with AbstractKeyValueService (and it can't be since it currently does not pass many of these tests). Instead we should probably distill the actual spec and have a specific test that tests only for that.

Don't export jmock

timestamp-impl exports jmock, but probably should not.

451 | +--- com.palantir.atlasdb:timestamp-impl:0.3.5
452 | | +--- com.palantir.atlasdb:timestamp-api:0.3.5 ()
453 | | +--- com.palantir.atlasdb:atlasdb-commons:0.3.5 ()
454 | | +--- junit:junit:4.11 -> 4.12 (*)
455 | | --- org.jmock:jmock:2.5.1
456 | | +--- org.hamcrest:hamcrest-core:1.1 -> 1.3
457 | | --- org.hamcrest:hamcrest-library:1.1
458 | | --- org.hamcrest:hamcrest-core:1.1 -> 1.3

Online vs. Offline CLI

I think I already know the answer to this, but wanted to put it out here for others to sanity check and provide thoughts on the last point.

I originally wanted to build out a more structured concept of offline vs. online CLIs but I ran into the following telling me it was impossible/a waste of time:

Off the top of my head I can't think of a use case for why an "online" CLI shouldn't be runnable "offline". So no reason for doing anything here.
The much more interesting use case is making sure a CLI is run when there are no time/lock services running against the atlas context/keyspace, but in the current world this is impossible to actually know.

My only other thought is somehow adding a concept of putting the keyspace in a locked state. I dont' know enough about lock server or generally how the transaction protocol works, but maybe at the start of an "offline" operation you spin up special lock/time server that sets a global lock, that timestampservice's have to check on spin-up and prevents clients from starting/commiting txns, and then you invalidate any in progress txns, do w/e work you need to, unset the lock, and exit successfully.

Native UUID support

Doing range scans with UUID row components is painful, because VAR_STRINGS shouldn't be range scanned. You either have to map UUID <=> (Long, Long) or UUID <=> (SHA256 hash)

Since UUIDs partition evenly, it makes sense to add a native type that supports range scanning.

Rework Cassasndra Configuration (Passwords/Defaults/Docs)

Passwords in CassandraKeyValueServiceConfig.java should probably be read from a hash? Also, maybe the default instance of the config should be somewhere else (it is mostly used in testing)? Default values should be documented for context.

AtlasDB-Cassandra fails oddly if replication factor > number of nodes

At the moment, AtlasDB fails poorly when the replication factor is greater than the number of nodes - it fails to connect 10 times and then gives up.

It should detect this edge case and fail with a more appropriate error message.

Remove PG leftovers from build system

There are a few PG leftovers here and there, for example in proto/
Should we remove those and make the eclipse setup more vanilla?

Findbugs warnings in generated schema code

Hi folks,
I get findbugs warnings EI_EXPOSE_REP and EC_BAD_ARRAY_COMPARE on generated XyzIdxTable code for indexes; any chance this could get fixed? Thanks!
Robert

Broken tests in atladb-shared-tests

Seeing test failures on 0.1-pre9:

:atlasdb-tests-shared:test
com.palantir.atlasdb.keyvalue.impl.InMemoryKeyValueServiceTest > testGetRowColumnSelection FAILED
java.lang.AssertionError: expected:<[]> but was:<[Cell [rowName=726f77, columnName=636f6c33], Cell [rowName=726f77, columnName=636f6c32], Cell [rowName=726f77, columnName=636f6c31]]>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:743)
    at org.junit.Assert.assertEquals(Assert.java:118)
    at org.junit.Assert.assertEquals(Assert.java:144)
    at com.palantir.atlasdb.keyvalue.impl.AbstractAtlasDbKeyValueServiceTest.testGetRowColumnSelection(AbstractAtlasDbKeyValueServiceTest.java:125)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:105)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:56)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
    at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
    at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
    at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

com.palantir.atlasdb.keyvalue.impl.partition.PartitionedKeyValueServiceTest > testGetRowColumnSelection FAILED
java.lang.AssertionError: expected:<[]> but was:<[Cell [rowName=726f77, columnName=636f6c33], Cell [rowName=726f77, columnName=636f6c32], Cell [rowName=726f77, columnName=636f6c31]]>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:743)
    at org.junit.Assert.assertEquals(Assert.java:118)
    at org.junit.Assert.assertEquals(Assert.java:144)
    at com.palantir.atlasdb.keyvalue.impl.AbstractAtlasDbKeyValueServiceTest.testGetRowColumnSelection(AbstractAtlasDbKeyValueServiceTest.java:125)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:105)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:56)
    at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:64)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
    at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
    at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:106)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
    at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.messaging.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:360)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
    at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Non-canonical sort of getRows response

Our Resource Store runs on AtlasDB and when performing certain transactions (specifically, getRows), we’d like to have the RowResults returned in the order that they were passed in as parameters. Unfortunately, the Transaction interface has a return value of SortedMap, which sorts the RowResults by their primary feature (in this case ResourceIdentifiers). On our end, this results in a server-side resorting of the returned results, which is not the end of the world per se but it’s a bit redundant. Would be great if there were a method that returned the list of results in the same order they were passed in.

Make sure Atlas Shell and Atlas Console are being built correctly / usable

Lee reports https://jcenter.bintray.com/com/palantir/atlasdb/atlasdb-shell/0.3.23/ isn't built correctly and gives JNI errors.

Tim isn't sure if we even ship the frontend portion of atlas console to OSS atlas.

Ticket made for tracking investigation / fixing.

The background sweeper is never started

Revisit compile.sh

proto/compile.sh is not good for regenerating protos (maybe this is not the intended use) for the following reasons:

it regenerates build.gradle (losing important config)
it does not handle protos that are not in their own project (e.g. PaxosPersistence.proto and ProfilePersistence.proto)
Perhaps a new script should be created that hardcodes what to compile where. Probably better is to have all autogenerated code generated by gradle (and not have to be checked in)?

Get rid of unused imports comments in favor of fixing checkstyle

look for comments with text "This exists to avoid unused import warnings"

Investigate mysterious protos

See #63

ExceptionProto
StackTraceElementProto

could not serialize HeldLocksToken

No json stuffs for HeldLocksToken, so blows up when passed via ForwardingLocksService

Pull in Netty based Lock, Timestamp, and Leader Election

@rjullman We should pull in the Netty based service that uses protobuf RPC for lock, timestamp, and leader election.

Stop using _ as an identifier

The build produces lots of warnings about how it might stop being a valid identifier in future versions of Java:

/Volumes/git/atlasdb/atlasdb-client/src/main/java/com/palantir/atlasdb/table/description/render/DynamicRowResultRenderer.java:65: warning: '_' used as an identifier
            _("for (Entry<byte[], byte[]> e : rowResult.getColumns().entrySet()) {"); {
            ^
  (use of '_' as an identifier might not be supported in releases after Java SE 8)

Atlasdb should fail-fast if type is "cassandra" but no has no leader config

At one of our deployments, atlasdb as configured to use cassandra but the "leader" section of the config wasn't filled out. This resulted in many transient non-deterministic problems like

com.palantir.timestamp.PersistentTimestampService: Throwable while
allocating timestamps.
! com.palantir.timestamp.MultipleRunningTimestampServiceError: Timestamp
limit changed underneath us (limit in memory: 123690317). This may
indicate that another timestamp service is running against this cassandra!

I'd be helpful if atlasdb could fail-fast in this case where config was forgotten.

Concretely the atlas yml:

atlas:
  keyValueService:
    keyspace: atlasdb
    port: "9160"
    replicationFactor: "3"
    servers:
    - cassandra-1.ops.palantir.local
    - cassandra-2.ops.palantir.local
    - cassandra-3.ops.palantir.local
    ssl: "false"
    type: cassandra

Globally false use of `equals` in ManyClientPoolingContainer

My IDE threw a warning on

private PoolingContainer<Client> delegateForHost(InetAddress host) {
    for (Map.Entry<InetSocketAddress, PoolingContainer<Client>> entry : containerMap.entrySet()) {
        if (entry.getKey().equals(host)) {
            return entry.getValue();
        }
    }
    log.warn("Unrecognized host {}, falling back to a randomly chosen client", host);
    return delegate();
}

and it looks like a legit bug. Specifically host is of type InetAddress and entry.getKey() is of type InetSocketAddress and so the error case is returned every time.

Use ToStringHelper

TransactionManagers needs to wrap its lock service with something like LockRefreshingLockService

LockRefreshingLockService is a client side wrapper that refreshes the client's locks, preventing long held locks from timing out as long as the client is alive.

There's an unfortunate type mismatch between LockService and RemoteLockService, so it can't be used as is.

InMemoryKeyValueServiceConfig is not registered as a Discoverable

Use a truststore from the config if ssl is set to true for cassandra

CassandraClientFactory should require a truststore in the config if ssl is enabled and use the trustore from the config to create an SSL socket factory preferably using: com.palantir.remoting.ssl.SslSocketFactories from http-remoting?

Currently it relies on JVM-wide SSL settings via -Djavax.net.ssl.trustStore

Online CLIs Can't Run Against Non-HA Atlas Clients

I’m trying to work through the concept of an “online atlas-cli”, as in I want to run a CLI against atlas where there is already a service (and thus a lock/timestamp service) running against that atlas keyspace. If that service is using an embedded lock/timestamp services, then it is impossible for the CLI to then piggy back on that lock/timestamp service because the endpoints never get registered.

So I see a couple of things here:

We require services in non-HA environments to use leadership election in order to enforce the creation of the endpoints. The CLI would just not be leader-eligible.
We simply register the endpoints, even when running in embedded mode. In which case we would make some sort of localServer uri config mandatory for AtlasDBConfig (like it is in the leader block) so it knows where to register them, and the CLI would just have to specify lock/timestamp blocks like it would if it were connecting to an external lock/timestamp

From what I understand @rjullman, the future sounded like we want to enforce external timestamp/lock services. Not sure what you think the better work around in the immediate term would be then, but both should be relatively straight forward to implement.