Giter Site home page Giter Site logo

akka-persistence-mongo's Introduction

An Akka Persistence Plugin for Mongo

Build Status

A replicated Akka Persistence journal backed by MongoDB Casbah.

Prerequisites

Release

Technology Version
Plugin
Scala 2.10.5, 2.11.7 - Cross Compiled
Akka 2.3.12 or higher
Mongo 2.6.x or higher

Snapshot

Technology Version
Plugin
Scala 2.11.7
Akka 2.4.1 or higher
Mongo 3.1.x or higher

Installation

SBT

Release

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/releases"

libraryDependencies ++= Seq(
  "com.github.ironfish" %% "akka-persistence-mongo-casbah"  % "0.7.6" % "compile")

Snapshot

resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"

libraryDependencies ++= Seq(
  "com.github.ironfish" %% "akka-persistence-mongo"  % "1.0.0-SNAPSHOT" % "compile")

Maven

Release

// Scala 2.10.5
<dependency>
    <groupId>com.github.ironfish</groupId>
    <artifactId>akka-persistence-mongo-casbah_2.10</artifactId>
    <version>0.7.6</version>
</dependency>

// Scala 2.11.7
<dependency>
    <groupId>com.github.ironfish</groupId>
    <artifactId>akka-persistence-mongo-casbah_2.11</artifactId>
    <version>0.7.6</version>
</dependency>

Snapshot

// Scala 2.11.7
<dependency>
    <groupId>com.github.ironfish</groupId>
    <artifactId>akka-persistence-mongo_2.11</artifactId>
    <version>1.0.0-SNAPSHOT</version>
</dependency>

Build Locally

You can build and install the plugin to your local Ivy cache. This requires sbt 0.13.8 or above.

sbt publishLocal


It can then be included as dependency:

libraryDependencies += "com.github.ironfish" %% "akka-persistence-mongo" % "1.0.0-SNAPSHOT"

Mongo Specific Details

Both the Journal and Snapshot configurations use the following Mongo components for connection management and write guarantees.

Mongo Connection String URI

The Mongo Connection String URI Format is used for establishing a connection to Mongo. Please note that while some of the components of the connection string are [optional] from a Mongo perspective, they are [required] for the Journal and Snapshot to function properly. Below are the required and optional components.

Standard URI connection scheme

mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][.collection][?options]]

Required

Component Description
mongodb:// The prefix to identify that this is a string in the standard connection format.
host1 A server address to connect to. It is either a hostname, IP address, or UNIX domain socket.
/database The name of the database to use.
.collection The name of the collection to use.

Optional

Component Description
username:password@ If specified, the client will attempt to log in to the specific database using these credentials after connecting to the mongod instance.
:port1 The default value is :27017 if not specified.
hostN You can specify as many hosts as necessary. You would specify multiple hosts, for example, for connections to replica sets.
:portN The default value is :27017 if not specified.
?options Connection specific options. See Connection String Options for a full description of these options.

Mongo Write Concern

The Write Concern Specification describes the guarantee that MongoDB provides when reporting on the success of a write operation. The strength of the write concern determine the level of guarantee. When a PersistentActor's persist or persistAsync method completes successfully, a plugin must ensure the message or snapshot has persisted to the store. As a result, this plugin implementation enforces Mongo Journaling on all write concerns and requires all mongo instance(s) to enable journaling.

Options Description
woption The woption requests acknowledgment that the write operation has propagated to a specified number of mongod instances or to mongod instances with specified tags. Mongo's wOption can be either an Integer or String, and this plugin implementation supports both with woption.

If woption is an Integer, then the write concern requests acknowledgment that the write operation has propagated to the specified number of mongod instances. Note: The woption cannot be set to zero.

If woption is a String, then the value can be either "majority" or a tag set name. If the value is "majority" then the write concern requests acknowledgment that write operations have propagated to the majority of voting nodes. If the value is a tag set name, the write concern requests acknowledgment that the write operations have propagated to a replica set member with the specified tag. The default value is an Integer value of 1.
wtimeout This option specifies a time limit, in milliseconds, for the write concern. If you do not specify the wtimeout option, and the level of write concern is unachievable, the write operation will block indefinitely. Specifying a wtimeout value of 0 is equivalent to a write concern without the wTimeout option. The default value is 10000 (10 seconds).

Journal Configuration

Activation

To activate the journal feature of the plugin, add the following line to your Akka application.conf. This will run the journal with its default settings.

akka.persistence.journal.plugin = "casbah-journal"

Connection

The default mongo-url is a string with a value of:

casbah-journal.mongo-url = "mongodb://localhost:27017/store.messages"


This value can be changed in the application.conf with the following key:

casbah-journal.mongo-url

# Example
casbah-journal.mongo-url = "mongodb://localhost:27017/employee.events"


See the Mongo Connection String URI section of this document for more information.

Write Concern

The default woption is an Integer with a value of:

casbah-journal.woption = 1


This value can be changed in the application.conf with the following key:

casbah-journal.woption

# Example
casbah-journal.woption = "majority"

Write Concern Timeout

The default wtimeout is an Long in milliseconds with a value of:

casbah-journal.wtimeout = 10000


This value can be changed in the application.conf with the following key:

casbah-journal.wtimeout

# Example
casbah-journal.wtimeout = 5000


See the Mongo Write Concern section of this document for more information.

Reject Non-Serializable Objects

This plugin supports the rejection of non-serializable journal messages. If reject-non-serializable-objects is set to true and a message is received who's payload cannot be serialized then it is rejected.

If set to false (the default value) then the non-serializable payload of the message becomes Array.empty[Byte] and is persisted.

the default reject-non-serializable-objects is a Boolean with the value of:

casbah-journal.reject-non-serializable-objects = false


This value can be changed in the application.conf with the following key:

casbah-journal.reject-non-serializable-objects

# Example
casbah-journal.reject-non-serializable-objects = true

Snapshot Configuration

Activation

To activate the snapshot feature of the plugin, add the following line to your Akka application.conf. This will run the snapshot-store with its default settings.

akka.persistence.snapshot-store.plugin = "casbah-snapshot"

Connection

The default mongo-url is a string with a value of:

casbah-snapshot.mongo-url = "mongodb://localhost:27017/store.snapshots"


This value can be changed in the application.conf with the following key:

casbah-snapshot.mongo-url

# Example
casbah-snapshot.mongo-url = "mongodb://localhost:27017/employee.snapshots"


See the Mongo Connection String URI section of this document for more information.

Write Concern

The default woption is an Integer with a value of:

casbah-snapshot.woption = 1


This value can be changed in the application.conf with the following key:

casbah-snapshot.woption

# Example
casbah-snapshot.woption = "majority"

Write Concern Timeout

The default wtimeout is an Long in milliseconds with a value of:

casbah-snapshot.wtimeout = 10000


This value can be changed in the application.conf with the following key:

casbah-snapshot.wtimeout

# Example
casbah-snapshot.wtimeout = 5000


See the Mongo Write Concern section of this document for more information.

Snapshot Load Attempts

The snapshot feature of the plugin allows for the selection of the youngest of {n} snapshots that match an upper bound specified by configuration. This helps where a snapshot may not have persisted correctly because of a JVM crash. As a result an attempt to load the snapshot may fail but an older may succeed.

The default load-attempts is an Integer with a value of:

casbah-snapshot.load-attempts = 3


This value can be changed in the application.conf with the following key:

casbah-snapshot.load-attempts

# Example
casbah-snapshot.load-attempts = 5

Status

Performance

Minimal performance testing is included against a native instance. In general the journal will persist around 8,000 to 10,000 messages per second.

Sample Applications

The sample applications are now located in their own repository.

Change Log

1.0.0-SNAPSHOT

  • Upgrade Akka 2.4.1.
  • Upgrade Casbah to Async driver 3.1.
  • Supports latest Plugin TCK.

0.7.6

  • Upgrade sbt to 0.13.8.
  • Upgrade Scala cross-compilation to 2.10.5 & 2.11.7.
  • Upgrade Akka to 2.3.12.

0.7.5

  • Upgrade sbt to 0.13.7.
  • Upgrade Scala cross-compilation to 2.10.4 & 2.11.4.
  • Upgrade Akka to 2.3.7.
  • Examples moved to their own repository.
  • Removed logback.xml in akka-persistence-mongo-casbah as it was not needed.
  • Added pomOnly() resolution to casbah dependency, fixes #63.

0.7.4

  • First release version to Maven Central Releases.
  • Upgrade Sbt to 0.13.5.
  • Upgrade Scala cross-compilation to 2.10.4 & 2.11.2.
  • Upgrade Akka to 2.3.5.
  • Added exception if /database or .collection are not accessible upon boot. Thanks @Fristi.
  • Modified snapshot feature for custom serialization support. Thanks @remcobeckers.

0.7.3-SNAPSHOT

  • Upgrade Sbt to 0.13.4.
  • Upgrade Scala cross-compilation to 2.10.4 & 2.11.2.
  • Upgrade Akka to 2.3.4.
  • @deprecated write confirmations, CasbahJournal.writeConfirmations, in favor of AtLeastOnceDelivery.
  • @deprecated delete messages, CasbahJournal.deleteMessages, per akka-persistence documentation.

Author / Maintainer

Contributors

akka-persistence-mongo's People

Contributors

aiacovella avatar buster84 avatar fristi avatar haghard avatar ironfish avatar rb3ckers avatar sean-walsh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

akka-persistence-mongo's Issues

Snapshots are always serialized with Java serializer

When trying out the Kryo serialization I noticed that my snapshots were never serialized with Kryo but always with the standard Java serializer. The cause is, I think, that the SelectedSnapshot is serialized which is not handled by the Akka peristence SnapshotSerializer that is registered.

See pull request #75 for a fix I tested with my project.

0.7.4-SNAPSHOT not available in sonatype repository

The README says you can fetch akka-persistence-mongo via the sonatype snapshot repo, and uses 0.7.4-SNAPSHOT as the example version to fetch from there.

I'm pretty sure this worked for me once in the past, as I'm using 0.7.4 on one of my machines -- but I've just tired building elsewhere, and I'm now unable to fetch 0.7.4 from there -- 0.7.3 seems to be the most recent version.

See https://oss.sonatype.org/content/repositories/snapshots/com/github/ddevore/akka-persistence-mongo-casbah_2.11/

processorId to persistenceId

Hello,

in the last versions processorId was renamed to persistenceId in the bson document.
Therefore all the events we already persisted before upgrading to the new version could not be read anymore. We could work around it, but it would be nice if there would be a warning in the documentation about it.
For the future - is it possible to name this key independent from how it is called in akka persistence - smthg. like pid?

thanks,

michael

Setting 'casbah-snapshot-store.mongo-snapshot-url' throws None.get exception

When setting casbah-snapshot-store.mongo-snapshot-url to something custom;

casbah-snapshot-store.mongo-snapshot-url = "mongodb://localhost:27017/wowcrafters-informant"
akka.persistence.snapshot-store.plugin = "casbah-snapshot-store"

Throws an exception:

[ERROR] [08/17/2014 11:15:20.679] [informant-akka.actor.default-dispatcher-2] [akka://informant/system/snapshot-store] None.get
akka.actor.ActorInitializationException: exception during creation
    at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
    at akka.actor.ActorCell.create(ActorCell.scala:596)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
    at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
    at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
    at akka.dispatch.Mailbox.run(Mailbox.scala:219)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.NoSuchElementException: None.get
    at scala.None$.get(Option.scala:313)
    at scala.None$.get(Option.scala:311)
    at akka.persistence.mongo.snapshot.CasbahSnapshotHelper$class.$init$(CasbahSnapshotHelper.scala:30)
    at akka.persistence.mongo.snapshot.CasbahSnapshotStore.<init>(CasbahSnapshotStore.scala:18)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at java.lang.Class.newInstance(Class.java:374)
    at akka.util.Reflect$.instantiate(Reflect.scala:45)
    at akka.actor.NoArgsReflectConstructor.produce(Props.scala:361)
    at akka.actor.Props.newActor(Props.scala:252)
    at akka.actor.ActorCell.newActor(ActorCell.scala:552)
    at akka.actor.ActorCell.create(ActorCell.scala:578)
    ... 7 more

Scala version: 2.10.4

"com.github.ddevore" %% "akka-persistence-mongo-casbah" % "0.3-SNAPSHOT" => NOT WORKING

[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: com.github.ddevore#akka-persistence-mongo-common_2.10;0.3-SNAPSHOT: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: com.github.ddevore#akka-persistence-mongo-common_2.10;0.3-SNAPSHOT: not found
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
at sbt.IvySbt$$anon$3.call(Ivy.scala:60)

Testfix

Extract unit test into JouranlSpec.

Overwrite snapshots

I am not really interested in keeping all snapshots;

Alt text

Would be nice to have a setting which allows overwriting snapshots?

0.7.4 snapshot unavailable from sonatype

It was resolving just fine yesterday but is gone today:

[warn]  module not found: com.github.ddevore#akka-persistence-mongo-casbah_2.10;0.7.4-SNAPSHOT
[warn] ==== local: tried
[warn]   /Users/jean/.ivy2/local/com.github.ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[warn] ==== Typesafe Releases Repository: tried
[warn]   http://repo.typesafe.com/typesafe/releases/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[warn] ==== sbt-plugin-snapshots: tried
[warn]   http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/com.github.ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/ivys/ivy.xml
[warn] ==== sbt-plugin-releases: tried
[warn]   http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/com.github.ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/ivys/ivy.xml
[warn] ==== Sonatype Snapshots: tried
[warn]   https://oss.sonatype.org/content/repositories/snapshots/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[warn] ==== playvertx Repository: tried
[warn]   http://fmasion.github.com/releases/com.github.ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/ivys/ivy.xml
[warn] ==== Rhinofly Internal Repository: tried
[warn]   http://maven-repository.rhinofly.net:8081/artifactory/libs-release-local/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[warn] ==== eu.byjean.repo: tried
[warn]   http://dl.bintray.com/jeantil/maven/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[warn] ==== FMasion bintray repository: tried
[warn]   http://dl.bintray.com/fmasion/maven/com/github/ddevore/akka-persistence-mongo-casbah_2.10/0.7.4-SNAPSHOT/akka-persistence-mongo-casbah_2.10-0.7.4-SNAPSHOT.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.github.ddevore#akka-persistence-mongo-casbah_2.10;0.7.4-SNAPSHOT: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::

Also I was wondering if there are plans to make a release of 0.7.4

thanks

Not working with Akka 2.3.0-RC2's sharding functionality

Here is how I used the sharding:

  ClusterSharding(system).start(
    typeName = "CoreProcessor",
    entryProps = if (cluster.selfRoles.contains("normal")) Some(Props[CoreProcessor]) else None,
    idExtractor = { case msg => ("1", msg) },
    shardResolver = { case _ => "1" })

When I stop the actor system that runs the event sourced processor, another actor system (that I believe tries to become a backup) raised this error:

[ERROR] [02/14/2014 01:49:44.104] [kio-akka.persistence.dispatchers.default-plugin-dispatcher-18] [akka://kio/user/sharding/CoreProcessorCoordinator/singleton] swallowing exception during message send
java.io.NotSerializableException: com.mongodb.ServerAddress
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1165)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1535)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1413)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1159)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1535)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1413)

why only 30/s persistent performance

Hello,
I'm testing the akka-sample-persistence-java in typesafe, but get only 30 persistent messages per second and I think I must get something wrong.

here is my build.sbt file,
name := """akka-sample-persistence-java"""
version := "2.3.5"
scalaVersion := "2.10.4"
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/releases"
resolvers += "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
libraryDependencies ++= Seq(
"com.github.ddevore" %% "akka-persistence-mongo-casbah" % "0.7.4" % "compile",
"com.typesafe.akka" %% "akka-persistence-experimental" % "2.3.5",
"com.github.ironfish" %% "akka-persistence-mongo-casbah" % "0.7.5-SNAPSHOT" % "compile"
)

And my application.conf is,
akka.persistence.journal.plugin = "casbah-journal"
casbah-journal.mongo-journal-url = "mongodb://localhost:27017/store.messages"
akka.persistence.snapshot-store.plugin = "casbah-snapshot-store"
casbah-snapshot-store.mongo-snapshot-url = "mongodb://localhost:27017/store.snapshots"

my mongodb is 2.4.12, and I just use command "mongod" to start the default mongo. And my test code is,
public static void main(String... args) throws Exception {
final ActorSystem system = ActorSystem.create("example");
final ActorRef persistentActor =
system.actorOf(Props.create(ExamplePersistentActor.class), "persistentActor-4-java");

for(int i = 0;i < 100000; i++){
    persistentActor.tell(new Cmd("a"), null);   
}        
    Thread.sleep(60000);
    system.shutdown();
}

when i run with cassadra with the same code, i can persist about 3000 messages per second, but only 30 with mongodb.

defaults in reference.conf

akka.loglevel = "ERROR" should not be defined in reference.conf, since it might change the defaults of akka. That is something that only applications should change.

Similar with akka.persistence.journal.plugin. The plugin should not be activated just by adding it as a dependcency. This should be specifed in application.

Why akka.persistence.publish-confirmations=on?

Is mongo-journal-write-concern = "acknowledged" a good default? Isn't it too unsafe?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.