chrisdinn / brando Goto Github PK
View Code? Open in Web Editor NEWA Redis client written with Akka's IO package
License: Other
A Redis client written with Akka's IO package
License: Other
ReplyParser re-parses the entire reply bytestring every time it gets new bytes from Redis. If you get lots of small responses from Redis (which happens, especially with commands like SMEMBERS) it can be surprisingly slow.
As I write this, it looks like your repository is down.
Do you know what might generate this exception? I am finding this using play-redis, which has a dependency on brando, on Heroku.
a.a.OneForOneStrategy - queue empty java.util.NoSuchElementException: queue empty at scala.collection.mutable.Queue.dequeue(Queue.scala:66) ~[org.scala-lang.scala-library-2.11.7.jar:na] at brando.Connection$$anonfun$receive$1$$anonfun$applyOrElse$4.apply(Connection.scala:91) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at brando.Connection$$anonfun$receive$1$$anonfun$applyOrElse$4.apply(Connection.scala:78) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at brando.ReplyParser$class.parseReply(ReplyParser.scala:145) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at brando.Connection.parseReply(Connection.scala:27) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at brando.Connection$$anonfun$receive$1.applyOrElse(Connection.scala:78) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at akka.actor.Actor$class.aroundReceive(Actor.scala:467) ~[com.typesafe.akka.akka-actor_2.11-2.3.13.jar:na] at brando.Connection.aroundReceive(Connection.scala:27) ~[com.digital-achiever.brando_2.11-3.0.3.jar:3.0.3] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) [com.typesafe.akka.akka-actor_2.11-2.3.13.jar:na] at akka.actor.ActorCell.invoke(ActorCell.scala:487) [com.typesafe.akka.akka-actor_2.11-2.3.13.jar:na]
In order to keep things clean, the package hierarchy needs to be set up. Something along the lines of com.digital_achiever.brando.
accept something like (key:String, batch: Batch).
Would be useful to have a way to do multi/exec, watch etc with sharding when doing multiple operations on a shard.
Previous versions of brando implemented Akka Stash and queued requests when the redis was not connected. As part of #46 this behavior was silently dropped and version 3 throws an exception instead, which is significantly different behavior.
Thanks!
Batched requests can get sent out of order when a given request fails to write to the tcp socket. The connection will get a Tcp.CommandFailed message and then retry the message. But at this point it is possible other requests have already been sent successfully.
The doc link in the README is broken
When testing the recovering of disconnects by creating a ipfw rule on OSX, I find that Brando doesn't seems to recover.
To block/allow a port I'm using :
sudo ipfw add 1 deny tcp from any to any 6379
sudo ipfw del 1 deny tcp from any to any 6379
Seems like when the connection is setup, Brando is able to detect the disconnection when the rule is set. However, it cannot connect or reconnect.
It might be an akka IO issue I just posted something on the user list : https://groups.google.com/forum/#!topic/akka-user/sL_wL_OQgbo
It appears if brando has a shard connected to a dead redis instance that actor will die.
If we try to send a new shard to the shard manager for that host, then it will send a Connect message to the dead actor so it can never resolve in some failover scenarios.
I'll look at the code a bit more later - these are high level observations.
Course of action is to make the connections immutable and just poisonpill and recreate connections on Shard messages to the shard manager.
It's currently difficult to determine the state of connections in Brando.
Tcp.ConnectionClosed, Tcp.Connected etc should send events (possibly to the parent or to listeners passed in) so that services can respond accordingly.
I traced issues reported as Brando #64 and play-redis #44 and #29
We noticed the first occurrence of the issue on Heroku because it secures its redis instance. It caused excessive logging without any reasonable error message.
However, I successfully reproduced in on localhost when I secured my redis instance.
Enable redis authentication in redis.conf, e.g., uncomment
requirepass foobared
Example of Play framework controller:
package controllers
import javax.inject.Inject
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
import play.api.mvc._
import akka.actor.ActorSystem
import akka.pattern._
import brando.Redis
class Application @Inject( )( system: ActorSystem ) extends Controller {
implicit val timeout = akka.util.Timeout( 1.second )
val actor = system actorOf Redis( "localhost", 6379, database = 1, auth = Some( "foobared" ) )
def ping( ) = actor ? brando.Request( "PING" )
def index = Action.async {
ping( ).map { case message =>
Ok( views.html.index( "done", message ) )
}.recover { case ex =>
Ok( views.html.index( "error", ex.toString ) )
}
}
}
When used Redis
actor the response is the redis is disconnected. When the StashingRedis
is used, requests timeout. Both no further reason provided.
To trace the issue, I inserted several logging statements into Brando. I figured out two things:
auth
is provided. The reason is it stucks in Connecting
state but the authentication is executed on Connected
.2016-04-21T13:07:53.071807+00:00 app[web.1]: Queue(Actor[akka://application/temp/$v])
2016-04-21T13:07:54.081759+00:00 app[web.1]: connecting
2016-04-21T13:07:54.093458+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:54.093624+00:00 app[web.1]: Queue(Actor[akka://application/temp/$w])
2016-04-21T13:07:55.112304+00:00 app[web.1]: connecting
2016-04-21T13:07:55.119680+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:55.119837+00:00 app[web.1]: Queue(Actor[akka://application/temp/$x])
2016-04-21T13:07:56.135040+00:00 app[web.1]: connecting
2016-04-21T13:07:56.140825+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:56.141029+00:00 app[web.1]: Queue(Actor[akka://application/temp/$y])
2016-04-21T13:07:57.152416+00:00 app[web.1]: connecting
2016-04-21T13:07:57.161871+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:57.162046+00:00 app[web.1]: Queue(Actor[akka://application/temp/$z])
2016-04-21T13:07:58.184895+00:00 app[web.1]: connecting
2016-04-21T13:07:58.200840+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:58.210118+00:00 app[web.1]: Queue(Actor[akka://application/temp/$A])
2016-04-21T13:07:59.221666+00:00 app[web.1]: connecting
2016-04-21T13:07:59.226342+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:07:59.226689+00:00 app[web.1]: Queue(Actor[akka://application/temp/$B])
2016-04-21T13:08:00.243840+00:00 app[web.1]: connecting
2016-04-21T13:08:00.250774+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:08:00.251051+00:00 app[web.1]: Queue(Actor[akka://application/temp/$C])
2016-04-21T13:08:01.264740+00:00 app[web.1]: connecting
2016-04-21T13:08:01.273008+00:00 app[web.1]: Some(Failure(brando.RedisException: NOAUTH Authentication required))
2016-04-21T13:08:01.273249+00:00 app[web.1]: Queue(Actor[akka://application/temp/$D])
2016-04-21T13:08:02.293235+00:00 app[web.1]: connecting
2016-04-21T13:08:02.297606+00:00 app[web.1]: Some(Failure(brando.RedisException: ERR max number of clients reached))
2016-04-21T13:08:02.297748+00:00 app[web.1]: Queue(Actor[akka://application/temp/$E])
When max number of clients is reached, it crashes at this line.
Finally, it goes out of memory after while. However, I am not sure what exactly causes the memory leaking, just guessing.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.