paypal/squbs

Akka Streams & Akka HTTP for Large-Scale Production Deployments

squbs's Issues

Implement Java API and samples

The Java API may be wrappers for the current Scala API.

Use java CompletableFuture for Java API

May be optionally, at least.

Clean up historic branches before 0.7.0 publish

Our branch structure should be clean as part of 0.7.0.

Validation Directives for squbs

Add Validation support for spray layer.

Lost system state update messages on extension init failures

In case of extension init failures, the system state event message sent to interested parties (admin console) is not sent. Consoles cannot update their state correctly.

Publish code coverage numbers publicly

Configure CI to publish code coverage numbers on README page, and accessible publicly.

Fix verifyURI in HttpClientManager to use Spray Uri path construct

This is more efficient than the current String manipulations. We should also pass Uri and not String as parameter.

Ensure JavaDoc and ScalaDoc is in good shape for all public APIs

Need a thorough review of the APIs and make sure all is documented. Actor-based interaction would need different kind of documentation, which should be pursued differently.

Move squbs request dispatching from Spray to akka-http

Akka-http 1.1 promises performance on par with Spray and will replace Spray going forward. squbs needs to adopt akka-http as the replacement for spray.

This issue is created to deal with akka-http from the server dispatch side.

Investigate the use of compiler macros for Orchestration DSL

The orchestration DSL is currently achieved by using the sbt boilerplate plugin. It could be made far more powerful and flexible by implementing the same using compiler macros. We should investigate and implement if feasible.

Merge TimeoutPolicy into the Patterns project.

It is a bit too small to be its own module. We better keep it together with Patterns.

Move squbs-httpclient to akka-http

As part of akka-http support, we need to move the squbs-httpclient from sitting on top of Spray to use akka-http instead.

HttpClient performance enhancement

Simplify httpclient with less actor creation.

HttpClient support for chunked response

So we can implement features like consuming an SSE stream.

Run elaborate performance comparison of Akka Http & Spray

This is to ensure akka-http is absolutely ready for prime time. This issue is a release blocker for 0.8.

Review all documentation and ensure they are all in good usable shape.

All documentation should be in good quality.

Update java-compat and logger artifacts to latest version

Also add logger artifact dependencies to tests as log messages will get lost without those.

Remove hardcoded header logging in spray client

https://github.com/paypal/squbs/blob/master/squbs-httpclient/src/main/scala/org/squbs/httpclient/HttpClientManager.scala#L77

If there are other places, those should be masked as well.

Samples should work

All squbs samples should be functional. We need to make sure all samples builds and runs. Few obvious issues with resolves:

• https://github.com/paypal/squbs/blob/master/samples/bottles/build.sbt#L19
• https://github.com/paypal/squbs/blob/master/samples/pubsub/build.sbt#L17

HttpClientManager.makeFullUri broken when path contains query and/or fragment

The '?' becomes encoded as part of path, not really a query string.

Support registration of stream artifacts in squbs modules/cubes

squbs has been supporting registration of actors and services so far. With the arrival of akka-streams in squbs, we need to support registration of stream artifacts. We need to prototype/discuss the proper manifestation of these artifacts before going about implementing it.

Unicomplex ReportStatus not usable from outside unicomplex.

The response to ReportStatus (a tuple) exposes member types that are private to unicomplex.

Pipeline Enhancement

Refactor some classes from unicomplex to pipeline, most of them are from org.squbs.proxy, now moved to org.squbs.pipeline in squbs-pipeline project.

Simplify pipeline management, remove some classes like PipelineRegistry.

Configure CI to use ScalaStyle and/or ScapeGoat

We should publish code quality metrics from ScalaStyle and/or ScapeGoat.

Add Travis CI Gitter Integration

Test RouteDefinition in squbs

As today, if we want to test RouteDefinition using spray-testkit, we need to add some workaround to create the route instance.

e.g.

val actor = TestActorRef[DummyActor]

val route = RouteDefinition
.startRoutes(new SellercommunitysvcSvc)(actor.underlyingActor.context).route
class DummyActor extends Actor {
override def receive: Receive = {
case whatever =>
}
}

Then apply the route assertion like:

REQUEST ~> route ~> check {
ASSERTIONS
}

Create/publish g8 and activator templates for squbs.

This will make getting started much easier.

Upgrade to sbt 0.13.9

Waiting the following issues to be resolved:

• scalastyle/scalastyle#156
• daniel-trinh/scalariform#73

Implement admin console cube

This should expose system stats as well as JMX MBeans. Console could consist of well documented REST APIs so users can build and/or integrate with their own UI.

Fix code coverage and file headers for org.squbs.pattern.spray.japi

Two issues with this package:

1. Many of the files do not have the proper header. This is very important.
2. The code coverage for this package is absolutely 0. Please build the test cases.

Frequent timeouts when building with coverage on Travis-CI

1. Centralized the timeout controls
2. Ensure the timeouts are large enough

Bump to latest Scala, Akka, and Spray versions

Also standardize on spray-routing-shapeless2.

Note: All projects based on squbs 0.7 MUST replace spray-routing with spray-routing-shapeless2

Formal CI setup on travis-ci

We should publish to Sonatype and maven central from Travis CI.

Clean up HTTPClient API not to pass in ask timeout, even implicitly

Instead, we should calculate the ask timeout to be marginally larger than the request timeout.

Fix null issues reported by Codacy

Especially in pipeline, unicomplex/proxy, httpclient.

Please ensure code coverage and correct headers for org.squbs.httpclient.japi

Many files don't have the headers. Also, code coverage is low. Can you please beef this up?

Give a readable actor name to ServiceActor when a proxy exists

Currently the actor name would be assigned to the ProxyActor if a processor exists, considering if we can give an name to the service actor in case of actor selection

https://github.com/paypal/squbs/blob/master/squbs-unicomplex/src/main/scala/org/squbs/unicomplex/Unicomplex.scala#L621

        proxyName.fold(pipelineManager.default) {
          case "" => None
          case other => pipelineManager.getProcessor(other)
        } match {
          case None =>
            val hostActor = context.actorOf(props, name) // disable proxy
            (hostActor, SimpleActor(hostActor))
          case Some(proc) =>
            val hostActor = context.actorOf(props)
            (hostActor, ProxiedActor(context.actorOf(Props(classOf[CubeProxyActor], proc, hostActor), name)))
        }

A problem we need to consider is, what the actor names should be for the ServiceActor and ProxiedActor

Build is sporadically hanging on unit tests

Need to investigate hanging build.

Fix build stability issues - builds must not sporadically fail.

UnicomplexSpec sporadically fails on Travis-CI

This is in particular to the StashCubeSvc calls in the test. Sometimes there is nothing stashed. Looks like a timeout issue.

Improve/Simplify Orchestration DSL

The orchestration DSL currently requires users to write the initial expectOnce block, which is boilerplate. We should be able to use a typed actor for the same purpose and eliminate this boilerplate, thus passing the received initial message and the original requester to the orchestrate function directly.

Ensure squbs startup is compatible with sbt-revolver plugin

This does not work as sbt-revolver does not pass the class path in the system properties. Need to scan the real class path instead of the properties.

Review Java API wrappers for performance neutrality

Wrappers should be at virtually no cost. We just need to review all of them.

Fix warnings issued by Codacy.

Cleanup SNAPSHOT cube versions

mock actorSelection during the actor test

How to mock actorSelection logic in the actor if we want to test the actor?

Sample code making use of ActorRegistry

An activator template for this sample code would be great!

Fix scope of HttpClient configuration to be ActorSystem scope

Add Travis CI Badge to README.md

scala-style is failing the build with parsing error

Build fails with the following:

message=Expected token RBRACKET but got Token(XML_START_OPEN,<,10847,<)

java.lang.RuntimeException: exists error
08:07:05    at scala.sys.package$.error(package.scala:27)
08:07:05    at scala.Predef$.error(Predef.scala:142)
08:07:05    at org.scalastyle.sbt.Tasks$.onHasErrors$1(Plugin.scala:99)

Please see:

• scalastyle/scalastyle#156
• daniel-trinh/scalariform#73