pnxtech/hydra

A light-weight library for building distributed applications such as microservices

Hydra is a NodeJS package which facilitates building distributed applications such as Microservices.

Hydra offers features such as service discovery, distributed messaging, message load balancing, logging, presence, and health monitoring. It was announced at EmpireNode 2016.

Install the latest stable version via npm install hydra --save

See our quick start guide and sample projects

If you're just getting started with Node Microservices and you have ExpressJS experience you should first look at our HydraExpress project.

If you want a lighter-weight Express integration or you're using Hapi, Koa, Sails.js, Restify or Restana then checkout the Hydra Integration Project.

Documentation

Visit our dedicated documentation site for hydra at: https://www.hydramicroservice.com

Hydra works great on AWS using Docker containers and Swarm mode, see: https://www.hydramicroservice.com/docs/docker/docker.html

Join us on Slack!

Are you using or planning on using Hydra on your project? Join us on Slack for more direct support. https://fwsp-hydra.slack.com To join, email [email protected] with your desired username and email address (for invite).

Related projects

There are many projects on NPM which contain the name hydra. The following are official projects related to the Hydra - microservice library.

• Hydra: hydra core project for use with Non-ExpressJS apps
• Hydra-Express: hydra for ExpressJS developers
• Hydra-Integration: Integrating third-party Node.js web frameworks with Hydra
• Hydra-Router: A service-aware socket and HTTP API router
• Hydra-cli: a hydra commandline client for interacting with Hydra-enabled applications
• Hydra Generator: A Yeoman generator for quickly building hydra-based projects
• Hydra-plugin-rpc: Create and consume remote procedure calls in hydra with ease
• Hydra-Cluster: A compute cluster based on Hydra
• UMF: Universal Message Format, a messaging specification for routable messages

Examples

• A sample hello-service project
• Hydra Hot Potato Service - an example of distributed messaging
• Hydra Message Relay - processing WebSocket calls via HydraRouter

Articles

• Tutorial: Building ExpressJS-based microservices using Hydra
• Building a Microservices Example Game with Distributed Messaging
• Deploying Node.js Microservices to AWS using Docker
• Using Docker Swarm for Deploying Node.js Microservices

Special thanks

A special thanks to Michael Stillwell for generously transferring his Hydra project name on NPM!

hydra's Issues

findService() returns incomplete object

Hello,
i am trying to use hydra in my project. I wanna get ip address and port of service.
hydra.findService('test-service') returns an object with only 3 properties:
{"serviceName":"test-service","type":"","registeredOn":"2017-07-21T11:25:14.530Z"}

How can I get ip and port?

Thx

[Question] Integrate with Awilix for DI

Hi @cjus,
I'm using Hydra for my project with Awilix. Everything work fine with Awilix and Awilix-express for REST API.
However, I'm facing a trouble to resolve objects when handle inter-service message. It's failed to resolve because there is nowhere to inject container when the hydra.on() get fired.

Do we have any ways to add inject container into (like app.use(middleware-to-inject-container)
const app = hydra.getExpressApp(); app.use(containerMiddleware);

  constructor (opts) {
    this.config = require('./config').HYDRA;
    this.umfHandler = umfHandler; // Resolve handler successfully
  }

  initialize () {
    return new Promise(function (resolve, reject) {
      hydra
        .init(config, function () {
          hydra.registerRoutes(routerConfig);
          hydra.getHydra().on('message', (message) => {
            this.umfHandler(message);  // Undefined
          });
        })
        .then(function () {
          resolve(hydra);
        })
        .catch(function (err) {
          reject(err);
        });
    });
  }
};

_sendMessage may have a possible silent failure

Looking at the _sendMessage code .. I think we have a corner case condition where we can silently fail to send a message ..

Use Case: Invoke sendMessage with an invalid (or dead) specific instance ID .

https://github.com/flywheelsports/hydra/blob/master/index.js#L1385

L1385          if (instance && instance !== '') {
            this._sendMessageThroughChannel(`${mcMessageKey}:${serviceName}:${instance}`, message);
          } else {

The code appears to blindly accept the instance given .. even if it is either totally bogus or not - running. It seems to be there should be a check before the _sendMTC to see if the instance is actually in the array returned from _getServicePresence() and if not, invoke reject()

Unable to connect to redis db

Hi 👋

I would like to connect to the redis db with something like that:

    "redis": {
      "url": "redis://:[email protected]:3002", // fake url
      "db":15
    }

If i use directly node redis client, it's ok:

var redis = require("redis"),
    client = redis.createClient({
      url:"redis://:[email protected]:3002"
    });

client.set("yo", "OK");

// This will return a JavaScript String
client.get("yo", function (err, reply) {
    console.log(reply.toString());
});

But with Hydra, I get some errors:

ERROR
{ event: 'error',
  message: 'Established Redis server connection has closed' }
ERROR
{ event: 'error', message: 'Reconnecting to redis server...' }
ERROR
{ event: 'error', message: 'Reconnecting to redis server...' }
ERROR
{ event: 'error', message: 'Reconnecting to redis server...' }
ERROR
{ event: 'error', message: 'Reconnecting to redis server...' }
ERROR
{ event: 'error',
  message: 'Max redis connection retry period exceeded.' }
ERROR
{ event: 'error',
  message: 'Error: Unable to establish a connection to Redis' }

so I don't understand how to set correctly my connection to redis

Can you help me please? 🙏 😄

Documentation links to the Slack team for flywheelsports which is password protected

The documentation has a link to Slack that requires a sign in to join. It is great to have a slack channel to talk about Hydra, but it would be even better if I could join the discussions ;)

Document hydra single instance use

Document the fact that the intended use of hydra is one per process instance.

Run in browser?

Hi.
I am exploring using microservices as a way to compose a web-app ie run it in a browser. I am transitioning from "old school" monolithic to microservices and am fairly new to Node etc.

The app has lots of business logic in the client with data storage a mixture of server via REST/JSON and local data via pouchdb.

The initial iteration was MVC which got messy pretty quickly. Next was MVVM which was much better but still coupled. Then I tried a global event bus as a way to decouple which was an improvement but still didn't feel right.

I want a system whereby I can use my Gherkin specs to run the app "headless" ie only run the models with no views at all to test the business logic. Then implement the model states as view representations with whatever renderer, persistence, logging, etc I go with.

So, in theory microservices fits the bill with its loose coupling, composition, etc.

Could I use Hydra in the browser? Can I simply Browserify it (or similar)? Is it even a good idea?

Any thoughts or directions would be helpful.

Thanks,
Murray

Improve redis reconnection logic

Hydra should attempt reconnection logic before and during runtime.

Invalid Hash Method

Line 1204 of index.js points to a method called Utils.stringHash(channel), however that method doesn't exist in utils.js. The nearest method is md5Hash(key). While trying to call the first method, messages don't get processed. Changing to the latter corrects the problem.

possible Integrating to nestjs?

hi @cjus
i'm very excited want to use hydra in my new project, this awesome we can manage service easily,
i have question it possible integrating to nestjs?
may be you have suggest for me?
thanks

Throw exception if already initialized.

Hydra should throw an exception if already initialized.

Convert redis.createClient calls to client.duplicate

In Hydra, channel messages are created using new Redis client - we should be able to duplicate (clone) the existing Redis client whereby consolidating reconnection handling.

Pattern matching

Hello,
I'm doing a deep research of all the microservices frameworks out there. I've checked Seneca, Hemera, Moleculer and now hydra.

Something I really like from Hemera and Seneca is pattern matching. This feature really allows incrementally adding features. As a bonus there is absolutely no need to know which service can handle the message beforehand, super cool.

Hydra has another thing that I also like a loot, which is direct service communications. There are sometimes when you need to transfer a big payload and know who should handle it, in that case direct http or tcp communication seems better. (if hydra is not doing this please clarify it, thanks)

My question is, are you aware of pattern matching and if so, do you have plans to implement it?

Regards

Function to retrieve routes table

There should be a function to retrieve service route table from redis.

Use redis scan instead of keys

Identified by @emadum as a potential performance enhancement.
https://stackoverflow.com/questions/32603964/scan-vs-keys-performance-in-redis

Various improvements possible in mocha test cases

1 -- There appear to be 3 duplicate tests in test.js
https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L182
https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L239

https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L198
https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L255

https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L219
https://github.com/sjmcdowall/hydra/blob/master/specs/test.js#L276

2 -- Inconsistent use of @description vs. @summary

3 -- No code coverage report appears to be generated. This could be a good thing to see?

A warning about storing application data in the same Redis db as Hydra metadata

I've just learned the hard way that it is NOT a good idea to mix your application data with Hydra's data in the default Redis database (db 0) if your services generate a lot of keys.

I've just spent an entire day trying to get to the bottom of very puzzling performance issues of a large system we launched a few days ago. As the number of keys in Redis kept climbing, the system got slower and slower until the point that it was unusable. After a lot of digging, it turned out that the SCAN MATCH command used by Hydra's checkPresence() function was the culprit. Apparently SCAN MATCH will iterate through the entire keyspace looking for a needle in a haystack. With a database of 100,000+ keys, that's 1000+ scan calls needed just to check service presence (with chunk size of 100). It easily took an R4.large Elasticache instance to its knees under moderate client load. Moving the app data out to a separate db immediately solved the issue.

Perhaps it wouldn't be a bad idea to add a prominent note to the docs somewhere about this situation, to save some other people a world of pain.

Heed my warnings!

Alternative to IP based services?

Has it been considered to also support service registry by providing a service host rather than only an IP?

I was interested in using hydra to register services deployed via http://zeit.co/now
Unfortunately they are not accessible via IP address, as the hostname is used for routing.

I considered just putting the hostname in the description property but it is pretty hacky. Any better suggestions that I'm just not seeing in the docs?

Improve error handling when service config isn't found in redis

Consider a better way of addressing this exception.

Error: Invalid service stored config
    at _getConfig.then (/usr/src/app/node_modules/hydra/index.js:201:30)
    at tryCatcher (/usr/src/app/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/src/app/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/usr/src/app/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/usr/src/app/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/usr/src/app/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/usr/src/app/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/usr/src/app/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues (/usr/src/app/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:672:20)
    at tryOnImmediate (timers.js:645:5)
    at processImmediate [as _immediateCallback] (timers.js:617:5)

Swagger integration was not seamless using hydra Express !!

Hello @cjus ,

I'm using the hydra-express wrapper, tried some ways and suggestion by the community but unable to integrate swagger hydra-express app, could please suggest me in this regards !!

Thanks for your time :)

Regards,
Sai.

Add a config option to not shorten UMF messages

This would be useful where message sizes do not need to be small.

How to send UFM message from non-hydra server to hydra micraservice?

HI all, First of all thanks for this awesome service.

I have a expressjs app (non hydra), and two microservice (hydra-express). i want to send a request to microservice from express app. i tried sample message from documentation. but it gives error. can you guys point me out how to implement message from non-hydra app to hydra microservice?

Don't allow service names with a colon character.

Service should fail to initialize if it encounters a service name with a colon character. The reason for this is because colon characters are used to delineate key segments in Hydra's use of Redis.

Running Hydra on AWS Elasticache

I am trying to run Hydra app with AWS Elasticache but unfortunately I got

ReplyError: ERR unknown command 'config'

error. It is caused by the fact that config command is restricted on AWS ElastiCache. Is there a way to start hydra without calling config command?

Unable to connect to redis

index.js

const hydra = require('fwsp-hydra'),
      configJSON = require('./config.json');

hydra.init(configJSON)

config.json

{
  "hydra": {
    "serviceName": "youtube-fetcher",
    "serviceIP": "127.0.0.1",
    "servicePort": 3001,
    "serviceType": "test",
    "serviceDescription": "",
    "redis": {
      "url": "...",
      "port": "...",
      "db": 0
    }
  }
}

url and port are hidden for purpose.

Terminal

Unable to establish a connection to Redis

redis url does not work

Hello,

With fwsp-hydra-express I can set the url of redis like this

    "redis": {
      "url": "redis://:[email protected]",
      "port": 3046,
      "db": 15
    }

and it's ok

but with the fwsp-hydra it's impossible to connect to the redis db

Opinion: _checkServicePresence should not shuffle resultant array

https://github.com/flywheelsports/hydra/blob/master/index.js#L985

I don't believe (obviously my opinion) that this routine should go ahead and randomize the results of the services. That's not really part of it's contract. Also -- this is a "lower" level routine that gets called "often" and any sort of "Shuffle" routine is O(N) at a minimum .. i.e. it takes at least SOME processing time and probably increases with size of array.

Also since this Shuffled array is cached, that means it's really not random upon multiple uses anyway. The code I see appears to use array[0] to get a "random" service, but if its cached its the same array each time?

Much better is to store just the pure array returned and cache that. Then in the FEW cases where we need a "random" element from it, just have a Util that does something like:

var randomInstance = instances[Math.floor(Math.random() * instances.length)]

Proxying socket.io connection

Hello everyone,
I am looking for some advice on how to best handle proxying of socket connections. I am using the API Gateway pattern to expose a single point of entry. REST requests are forwarded using hydra.makeAPIRequest() and they call appropriate services.

I am struggling a bit with how to handle Socket.io connections, not to expose our Notification Service directly. There are some packages (socket.io-proxy) that offer this functionality but they require direct addressing while hydra operates on dynamic ports and instances.

What are your thoughts on the issue? How to best handle this case?

if shutdown message is first message in health log then it never expires

Hi there,

I've been playing with a test setup of Hydra for a few days and doing a lot of starting/stopping of services in the process. I've noticed that Redis is slowly filling up with hydra:service:*:health:log keys from the beginning of time. These keys appear to be persistent.

E.g.

127.0.0.1:6379> keys *:health:log
1) "hydra:service:test-service:fbcef59e18874aa3993b6eff782285ff:health:log"

27.0.0.1:6379> llen hydra:service:test-service:fbcef59e18874aa3993b6eff782285ff:health:log
(integer) 1

27.0.0.1:6379> lindex hydra:service:test-service:fbcef59e18874aa3993b6eff782285ff:health:log 0
"{\"ts\":\"2017-10-24T08:29:27.609Z\",\"serviceName\":\"test-service\",\"type\":\"error\",\"processID\":6672,\"msg\":\"Service is shutting down.\"}"

127.0.0.1:6379> pttl hydra:service:test-service:fbcef59e18874aa3993b6eff782285ff:health:log
(integer) -1

Looking deeper into it, I notice that expire is called on the *:health:log keys as part of updatePresence(). This seems to be run on a timer. Nevertheless, my keys do not expire.

It's looking like I have a situation where the *:health:log key is created (i.e. the first log entry for the service is written) on service shutdown. In this case, the expire command never gets a chance to run on the key before the service shuts down, leaving the key in a persistent state.

I'm not sure if this situation is unique to my test setup, or if I've missed something somewhere. Does anybody else have this issue?

Would be great if you could provide typescript definitions for hydra, hydra-express ...

Would be great if you could provide typescript definitions for hydra, hydra-express ...

Improve makeAPIRequest to make other service requests if an error is encountered

The hydra makeAPIRequest call currently makes a remote request and, as expected, returns the status of that request. So if a call to a service is made and the service isn't available or returns an HTTP 5XX class error then that response is immediately returned - despite the fact that there maybe other service instances available. Ideally makeAPIRequest should attempt to contact other available services before turning an error.

This enhancement would allow hydra to help service requests in a more resilient manner.

Performance improvement: Pub/sub should pass messages by reference

Messaging systems work best with small message payloads. Redis in particular suffers huge performance losses when pub/sub message payloads get large.

Since UMF messages already have a unique message ID, we can cache the message payload in redis when a message is sent and only send the message ID over pub/sub, minimizing the message payload.

sendMessage() callback functionality

I have 1 router service. It is listening to RESTful requests. When it gets a request it takes the post data and transfers it to another service via sendMessage(). The other process is built to take the data, process it, and return the result. The problem is that sendMessage has no callback functionality and cannot be chained with a then() to start working on the result - by either sending it to another service or routing it back to the user.

Is the only way to achieve this is by defining the process.js service as a hydra-express process with a restfulAPI? makeAPIrequest does allow for a callback function handling the response.

Another cumbersome way would be to open a different route on the router.js which will listen to the response from the service. But then how do i relay this response back to the user?

Combine REST with Socket protocol in microservice architecture

hi, firstly, i must say this is super cool project, and i very love it. But i have some confuse, if you have time, please explain clearly for me.

i want to build a microservice, Client will call to one main service by HTTP, and then main service call to another service by message (websocket). But how i can skip the HTTP request and wait the message go to all other services, get the response and send back to to client :(
i want to use websocket to communication between internal microserivce because it's faster than HTTP protocol

many thank guys

How to view hydra key-value pairs in native redis cli ??

Firstly, thank you for awesome module.

I know using hydra-cli , I can get total list of nodes,routes etc . I'm curious how hydra really use redis to store this key-value pair.

Once everything up and running, I tried view hydra service related registries in redis server using redis-cli, but unable to see anything ?? Is there any best approach?

Regards,
Sai.

How to integrate with Netflix Eureka?

Hi everyone,

I'm trying to choose a microservices framework to use with node
but I really need to be able to integrate it with Eureka since we already have a
microservices infrastructure using Eureka :'(

Does anybody know how one could integrate hydra with Eureka for service register/discovery?

I don't necessarily want to make the hydra microservices to discover each other using Eureka.
I want the possibility to register to it and to eventually use services that are out of the hydra microservices network.

fwsp-hydra examples

Hello!

First things first, i think this is great! It's nice to have this package which bundles together the essentials for quick development of service-based applications!

However it would be great to see some examples of a real-world application.

Right now i'm kind of confused how hydra-express integrates with hydra. Some clarification with an example project would be lovely.

I'm also curious if there is a way to do RPC/request-response kind of communication with sendMessage/sendReplyMessage? What if i, on a GET /api/v1/login, want to communicate with another service before sending a response? How can i do this? I don't want the communication between the services to be available via a public API so i guess i can't use makeAPIRequest

I've just started using hydra and i'm eager to learn more about it in general so a simple example project would help a lot for everyone who wants to get involved i think.

How to add new services with out restarting API gateway /hydra-router?

This is not a bug but not sure where to ask? How do i add new microservices in hydra with out restarting API gateway or hydra-router?

When redis is down, the init function does not resolve/reject its promise.

When redis is down, the init function does not resolve/reject its promise.
index.js line 125 -> this.redisdb.select(HYDRA_REDIS_DB, (err, result) => {
never gets into callback somehow when redis is down

Add getServiceNodes() to documentation

https://www.hydramicroservice.com/docs/hydra-core/hydra-methods.html

Update Websockets Messaging Documentation

Hi there,

seems like the docs about Websockets Messaging is not updated, for example the use of the
method sendReplyMessage.

I have a question, the only way to know If a message is coming from one service of other is checking the from property of the message, right?

Nodejs crashes when redis connection dies - How to handle it ?

Hi,

I'm considering using hydra for a small project but I have a big problem. When I restart redis, all the related microservices crashes, without retrying the connection

I tried to tweak the retry_strategy but I'm a bit lost.

How can I catch theses exceptions or prevent them ?

log entry { ts: '2018-06-13T09:36:04.251Z',
serviceName: 'my-awesome-microservice',
type: 'error',
processID: 1953,
msg: 'Established Redis server connection has closed' }
Unhandled Exception { Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 6379 }
Unhandled Exception { Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)
errno: 'ECONNREFUSED',
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 6379 }
log entry { ts: '2018-06-13T09:36:04.458Z',
serviceName: 'my-awesome-microservice',
type: 'error',
processID: 1953,
msg: 'Reconnecting to Redis server...' }
Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1174:14)

Thank you in advance.

Consider supporting YAML for hydra configs

This might be useful.
https://www.npmjs.com/package/node-yaml

Allow more options for redis connection

redis.createClient supports a number of invocation formats:

1. redis.createClient([options])
2. redis.createClient(unix_socket[, options])
3. redis.createClient(redis_url[, options])
4. redis.createClient(port[, host][, options])

We're currently using 4, but we're not passing through options we don't use. I think it'd be better to use 1, and pass through options from the hydra.redis config entry.

One thing to point out is that we use url instead of host for the host option. url is also a valid redis createClient option (also used by createClient invocation option 3), with format: [redis:]//[[user][:password@]][host][:port][/db-number][?db=db-number[&password=bar[&option=value]]] This could be confusing and we should probably try to be more consistent with the redis options.

At the very least, we should support passing in a password for Redis servers with auth enabled.

How to connect to redis via unix socket?

Unfortunately I am hosting my projects on a virtual host and the only way I can access my redis instance is using unix sockets.

Is there any way to pass the path to the socket file in hydra?

SWIM membership protocol integration ??

Hi @cjus,

Instead of centralized services registry using, how about the integration SWIM membership protocol.

• Though we create Redis cluster services registries and load balance them, which involves again resource
  consumption and network hops.
• Pattern matching, with no or minute dependency with location of the other MS, when there are being
  called by another MS.
• How matured in logging ??
• Does Hydra was production ready ??

How far Hydra has these features ??If not is there plans for implementation?

Regards,
Sai.

hydra makeAPIRequest, queryparameters

Hello, i would to send an api request to one microservice using query params

hydra.makeAPIRequest(hydra.createUMFMessage({
to: 'users-service:[get]/api/users' + '?email=' + email + '&password=' + password,
from: 'authentication-service:/',
body: {}
}))
I got this error to response:

_sendMessage should use _reject()

https://github.com/flywheelsports/hydra/blob/master/index.js#L1364

According to the "specs" for the function, on error(s) the method should call a reject .. but in all instances it actually invokes resolve() with an error message. Besides breaking the contract -- it probably should throw a reject() on errors.

Suspicious lines:

https://github.com/flywheelsports/hydra/blob/master/index.js#L1382

https://github.com/flywheelsports/hydra/blob/master/index.js#L1396

_checkServicePresence throws a reject although it says it will never do so

https://github.com/flywheelsports/hydra/blob/master/index.js#L941

According to the "contract" this method should never throw a reject() -- yet it does here:

https://github.com/flywheelsports/hydra/blob/master/index.js#L973

Question(s) on using Message Queues

First -- loving the project. We are looking at this instead of a low-level Redis/Kue for implementing our microservices. However, I do have a question (or two) as it comes to our desires (and maybe best practices tips!) ..

We have a bunch of "work queues" for our services -- things that can take some time to process and the more often than not -- invoke another step in the "recipe" of what needs to be done. I see a bit of documentation on "Message Queues" but it's a bit vague per se and I don't really see any examples?

Also -- part of the "Health Check" our users want to see is the length of any particular "Message Queues" so we can see if they are growing and hence, maybe we need for service instances -- or if they idle a long time and thus maybe reduce the service instances. But I don't see where we can get the current message queue length?!

Also -- once a job is completed (either ok or error) and we mark is so .. how / when does it get removed from the "Queue" so it doesn't build up to "infinity"?

Also is there some examples on how to setup a good Redis cluster and how to use it? We most certainly want a HA environment and having a single Redis node appears to be a Single Point of Failure.. ?

Is there the idea of routing of messages based on "advanced" criteria other than just service name? Use case is this -- we have a "file processor" that .. no surprises ..processes files .. however, the physical location of the file may only be available some of the service instances (local to a single machine for example) etc. Thus the service is still "file-processor" but I would like a clean way to indicate there maybe constraints on to WHICH file-processor is suitable..

Since this is nodeJS (for us anyway) -- is there a way to set / limit the amount of concurrency a single instance can handle similar to Kue's "Processing Concurrency" argument to queue.process() ?

Are there best practices on how to "share" in an installation common needed configuration information on deployment (versus development). We will have an average of 6-10 compute nodes per install (at different customers) and hand configuring the Redis stuff maybe a PITA .. Just wondering how other people may have solved this ..

TIA and I'm excited about going forward ..