Giter Site home page Giter Site logo

aws-swf's Introduction

A Node.js library for accessing Amazon Simple Workflow (SWF)

NPM version Build Status Coverage Status Code Climate

aws-swf provides high-level classes to build Amazon Simple Workflows using Node.js.

It is built on top of the official Node.js aws-sdk for low-level API calls. You can find the full API reference here.

Requirements

Installation

npm install aws-swf

Setting AWS Credentials

Cf. http://docs.aws.amazon.com/AWSJavaScriptSDK/guide/node-configuring.html#Setting_AWS_Credentials

Usage

Step 1 : Register Domains, ActivityTypes, WorkflowTypes

The AWS SDK is sufficient to register SWF objects, since those are just direct API calls. (You can also register them through the AWS console.)

You can run the following example to register the objects used in the following examples :

Example to register "test-domain", "simple-activity" and "simple-workflow"

Step 2 : Create Activity Workers

An ActivityPoller will wait for new tasks from SWF, and emit an activityTask event. The event receives an instance of ActivityTask, which makes it easier to send the response to SWF.

This example starts an Activity Worker which completes immediatly.

Step 3 : Create Workflow Deciders

The Decider class will wait for new decision tasks from SWF, and emit a decisionTask event. The event receives an instance of DecisionTask, composed of :

  • an EventList instance, to query the state of the workflow
  • a DecisionResponse instance, to prepare the decider response with decisions

Simple decider worker example : decision worker, which schedules an activity task, then stop the workflow.

Step 4 : Start a workflow

To start a workflow, call the start method on a Workflow instance. This call will return a WorkflowExecution instance, which you can use to signal or terminate a workflow.

AWS Options

Sometimes you may want to configure the AWS SDK instance. A possible reason is to set a specific region for aws-swf. Because Node.js allows multiple instances of the same package for maximal compatibility among libraries, you would need to do something similar to:

var AWS = require('./node_modules/aws-swf/node_modules/aws-sdk/lib/aws');

Instead, you simlpy need to do:

var AWS = require('aws-swf').AWS;

An example use case would be:

var AWS = require('aws-swf').AWS;

AWS.config = new AWS.Config({
  region: process.env.AWS_REGION || 'us-west-2',
  apiVersions: {
    swf: '2012-01-25'
  }
});

API Documentation

The API documentation is available online at http://neyric.github.io/aws-swf/apidoc/

To rebuild the documentation, install jsdoc, then :

jsdoc lib/*.js README.md -d apidoc

Test

Tests can be executed with Mocha :

$ mocha

To get the coverage, run :

$ ./node_modules/.bin/istanbul cover ./node_modules/mocha/bin/_mocha

Then open coverage/lcov-report/index.html

To send the coverage to coveralls, I run locally (I don't know why travis-ci after-script doesn't work...):

$ NODE_ENV=test ./node_modules/.bin/istanbul cover ./node_modules/mocha/bin/_mocha --report lcovonly -- -R spec && cat ./coverage/lcov.info | COVERALLS_REPO_TOKEN=xxxx ./node_modules/coveralls/bin/coveralls.js && rm -rf ./coverage

License

MIT License

See also

aws-swf's People

Contributors

ericabouaf avatar jakubknejzlik avatar jodem avatar ktonon avatar mdlavin avatar melnicki avatar michaelkovalchuk avatar sukrit007 avatar trym avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

aws-swf's Issues

Loop executing child workflows

Thanks a lot for the library and default activities - works great.

One question, which I was not able to do yet... Is it possible to define a loop based on array returned as a result from the previous step and start a child process for each element of the array as input data for the child workflow.

Thanks a lot,
Vladimir

event_list.has_schedule_activity_task_failed is not detecting ACTIVITY_TYPE_DOES_NOT_EXIST

decisionTask.eventList.failed(activityId) returns false when this activityId is in fact failed.

I tried to trace the code through the debugger, and the way it is implemented right now, activityIdFor in event-list.js on line 19 receives scheduledEventId === undefined

screen shot 2014-03-06 at 3 48 32 pm
screen shot 2014-03-06 at 3 50 56 pm

I was able to fix this issue by doing this:

has_schedule_activity_task_failed: function (activityId) {
   return this._has_eventType_for_activityId(activityId, 'ScheduleActivityTaskFailed');
},

multiple providers for activities

The idea would be to send a 'provider' in the task input.

The Activity Worker could then use different config files depending on the provider value.

(instead of single config.js file at the moment...)

Raw event history should be exposed on EventList instance

I notice that EventList instances have a private _events property that contains the raw event history provided by SWF. It would be very useful to me if this was exposed, perhaps as a get_history method that would return a clone of the array. Would you be open to a pull request for this?

Consistent method names (camel/snake case)

Most of the APIs in the library consistently follow JavaScript naming conventions and uses camel case (parameter and method names).

However, several of the classes have a mixture of snake-case and camel-case method names. For example, DecisionsResponse has an add_marker() method and an addDecision() method. Similarly, EventList() has a has_timer_event() method and an eventById() method.

It would be nice if the public API of the library consistently used camel case for method names. For backwards compatibility, the existing snake-case function names should be kept as (deprecated) aliases.

Control Information for timer

Refer
https://github.com/neyric/aws-swf/blob/master/lib/decision-response.js#L262

The start_timer method uses the "name" parameter as control information. Instead, we should call it "control". Reason: We can add a JSON message to this parameter, which will enable us to implement different strategies like:
ExponentialBackoff
LinearBackoff
.....

In my current use case, I will be implementing my own exponential strategy for retries, however need to add the state for the same as control information.
http://docs.aws.amazon.com/amazonswf/latest/apireference/API_TimerStartedEventAttributes.html

So that decider can evaluate next delay and fire the timer again.

New polling request created on every new wf execution

we are experiencing a behaviour in our decider (and activity) worker where a new pollForDecisionTask connection is created (identified from aws client logging) every time a new workflow instance is created.

When the worker is first deployed we see log like the following every 60 seconds (default long polling period)

[AWS swf 200 60.289s 0 retries] pollForDecisionTask({ domain: 'dev',
  taskList: { name: '...' },
  identity: '...',
  maximumPageSize: 100,
  reverseOrder: false })

after we start a new workflow execution, and the decider go through its logic and activities then we start seeing another log for pollForDecisionTask.

Is that the expected behaviour ? or is there some kind of clean up we need to do when the execution is suppose to be terminated ?

we have been fighting this issue for a while now, as the time goes the response time for task.response.respondCompleted become very very slow (5 mins +).

AWS error: Specified token has exceeded its maximum lifetime

I've run into issue that instance of my SWF decider got response from AWS saying:
ValidationException: Specified token has exceeded its maximum lifetime

It happened after around 4 days of running.

Maybe it's not related, but before this error the decider receives another one for the few times (2x):
UnknownResourceFault: Unknown decision task

Does anyone run into this issue?

Poller stops polling after long idle

I noticed that after long idle of a few hours, both activity worker and decider stop processing requests. The requests are submitted to SWF, but they aren't processed. I was able to reproduce this repeatedly. No errors occur on the client, just nothing is being processed.

Support for lambda task

Updating the aws-sdk dependency to the latest will allow:

decisionTask.response.addDecision({
"decisionType":"ScheduleLambdaFunction",
"scheduleLambdaFunctionDecisionAttributes": {
"id": "someid",
"name": "MyLambdaFunction",
"input": "input"
}
});

This allows calling lambda functions as activities... A real nice feature.

Right now fails because the swf model in the aws-sdk does not include the newer lambda task items (ScheduleLambdaFunction,scheduleLambdaFunctionDecisionAttributes).

Clearly schedule_lambda would be a nice addition to the decision-response module as well.

Code does not allow you to set activity type and version.

The code which lets you set the activity type and version looks more like a hack.
https://github.com/neyric/aws-swf/blob/master/lib/decision-response.js#L169

The code should be like, if sometthing is not there then it should set default value. Else it should take what value is supplied. Currently it sets value only if the first parameter does not contain the key "activity".

if(!swfAttributes) {
if(scheduleAttributes.activity) {
ta.activityType = scheduleAttributes.activity;
}
if (typeof ta.activityType === "string") {
ta.activityType = { name: ta.activityType, version: "1.0" };
}
}

APIs for cancelling WorkflowExecution

I'd like access to APIs for cancelling a WorkflowExecution (i.e. requestCancelWorkflowExecution) and to test the EventList whether a cancel has been requested (i.e. a WorkflowExecutionCancelRequested event has been recorded).

Specifically, I'd hope to see the following:

  • WorkflowExecution.requestCancel() method
  • EventList.requested_cancel_workflow() method (though not sure about the name)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.