robksawyer/brenda-web

Web interface for submitting Blender render jobs to Amazon Web Services.

                                                        ,---,    ,---,    ,---,  
                                                     ,`--.' | ,`--.' | ,`--.' |  
               ,---,    ,---,.     ,---,.  .--.--.   |   :  : |   :  : |   :  :  
       ,---.,`--.' |  ,'  .'  \  ,'  .' | /  /    '. '   '  ; '   '  ; '   '  ;  
      /__./||   :  :,---.' .' |,---.'   ||  :  /`. / |   |  | |   |  | |   |  |  
 ,---.;  ; |:   |  '|   |  |: ||   |   .';  |  |--`  '   :  ; '   :  ; '   :  ;  
/___/ \  | ||   :  |:   :  :  /:   :  |-,|  :  ;_    |   |  ' |   |  ' |   |  '  
\   ;  \ ' |'   '  ;:   |    ; :   |  ;/| \  \    `. '   :  | '   :  | '   :  |  
 \   \  \: ||   |  ||   :     \|   :   .'  `----.   \;   |  ; ;   |  ; ;   |  ;  
  ;   \  ' .'   :  ;|   |   . ||   |  |-,  __ \  \  |`---'. | `---'. | `---'. |  
   \   \   '|   |  ''   :  '; |'   :  ;/| /  /`--'  / `--..`;  `--..`;  `--..`;  
    \   `  ;'   :  ||   |  | ; |   |    \'--'.     / .--,_    .--,_    .--,_     
     :   \ |;   |.' |   :   /  |   :   .'  `--'---'  |    |`. |    |`. |    |`.  
      '---" '---'   |   | ,'   |   | ,'              `-- -`, ;`-- -`, ;`-- -`, ; 
                    `----'     `----'                  '---`"   '---`"   '---`"  

brenda-web's Issues

Implement Support/Help Forum

I think it'd be nice to have a help section or forum that contains helpful FAQs and tips. I'm thinking that this looks similar to SailsJS docs. However, the SailsJS doesn't handle deeplinking very well, so this would need to be fixed.

Angular Sails Bind

Look into this https://github.com/diegopamio/angular-sails-bind.

Add Other Passport Strategies

This is for the future.

Available authentication strategies

Google+

1. Create a new app here
   • Name sails-social-auth-example
   • URL http://localhost:1337
   • Callback URL http://localhost:1337/auth/google/callback
2. In the config/express.js replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with the generated keys

GitHub

1. Create a new app here
   • Name sails-social-auth-example
   • URL http://localhost:1337
   • Callback URL http://localhost:1337/auth/github/callback
2. In the config/express.js replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with the generated keys

Facebook

1. Create a new app here
   • Name sails-social-auth-example
   • URL http://localhost:1337
   • Callback URL http://localhost:1337/auth/facebook/callback
2. In the config/express.js replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with the generated keys

Twitter

1. Create a new app here
   • Name sails-social-auth-example
   • URL http://localhost:1337
   • Callback URL http://localhost:1337/auth/twitter/callback
2. In the config/express.js replace YOUR_CLIENT_ID and YOUR_CLIENT_SECRET with the generated keys

sockets.io is throwing (global.File && obj instanceof File) error

/usr/local/lib/node_modules/sails/node_modules/sails-hook-sockets/node_modules/socket.io/node_modules/has-binary-data/index.js:30
         (global.File && obj instanceof File)
                                        ^
TypeError: Expecting a function in instanceof check, but got #<bound>
  at _hasBinary (/usr/local/lib/node_modules/sails/node_modules/sails-hook-sockets/node_modules/socket.io/node_modules/has-binary-data/index.js:30:41)
  at hasBinary (/usr/local/lib/node_modules/sails/node_modules/sails-hook-sockets/node_modules/socket.io/node_modules/has-binary-data/index.js:56:10)
  at Object.socketIOClientCallback (/usr/local/lib/node_modules/sails/node_modules/sails-hook-sockets/node_modules/socket.io/lib/socket.js:349:16)
  at MockServerResponse._clientCallback (/usr/local/lib/node_modules/sails/node_modules/sails-hook-sockets/lib/receive-incoming-sails-io-msg.js:217:17)
  at MockClientResponse.<anonymous> (/usr/local/lib/node_modules/sails/lib/router/res.js:72:18)
  at MockClientResponse.emit (events.js:129:20)
  at finishMaybe (_stream_writable.js:484:14)
  at endWritable (_stream_writable.js:493:3)
  at MockClientResponse.Writable.end (_stream_writable.js:459:5)
  at MockServerResponse.onend (_stream_readable.js:505:10)
  at MockServerResponse.g (events.js:199:16)
  at MockServerResponse.emit (events.js:129:20)
  at _stream_readable.js:908:16
  at process._tickDomainCallback (node.js:381:11)

Figure out how to get a callback from the Spot Instance Request status

I need to update the Render.status property once the spot instance request is fulfilled. Not quite sure how to handle this at the moment. Any ideas?

Instance Store on EC2 Instance?

# use EC2 instance store on render farm instance?
 use_istore = int(conf.get('USE_ISTORE', '1' if istore_dev else '0'))

in run.py

Total $pent

It'd be nice to show the total amount of money accrued for the month. I'm assuming there's a hook in the Amazon SDK that would allow me to pull this.

Passport and MongoDB Issue?

Since I've hooked up Mongodb something isn't acting right. I keep getting username already exists. And I also removed the localDisk.db from tmp as well. Not sure what's going on here. Any ideas?

Read SailsJS Scaling Doc if pushed to Heroku

http://sailsjs.org/#!/documentation/concepts/Deployment/Scaling.html?q=configuring-your-app-for-a-clustered-deployment

Add an input for jumping frames on the Start Spot Instance Form

To take advantage of the -j (jump) flag in the brenda-work script, I should add an input field that consumes this value.

Historical Price Data in Interface

This is a stub for the future task of showing historical price data in the interface. I figured the following might be helpful. This comes from the article: http://www.stratex.com/blog/2014/12/launching-spot-instances-aws-command-line/

Once you have the instance up and running there are few handy commands to manage things. To check the recent prices:

aws ec2 describe-spot-price-history –start-time $(date -j -u +”%Y%m%dT%H0000″) –product “Linux/UNIX (Amazon VPC)” –instance-type “r3.large” –availability-zone $zone

This will give you the spot prices as of the current hour.

To check on your request status, which you will need to do a lot initially till you figure out all the command switches as you will run into a lot of “failed due to bad parameters” cases:

aws ec2 describe-spot-instance-requests

Finally to cancel your request

aws ec2 cancel-spot-instance-requests –spot-instance-request-ids $requestId

AWS Credentials Check

On launch of the web app, ensure the user has a ~/.aws/crendials (need to look up PC version) file. If they don't allow them to add their credentials via the settings.

jobs/add_spot: Show visual feedback when file added.

There needs to be some kind of visual feedback that shows you that a file was actually attached after selecting one on the jobs/add_spot page.

AWS Renderfarm Market Analysis/Study

Some sites related to a quick market analysis (google search) on AWS and rendering.

http://www.bmkmp.com/?p=1495
http://www.thinkboxsoftware.com/deadline/
http://area.autodesk.com/blogs/cory/setting-up-an-amazon-ec2-render-farm-with-backburner
https://vimeo.com/103880013

Active Instance Notification in Nav

In order to not get charged out the wazoo, I think a notification in the nav is appropriate. Maybe something that shows the count of instances running or just a symbol that lets you know that there are some running.

Build Backend Functionality for 'Add Spot Instance Render Job:Add a Job'

The following needs to happen in the backend for the Add Job function.

1. Take submitted .blend file or .zip, check to see which was submitted
   • .blend: Zip the file up
   • .zip, .gz: Just pass the file along
2. Create a Brenda configuration file that brenda-work will use.
3. Run the brenda-work command and produce a Amazon SQS Queue that contains all of the frames in project. e.g. brenda-work -T ~/Brenda/frame -s 1 -e 50 push

   ~/Brenda/frame should be the local copy of frame located in lib/task-scripts/frame.
   Support subframe at a later date.

4. After the frames have been processed and the queue has been established. Run the brenda-run -N 4 -p 0.07 spot command and create the spot instance requests.

   These currently take Amazon around ~5 minutes to approve. And it's worth noting that it could fail. This may be due to price or other issues.

5. Show the status of the job.
6. Send an email notification when the job has been approved and has started.
7. Figure out how to stop the instances after the jobs have been completed.

I'm sure there's a lot more that needs to happen here, but this is all I could think of at the moment.

Note on implementation

• It'd be nice to show the process live as it's happening via AJAX. I can visualize some kind of progress as things are processing.

Pagination on the /jobs

Need to hook up pagination on the jobs index page.

http://localhost:1337/jobs

Get the EC2 Startup Script Working

This script handles sending the SQS queue messages to the EC2 instances. This is located in api/services/amazon.js -> startupScript.

It looks something like this:

startupScript: function(){
        //def startup_script(opts, conf, istore_dev):
        var login_dir = "/root";

        var head = "#!/bin/bash\n";
        var script = "";

        // use EC2 instance store on render farm instance?
        //var use_istore = int(conf.get('USE_ISTORE', '1' if istore_dev else '0'))

        /*if use_istore{
                # script to start brenda-node running
                # on the EC2 instance store
                iswd = conf.get('WORK_DIR', '/mnt/brenda')
                if iswd != login_dir:
                    head += """\
        # run Brenda on the EC2 instance store volume
        B="%s"
        if ! [ -d "$B" ]; then
          for f in brenda.pid log task_count task_last DONE ; do
            ln -s "$B/$f" "%s/$f"
          done
        fi
        export BRENDA_WORK_DIR="."
        mkdir -p "$B"
        cd "$B"
        """ % (iswd, login_dir)
                else:
                    head += 'cd "%s"\n' % (login_dir,)
            else:
                head += 'cd "%s"\n' % (login_dir,)

            head += "/usr/local/bin/brenda-node --daemon <<EOF\n"
            tail = "EOF\n"
            keys = [
                'AWS_ACCESS_KEY',
                'AWS_SECRET_KEY',
                'BLENDER_PROJECT',
                'WORK_QUEUE',
                'RENDER_OUTPUT'
                ]
            optional_keys = [
                "S3_REGION",
                "SQS_REGION",
                "CURL_MAX_THREADS",
                "CURL_N_RETRIES",
                "CURL_DEBUG",
                "VISIBILITY_TIMEOUT",
                "VISIBILITY_TIMEOUT_REASSERT",
                "N_RETRIES",
                "ERROR_PAUSE",
                "RESET_PERIOD",
                "BLENDER_PROJECT_ALWAYS_REFETCH",
                "WORK_DIR",
                "SHUTDOWN",
                "DONE"
                ] + list(aws.additional_ebs_iterator(conf))

            script = head
            for k in keys:
                v = conf.get(k)
                if not v:
                    raise ValueError("config key %r must be defined" % (k,))
                script += "%s=%s\n" % (k, v)
            for k in optional_keys:
                if k == "WORK_DIR" and use_istore:
                    continue
                v = conf.get(k)
                if v:
                    script += "%s=%s\n" % (k, v)
            script += tail*/
        return script
    },

Build Travis-like Log Window Containing Server Status

This task involves developing a server status window. I'd like this to look similar to the log window on Travis CI.

I was curious as to how the Travis CI log window was built and put the question on Stackoverflow. Shortly after, I received an awesome answer that led me to deciding on using CodeMirror.

CodeMirror Themes
https://codemirror.net/demo/theme.html

Code Folding
https://codemirror.net/demo/folding.html

Block Device Mapping?

What's this actually used for?

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/block-device-mapping-concepts.html

def script(opts, conf):
    itype = brenda_instance_type(opts, conf)
    snapshots = aws.get_snapshots(conf)
    bdm, snap_description, istore_dev = aws.blk_dev_map(opts, conf, itype, snapshots)
    script = startup_script(opts, conf, istore_dev)
    print script

in run.py of the brenda script

AWS Spot Instance Price Strategies

I stumbled across these interesting looking articles and didn't want to read them at the time, so I'm putting them here.

http://santtu.iki.fi/2014/03/19/ec2-spot-usage/
http://santtu.iki.fi/2014/03/12/ec2-spot-instances/

Error in services/BrendaWork -> start

debug: Running `brenda-work` with arguments:
debug: -c "/Users/robsawyer/Brenda/brenda-web/jobs/testing_job_drquretesting_job_drqure.conf" -T "/Users/robsawyer/Brenda/brenda-web/lib/brenda/task-scripts/frame" -s 1 -e 10 push
error: Error: no work, run with -h for usage

  at [object Object].PythonShell.parseError (/Users/robsawyer/Brenda/brenda-web/node_modules/python-shell/index.js:173:17)
  at ChildProcess.<anonymous> (/Users/robsawyer/Brenda/brenda-web/node_modules/python-shell/index.js:80:28)
  at ChildProcess.emit (events.js:110:17)
  at Process.ChildProcess._handle.onexit (child_process.js:1067:12)
 { [Error: no work, run with -h for usage
]
  executable: '/usr/local/bin/python',
  options: [ '-u' ],
  script: 'lib/brenda/brenda-work',
  args: [ '-c "/Users/robsawyer/Brenda/brenda-web/jobs/testing_job_drquretesting_job_drqure.conf" -T "/Users/robsawyer/Brenda/brenda-web/lib/brenda/task-scripts/frame" -s 1 -e 10 push' ],
  exitCode: 2 }
debug: results: undefined
error: Sending 500 ("Server Error") response:
 Error: no work, run with -h for usage

  at [object Object].PythonShell.parseError (/Users/robsawyer/Brenda/brenda-web/node_modules/python-shell/index.js:173:17)
  at ChildProcess.<anonymous> (/Users/robsawyer/Brenda/brenda-web/node_modules/python-shell/index.js:80:28)
  at ChildProcess.emit (events.js:110:17)
  at Process.ChildProcess._handle.onexit (child_process.js:1067:12)
 { [Error: no work, run with -h for usage
]
  executable: '/usr/local/bin/python',
  options: [ '-u' ],
  script: 'lib/brenda/brenda-work',
  args: [ '-c "/Users/robsawyer/Brenda/brenda-web/jobs/testing_job_drquretesting_job_drqure.conf" -T "/Users/robsawyer/Brenda/brenda-web/lib/brenda/task-scripts/frame" -s 1 -e 10 push' ],
  exitCode: 2 }

According to the lib/brenda-work script. The error

#print "OPTS", (opts, args)
    if not args:
        print >>sys.stderr, "no work, run with -h for usage"
        sys.exit(2)

appears when the script is run without any arguments.

So it seems that my current method of passing arguments isn't working. Maybe I need to RTFM for python-shell.

var arguments = '-c "' + configFilePath + '" -T "' + taskFilePath + '" -s ' + jobs[0].animation_start_frame + ' -e ' + jobs[0].animation_end_frame + ' push';
sails.log('Running `brenda-work` with arguments: ');
sails.log(arguments);

var options = {
    mode: 'binary',
    pythonPath: '/usr/local/bin/python', /* If installed with 'brew install python' */
    pythonOptions: ['-u'],
    scriptPath: 'lib/brenda/',
    args: [arguments]
};
sails.python.run('brenda-work', options,
    function (err, results) {
        if (err) {
            sails.log.error(err);
            reject(err);
        }
        // results is an array consisting of messages collected during execution
        sails.log('results: %j', results);
        fulfill(results);
    }
);

Update the interface on the start job page.

This page currently looks like crap. Need to update it and make it prettier. It might not even be needed. You could just start the job from the list view of jobs/index. This might be the best option now that I think about it.

After creating a spot instance job you're automatically pushed to this page. You can also access it from the jobs/index page via the green play button.

http://localhost:1337/jobs/submit/[job_id]

Write tests

Be sure to read the migration guide example using sails.request()
https://github.com/balderdashy/sails/blob/master/0.11-migration-guide.md

Add brenda permission to generated SQS queue?

Should I add the brenda permission that is created when installing brenda? Or will this matter in the end. I've stubbed this out in the /services/amazon/createSQSWorkQueue method.