rfdonnelly/jobrnr

Jobrnr runs jobs.

jobrnr's Issues

Indent for multi-line description (PlusOptionParser)

Multi-line descriptions are not indented properly. Only first line is indented.

Example:

Jobrnr::PlusOptionParser.parse({description: "a\nb", options: {}}, '+help')

DESCRIPTION

  a
b

Add a DSL for the plus option parser

options = parse_plus_options(plus_options) do
  name "command"
  synopsis "command [options]"
  description <<-EOF
    Multi-line
    description
  EOF

  option :option1 do
    description "This is the first option."
    default "default-value"
  end

  option :option_two do
    description "This is option two."
    default 5
  end

  extra <<-EOF
    Multi-line
    extra
  EOF
end

Typo in lib/jobrnr/dsl/job_builder.rb

Error

lib/jobrnr/dsl/job_builder.rb:26:in `repeat': uninitialized constant Jobrnr::TypeError (NameError)
Did you mean?  TypeError

Fix

diff --git a/lib/jobrnr/dsl/job_builder.rb b/lib/jobrnr/dsl/job_builder.rb
index 66f392c..acd306c 100644
--- a/lib/jobrnr/dsl/job_builder.rb
+++ b/lib/jobrnr/dsl/job_builder.rb
@@ -23,7 +23,7 @@ module Jobrnr

       def repeat(times)
         if !times.is_a?(Integer) || times < 0
-          raise Jobrnr::TypeError, "'repeat' expects a positive Integer" \
+          raise Jobrnr::ArgumentError, "'repeat' expects a positive Integer" \
             " but was given value of '#{times}' of type" \
             " '#{times.class.name}' @ #{source}"
         end

Catch User Errors

In general, catch bad usage and provide good error messages. Some work has already been done to validate DSL methods' inputs. This issue is an attempt to identify all areas where we can check, catch, and report bad usage.

• job command
  • predecessor not found (0b95522)
  • requires block (cc55954)
  • requires execute command (a2346fb)
• job.execute command
  • requires String or block (101e8eb)
  • requires String or block, not both (101e8eb)
  • requires String if not null (101e8eb)
• job.repeat command
  • requires Integer (c1a54d4)
  • requires positive Integer (c1a54d4)
• import command
  • require string prefix (6a0a017)
  • require non-empty prefix (6a0a017)
  • require file exists (6a0a017)
• check job graph has no cycles (Check implemented indirectly as job/'predecessor not found' check. User is unable to create cycles since jobs cannot be referenced until defined.)
• job description syntax errors (3c02885)
• jobrnr command
  • require valid file argument (#34)
  • require valid options (#36)

Add contracts

For method type checking.

https://github.com/egonSchiele/contracts.ruby

Auto add +help to options (PlusOptionParser)

When outputting option help, +help should be auto added to the help text.

Investigate making runs reproducible

• Add --seed option
• Per job PRNG?
• Use of uncontrolled Ruby random methods (e.g. Array::shuffle)

Formal job description help

Add features to allow job descriptions to produce man page like output.
Jobrnr can already produce the OPTIONS section.

• Add pre-option text
• Add post-option test
• Add support for +help plus option. (c42761b)

Example Output

SYNOPSIS

  jobrnr index.jr 

DESCRIPTION

  My job description.

OPTIONS

  +bool-option[=<value>]
    This is a Boolean option. Default: false

  +iter=<value>
    Number of iterations Default: 2

  +test=<value>
    Name of test to run. Default: 'default'

EXAMPLES

  Run all basic regression.

    jobrnr file.jr

  Run more iterations.

    jobrnr file.jr +iter=50

job::repeat 0 causes infinite loop

Given

#!/usr/bin/env jobrnr

job :test do
  execute 'echo'
  repeat 0
end

Expected

Terminates w/o running any jobs.

Actual

Loops endlessly with:

Running:0 Queued:0 Completed:0 Passed:0 Failed:0
Running:0 Queued:0 Completed:0 Passed:0 Failed:0
Running:0 Queued:0 Completed:0 Passed:0 Failed:0
Running:0 Queued:0 Completed:0 Passed:0 Failed:0
Running:0 Queued:0 Completed:0 Passed:0 Failed:0

Remove use of deprecated Fixnum

jobrnr/lib/jobrnr/plus_option_parser.rb:243: warning: constant ::Fixnum is deprecated           

Obtuse error message on file not found.

Error Message

jobrnr/lib/jobrnr/script.rb:24:in `read': No such file or directory @ rb_sysopen - non-existent-file.jr (Errno::ENOENT)

Reproduce

jobrnr non-existent-file.jr 

Repeat jobs hold up other jobs

Repeat jobs head of line block the job queue until all iterations have been scheduled. See

jobrnr/lib/jobrnr/job/dispatch.rb

Line 117 in c968b4a

job_queue.shift if job_instance.job.state.scheduled?

Instead, repeat jobs should be sent to the back of the queue after scheduling repeat job iteration. This will give other jobs a change to schedule.

Exit with non-zero status when job failures present

JobRnr always exits with zero status even when failures present.

Change 'dependent' and variations to 'prerequisite'/'predecessor'

Rename job description (*.jr) 'options' to 'jobrnr'

Rename to better differentiate between job description options vs jobrnr application options.

Before:

options.output_directory = "..."

After:

jobrnr.output_directory = "..."

Add GNU style boolean options

+boolean-option => options[:boolean_option] = true
+no-boolean-option => options[:boolean_option] = false

Need formal method of passing options to job descriptions (*.jr)

Requirements

• Pass options to job descriptions via the command line
• Pass options to job descriptions via the import statement
• Pass options to imports via the command line?

Alternatives

1. Pass inline with jobrnr options

Example:

jobrnr --opt0 dot-jr.jr --opt1 --dot-jr-opt0 --dot-jr-opt1

Call API defined method in dot-jr.jr to add options to OptionParser. How does this work with import?

Pro: dot-jr options show in --help.
Con: Not obvious from looking at command line what is a jobrnr option and a dot-jr option.

2. Pass options using a jobrnr option.

Example using --opts option:

jobrnr --opt0 dot-jr.jr --opt1 --opts '--dot-jr-opt0 --dot-jr-opt1'

Con: Additional typing overhead.
Con: Need for quotes.

3. Pass using different prefix

Example using + as prefix:

jobrnr --opt0 dot-jr.jr --opt1 ++dot-jr-opt0 ++dot-jr-opt1

Document seed substitution

The seed substitution isn't documented anywhere. Add doc to --help-format.

`--help` option repeated in jobrnr.1.ronn man page

Add -f, --max-failures option

JobRnr currently terminates when all jobs have completed regardless of success. Add option to terminate after seeing some number of failures.

Search and replace 'job_rnr' with 'jobrnr'

`import` all jobs

Add ability to import all jobs from a job description file.

import 'prefix', :all, 'file.jr'

Add Plugin and Script Specific Exceptions

So we can give more information to the user and differentiate between a jobrnr error and other errors.

Allow job descriptions (*.jr) to override Jobrnr options

Determine options in the following manner

1. Load default value
2. Get value from environment variable
3. Get value from command line options before dot-jr
   
   jobrnr <options-before-dot-jr> dot-jr.jr
4. Get value from dot-jr
5. Get value from command line options after job description
   
   jobrnr dot-jr.jr <options-after-dot-jr>

TODO

• Load default value
• Get value from environment variable
• Get value from command line options before dot-jr
• Get value from dot-jr
• Get value from command line options after dot-jr
• Document behavior

Example

--max-jobs default is 1

dot-jr.jr:

options.max_jobs = 4

command line:

JOBRNR_MAX_JOBS=2 jobrnr --max-jobs 3 dot-jr.jr --max-jobs 5

Initial value for the --max-jobs option is 1 (default). It gets overridden by environment variable to 2. The overridden by --max-jobs option on command line to 3. Then overriden by dot-jr.jr to 4. Finally overridden to 5 by another --max-jobs option on the command line.

Add per job timeout

Would be useful to to have a per job timeout feature to fail the job if it takes longer than expected.

Rename 'plugin' to 'extension'

'plugin' sounds old

Add import as prerequisite to jobrnr.5.ronn

Allow Relative Paths in Job Descriptions

Job description files should support relative paths to simplify specification of paths.

snippet from current examples/import/index.js

import 'child', [:compile, :run_random], File.join(File.dirname(__FILE__), "child.jr")

options.directory = File.join(File.dirname(__FILE__), "output")

snippet from desired examples/import/index.js

import 'child', [:compile, :run_random], "child.jr"

options.directory = "output"

Add multiple queues w/ per queue job limits

Add queues to help with resource (e.g. licenses) management. There would be both a per queue job limit and the pre-existing global job limit --max-jobs.

Example queue configuration:

• global limit is a function of CPUs
• compile queue - does not require a license, no limit
• short - requires a license, but only for short period, limit is a function of licenses
• long - requires a license for a long period of time, limit to a single job

Add iterations to GraphViz dot output

Before

After

Patch

diff --git a/lib/jobrnr/graph.rb b/lib/jobrnr/graph.rb
index 7bf9f83..afb9da8 100644
--- a/lib/jobrnr/graph.rb
+++ b/lib/jobrnr/graph.rb
@@ -42,6 +42,8 @@ module Jobrnr
             lines << "#{j.id} -> #{s.id}"
           end
         end
+
+        lines << '%s -> %s [ label = "%d" ]' % [j.id, j.id, j.iterations] if j.iterations > 1
       end

       [

Plugin API Additions

• Add currently running instances to post_interval event.
• Add pre_parse event
• Add post_parse event
• Add ability to add options to OptionParser instance

Add `--dry-run` option

To print the commands that will be run without actually running them.

Same stats print multiple times

Same stats print multiple times. Stats should print only when they change.

Reproduce

Apply patch:

diff --git a/examples/repeat/index.jr b/examples/repeat/index.jr
index aedbaf1..f8a8395 100644
--- a/examples/repeat/index.jr
+++ b/examples/repeat/index.jr
@@ -1,5 +1,5 @@
 job :repeat do
-  execute 'sleep #{rand(5)}'
+  execute 'sleep #{4 + rand(5)}'
   repeat 50
 end

Run:

jr examples/repeat/index.jr

Output

jobrnr: DEBUG: digraph DependencyGraph {
  repeat;
}
Running: 'sleep 6' regr00 iter0
Running: 'sleep 7' regr01 iter1
Running: 'sleep 7' regr02 iter2
Running: 'sleep 6' regr03 iter3
Running: 'sleep 6' regr04 iter4
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
Running:5 Queued:45 Completed:0 Passed:0 Failed:0
...

Show reason for termination

When the max failures limit is reached, it is unclear why jobrnr terminates early. Add a notification for termination reason.

Expose more information to plugins

Plugins need more information. For example, the jobrnr output directory.

Running jobs terminated prematurely when `--max-failures` reached

When --max-failures is reached, JobRnr exists immediately. Instead, it should stop submitting new jobs and let and running jobs finish.

Add version

Make seed hex format more flexible

There is no way to specify a hexadecimal seed without the '0x' prefix. Remove '0x' prefix from format and require job descriptions make the prefix explicit.

Add option to customize log recycling

The current behavior is to recycle job logs on job success (passing job logs get overwritten).

Add --save-all option to save all job logs. This option should qualify the last term (!job_instance.success?) of https://github.com/rfdonnelly/jobrnr/blob/master/lib/jobrnr/job/dispatch.rb#L78.

Using job description file basename as classname is insufficient

Importing a job description file with the same basename as the current job description file results in defining different classes with the same name.

Error

job_rnr/lib/job_rnr/script.rb:11: warning: already initialized constant Regression
job_rnr/lib/job_rnr/script.rb:11: warning: previous definition of Regression was here

Notes

See call to classify.

scripts/stow/job_rnr/lib/job_rnr/script.rb
18-    def from_file(filename, base_class = nil)
19-      basename = File.basename(filename, '.jr')
20:      classname = classify(basename)
21-
22-      code_string = IO.read(filename)
23-      from_string(classname, code_string, filename, base_class)
24-    end

Boolean Plus Option Help Format

Boolean plus option help format is incorrect.

Expected

OPTIONS

  +default-false[=<value>]
    Default: false

  +default-true[=<value>]
    Default: true

Actual

OPTIONS

  +default-false=<value>
    Default: false

  +default-true
    Default: true

Add comparison to GNU Make to README

Add ARGV to options.argv for use by plugins

Jobs run with a failing parent if at least one other parent passes

Jobs should only be run if all parents pass.

Input:

job :passing_parent do
  execute 'true'
end

job :failing_parent do
  execute 'false'
end

job :failing_child, [:passing_parent, :failing_parent] do
  execute 'ERROR: this job should not run'
end

Command:

jobrnr examples/multiple-parents/index.jr --max-failures 2

Output:

Running: 'true' jobrnr00
Running: 'false' jobrnr01
Running:2 Queued:0 Completed:0 Passed:0 Failed:0
PASSED: 'true' jobrnr00 in 0.00s
FAILED: 'false' jobrnr01 in 0.00s
Running: 'ERROR: this job should not run' jobrnr02
Running:1 Queued:0 Completed:2 Passed:1 Failed:1
FAILED: 'ERROR: this job should not run' jobrnr02 in 0.01s <===========
Running:0 Queued:0 Completed:3 Passed:1 Failed:2

Obtuse error message on invalid option

jobrnr/lib/jobrnr/options.rb:59:in `parse': invalid option: --maxjobs (OptionParser::InvalidOption)
        from jobrnr/lib/jobrnr/application.rb:10:in `run'
        ...

No Checks on Imported Job Existence

Importing non-existent jobs silently fails. This condition should result in an error.

Reproduce

command:

jr index.jr

index.jr

import 'import', [:non_existent_job],  File.join(File.dirname(__FILE__), "import.jr")

import.jr

job :job do
  execute 'sleep 1'
end

Add graceful terminate on ctrl-c

Needs to account for subprocesses.

See https://stackoverflow.com/questions/14635318/having-a-io-popen-command-be-killed-when-the-caller-process-is-killed

"require 'rubygems'" is wrong?

http://2ndscale.com/rtomayko/2009/require-rubygems-antipattern

Specious error message when successor does not exist

Expected Error

Something like

Job 'absent_parent' does not exist for 'child' job @ error.jr:1

Actual Error

.../jobrnr/lib/jobrnr/job/definition.rb:19:in `block in initialize': undefined method `successors' for nil:NilClass (NoMethodError)

Reproduce

error.jr

job :child, :absent_parent do
  execute 'true'
end

jobrnr error.jr

Stack trace on bad option

f2e739b

> jobrnr --bad-option                                                                                                                                                          rdonnell@dline06 /home/rdonnell/sbox/cvac/1
/home/rdonnell/sbox/cvac/1/common/scripts/stow/jobrnr/lib/jobrnr/log.rb:14:in `report': uninitialized class variable @@verbosity in Jobrnr::Log (NameError)
Did you mean?  verbosity=
        from /home/rdonnell/sbox/cvac/1/common/scripts/stow/jobrnr/lib/jobrnr/log.rb:22:in `error'
        from /home/rdonnell/sbox/cvac/1/common/scripts/stow/jobrnr/lib/jobrnr/application.rb:13:in `rescue in run'
        from /home/rdonnell/sbox/cvac/1/common/scripts/stow/jobrnr/lib/jobrnr/application.rb:10:in `run'
        from /home/rdonnell/sbox/cvac/1/common/scripts/jobrnr:7:in `<main>'