Giter Site home page Giter Site logo

fudge's Introduction

Fudge

Build Status Maintainability Test Coverage Gem Version

Description

Fudge is a CI build tool for Ruby/Rails projects.

Features

  • Define your CI build process in a Ruby DSL.
  • Keep your build definition in your code repository and version control.
  • Define different build processes for different branches of your code.
  • Run common tasks including code coverage assertion.
  • Define custom tasks for your build process.

Installation

Add to your project's Gemfile:

gem 'fudge'

Run in your project root:

bundle install

Usage

To create a blank Fudgefile, run in your project root:

bundle exec fudge init

To run the CI build (this is what you'd put in your CI server):

bundle exec fudge build

This will run the build named 'default'. To run a specific build in the Fudgefile:

bundle exec fudge build the_build_name

To list builds defined in your Fudgefile:

bundle exec fudge list

The list of builds can be filtered to include only those whose name match a given string (case-insensitive):

bundle exec fudge list a_build_name

Fudgefile syntax

To define a build with a given name (or for a given branch):

build :some_name do
end

An optional description can be provided by supplying an about string (descriptions, if supplied, are output by the list command):

build :some_name, :about => "Runs Rspec unit tests" do
end

To define some tasks on that build:

build :some_name do
  task :rspec, :coverage => 100
end

Any options passed to the task method will be passed to the task's initializer.

You can also use one of the alternative method missing syntax:

build :some_name do |b|
  rspec :coverage => 100
end

Composite Tasks

Some tasks are composite tasks, and can have tasks added to themselves, for example the each_directory task:

build :some_name do
  task :each_directory, '*' do
    task :rspec
  end
end

Task Groups

You can define task groups to be reused later on in your Fudgefile. For example:

task_group :tests do
  rspec
end

task_group :docs do
  yard
end

task_group :quality do
  cane
end

build :some_name do
  task_group :tests
  task_group :docs
  task_group :quality
end

build :nodoc do
  task_group :tests
end

Task groups can take arguments, so you can make conditional task groups for sharing between build. For example:

task_group :deploy do |to|
  shell "cp -r site/ #{to}"
end

build :default do
  task_group :deploy, '/var/www/dev'
end

build :production do
  task_group :deploy, '/var/www/live'
end

Callbacks

You can define success and failure callbacks using the following syntax:

build :default do
  rspec

  on_success do
    shell 'deploy.sh'
  end

  on_failure do
    shell 'send_errors.sh'
  end
end

Build will by default run without callbacks enabled. To run a build with callbacks, run:

bundle exec fudge build --callbacks

You can mix task groups with callbacks however you like, for example:

task_group :deploy do
  shell 'deploy.sh'
end

task_group :error_callbacks do
  on_failure do
    shell 'send_errors.sh'
  end
end

build :default do
  on_success do
    task_group :deploy
  end

  task_group :error_callbacks
end

Built-in tasks

Fudge supports several tasks by default. Most of them depend on a gem which also must be included in your project's Gemfile (or with add_development_dependency in your gem's .gemspec).

brakeman

Run the Brakeman Rails security scanner.

    task :brakeman

will fail if any security warnings are encountered.

    task :brakeman, :max => 2

will allow a maximum of two known security issues to get through.

cane

Checks code style using the cane gem. This can be run over the entire tree or in the enclosing subdirectory (each_directory or in_directory). Options can be set in the .cane file, but for convenience the max_width option can be used to easily override the default line width of 80.

  cane :max_width => 120

clean_bundler_env

Ensures that the code block runs in a clean Bundler environment.

each_directory

Run the resulting block in each directory (see examples above).

  each_directory 'meta_*', :exclude => ['meta_useless', 'meta_broken'] do
    ...
  end

rake

Run a rake command, requiring that it return success.

shell

Run a generic shell command, requiring that it return success.

sub_process

Like shell, but does not spawn a new shell to run the command, and allows more control over the command's process and environment. See examples below.

flay

Code duplication can be detected by Flay. See examples below.

flog

Flog calculates code complexity using an ABC metric and allows for maximum individual values and maximum average values. This can be used to ensure that you are alerted quickly when new complex code is added to your project. See examples below.

in_directory

Run the resulting block in a specific directory (as with each_directory).

rspec

Run rspec, enforcing minimum :coverage as a percent (using simplecov). See examples above.

cucumber

Run cucumber, enforcing minimum :coverage as a percent (using simplecov), like rspec.

yard

Runs YARD to ensure documentation coverage.

  yard 'stats --list-undoc', :coverage => 100

will require 100% coverage, and show all code that is not documented.

Defining tasks

A task is a class that responds to two methods:

  • self.name - A class method that returns a symbol representing the task. This is what will be used to identify your task in the Fudgefile. If not defined, it will be derived from the class name (e.g. in below example, it will be :loud_task).
  • run - An instance method which carries out the contents of the task. Should return true or false depending on whether the task succeeded.

For example, here is a simple task which will print some output and always pass:

class LoudTask < Fudge::Tasks::Task
  def self.name
    :loud
  end

  def run
    puts "I WAS RUN"

    true
  end
end

Registering your task

To make your task available to Fudge, you simply register it in Fudge::Tasks:

require 'fudge'

Fudge::Tasks.register(LoudTask)

This will make the LoudTask task available in your Fudgefile's like so:

build :some_name do
  task :loud
end

Extending the Shell task

Many tasks simply run a shell command and may accept some extra configuration options. To define a task of this kind, you can sublcass Shell and simply define the cmd method:

class LsTask < Fudge::Tasks::Shell
  def cmd
    "ls #{arguments}"
  end
end

The arguments method is provided by the Shell base class and will be a string of all other positional arguments passed to the task. For example:

build :default do
  task :ls, '-l', '-a'
end

would run the command ls -l -a.

You can take hash-like options, which will automatically be set if you have an attribute with the same name. For example:

class LsTask < Fudge::Tasks::Shell
  attr_accessor :all

  def cmd
    arguments << ' -a' if all
    "ls #{arguments}"
  end
end

Now this task can be used like so:

build :default do
  task :ls, :all => true
end

Checking output with the Shell task

You can define that some output from a command is required by responding to check_for with a regexp. For example:

class LsTask < Fudge::Tasks::Shell
  def cmd
    "ls #{arguments}"
  end

  def check_for
    /4 files found/
  end
end

The above task will only pass if the output contains "4 files found".

If you want to do some further processing on the contents matched by the regexp, you can provide an array with the second element being a lambda, which wil be called to process the output:

class LsTask < Fudge::Tasks::Shell
  def cmd
    "ls #{arguments}"
  end

  def check_for
    [/(\d+) files found/, lambda { |n| n.to_i >= 4 }]
  end
end

The above task will only pass if the output contains "n files found", where n is a number, and also n is at least 4.

Using the SubProcess task

This task is useful if you want to set an environment variable for a shell command, but the command won't allow the variable to be supplied at the end of the command line. That is, if something like this doesn't work because the command treats the variable assignment as a parameter:

task :shell, 'awkward_command SOME_VAR=true'
# This won't work either because shell tries to run a command called SOME_VAR=:
task :shell, 'SOME_VAR=true awkward_command'

SubProcess allows you to set the variable this way:

task :sub_process, 'awkward_command', :environment => { 'SOME_VAR' => 'true' }

SubProcess is also useful if you need to manipulate the process's execution environment, for example, by clearing environment variables, or redirecting IO. For example, this invocatin will unset all environment variables, except SOME_VAR which is explicitly supplied, before running command:

task :sub_process, 'command', :environment => { 'SOME_VAR' => 'true' },
                              :spawn_options => { :unsetenv_others => true }

See the Ruby doc for Process::spawn for details of the options that can be passed in spawn_options.

This task should otherwise act like the Shell task.

Defining composite tasks

Some tasks may require you to run a number of commands one after the other. You can hook into other fudge tasks by including the Fudge DSL into your composite task:

class DeployTask < Fudge::Tasks::CompositeTask
  include Fudge::TaskDSL

  def self.name
    :deploy
  end

  def initialize(*args)
    super

    task :shell, 'build_docs.sh'
    task :shell, 'cp -r docs/ /var/ww/deploy/docs'
  end
end
Fudge::Tasks.register(DeployTask)

The above will run the given tasks in the order defined, and only pass if both tasks pass. It can then be used in a FudgeFile like so:

build :default do
  task :deploy
end

Setting per-directory options for tasks

Sometimes you'll want different options to be used for specific subdirectories. This is especially useful with code metric tools.

Instead of having all of these values listed explicitly in your Fudgefile you can instead place them in a fudge_settings.yml file in each subdirectory.

So instead of this in your Fudgefile...

  in_directory 'meta_addresses' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/'
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :max => 20, :average => 5, :methods => true
  end
  in_directory 'meta_banks' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/', :max => 172
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :max => 74.9, :average => 9.1, :methods => true
  end

you can just have this:

  each_directory 'meta_*' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/'
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :methods => true
  end

and this in your meta_addresses/fudge_settings.yml:

flog:
  max: 20
  average: 5

and this in your meta_banks/fudge_settings.yml:

flay:
  max: 172
flog:
  max: 74.9
  average: 9.1

You can set the default values in your Fudgefile and override them only as necessary in specific subdiretories.

License

This gem is available as open source under the terms of the MIT licence.

Copyright (c) 2018 Sage Group Plc. All rights reserved.

fudge's People

Contributors

angelasilva avatar celeduc avatar chrisbarber86 avatar egilburg avatar kevinbrowne avatar kupkovski avatar loz avatar martinezcoder avatar pete-brown avatar slloyd-hirst avatar stevebelicha avatar tobscher avatar victormartins avatar whilefalse avatar wteuber avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fudge's Issues

Fuji 0.3.3 raises an error with Foreground error

After upgrading to this fudge version, this error is raised:

...lib/fudge/runner.rb:23:in output_start': undefined methodforeground' for "Running build ":String (NoMethodError)

After researches, seems that rainbow requires to add:
require 'rainbow/ext/string'

I tried to add it in the lib/fudge.rb:

require 'rainbow'
require 'rainbow/ext/string' # -> this fixes the issue
require 'active_support/all'

and it works

RSpec tests are incompatible with RSpec 3

Fudge has a development dependency on RSpec >= 2.8.0. Version 3.0 was released recently, which meets the dependency, but removes or deprecates features that our tests use:

  • its has been removed
  • the should syntax is deprecated and must be explicitly enabled

Available resolutions include:

  • its is available in a separate gem we can depend on, and the should syntax can be enabled via a config option
  • locking RSpec to >= 2.11 and < 3.0
  • rewriting the tests to eliminate its and use the expect syntax

The last option is probably best, because the should syntax will eventually be removed altogether.

Analysis plugin support

There should be a way for task gems to register analytics data into some adapter. There should be a null adapter, and flags to run with a chosen adapter (local builds would use null by default).

Then we could write, for example, an elastic search adapter which can put the task analysis into elastic search for something to produce nice trend reports.

We really just need to basic framework in place for the stable 1.0 release, that way the adapters can be written and improved later..

Complex regex not supported in exclude

You can't specify "complex" regex in the :exclude directories for file_find. This means you can't specify more than one subdirectory.

:exclude => 'spec' will properly exclude that word; :exclude => '(spec|db)' will include both.

Randomly order rspec to detect order dependencies

sop_ui_components has some order-dependent specs as well. These show up in Ubuntu because the OS returns files in file creation order instead of file update order in globbing, while (I believe) in OSX it always returns in alphabetical order.

I'd like to propose adding "rspec --order random" to Fudge so that it helps find these order dependency problems sooner. I feel conflicted about this because that would make each build a shiny unique sparklepony instead of a methodically reproduced process. It also dents the Principle of Least Surprise.

Fudge::Tasks::Task#initialize doesn't extract its options correctly.

Fudge::Tasks::Task#initialize copies and stores incoming arguments in the instance variable @args, and then extracts the options hash from the arguments and stores those in @options.

However, it extracts the options from the incoming arguments, not the copy in @args. Aside from the fact that #initialize alters its parameters—which is bad form—@args still retains the options hash, which is probably unintended.

This could affect a subclass that uses @args expecting—not unreasonably—that it doesn't include the options hash that was stored in @options.

Example (admittedly a little contrived):

# file: my_shell.rb
module Fudge
  module Tasks
    class MyShell
      def initialize(*args)
        super
        self.arguments = @args.join(' ')
      end
    end
    register MyShell
  end
end

# file: Fudgefile
build :default do
  task :my_shell, 'ls -l', foo: 'bar'
end

Running fudge with the above Fudgefile would result in

Running build default
Running task my_shell ls -l, {:foo=>"bar"}
ls: cannot access {:foo=: No such file or directory
Skipping callbacks...
Build FAILED!

Build directories in parallel

Extend each_directory to implement each_directory_parallel, which will allow the build process to use multiple cores. This would be explicit because if a build process has order dependencies it would break.

As a developer, I am tired of waiting for continuous integration to run. I want the build process to use all of the cores on my machine.

fudge_settings leak across directories

If I specify a value in fudge_settings.yml in a directory that same value will carry over to the next directory unless overridden by a fudge_settings.yml file with that same key.

e.g.:

  each_directory ['meta_one', 'meta_too'] do
    task :torture
  end

meta_one/fudge_settings.yml:

torture:
  threshold: 8
  average: 4

meta_two/fudge_settings.yml:

torture:
  threshold: 8

In this case, the torture task will still apply the average of 4 in meta_two.

Readme

Needs a nice readme on the current status, and how to define your own tasks etc.

colorize mutates source string

I couldn't replicate it in the fudge specs (maybe different versions of colorize or something else) but the following happens in my project that uses fudge:

str = 'asdf'
str.bright
str #=> "\e[1masdf\e[0m"

Causing the builds to break.

Changing str.bright to str.dup.bright fixes the problem

Collaboration with rubycritic

Hi,

just stumbled upon your repository. I will look into more, from my seconds of expose especially the completeness (brakeman etc. ) is interesting. But I couldn't see the report outcome on first sight.

Do you know of https://github.com/whitesmith/rubycritic, it's scope is very similar and it offers nice reports.

Maybe you should discuss some collaboration to make both projects better or to have one even better project.

Kind regards

New projects with no code fail Yard coverage

A new project with no files will fail yard because Yard calculates coverage at 0% instead of 100%.

Running task yard stats --list-undoc, {:coverage=>100}
Files:           0
Modules:         0 (    0 undocumented)
Classes:         0 (    0 undocumented)
Constants:       0 (    0 undocumented)
Methods:         0 (    0 undocumented)
 0.00% documented
Insufficient Documentation.

This should pass.

Task rspec doesn't load the whole environment

Hi Steven,

I try to use fudge to implement a rake ci task. I got the following error after running fudge build:

/home/tobias/Documents/dev/sage/einfach_buero/germanify/lib/germanify/engine.rb:67:in `block in <class:Engine>': uninitialized constant Germanify::Engine::AccountsEngine (NameError)

So the accounts_engine is not loaded.

When I cd germanify and run bundle exec rspec spec/ all works fine.

It only works partly when I add all my dependencies to my top level Gemfile. This is my Gemfile:

source :rubygems
source '[internal gem server redacted]'

gemspec :path => 'germanify/'
gemspec :path => 'germanify_ui_components/'

gem 'fudge', :git => '[email protected]:Sage/fudge.git'

How can I solve this problem?

Need to look at formatting of output code

As I've been tidying up a lot of the code to pass the newer quality metrics, it's apparent that the formatting of output is quite common, but is distributed throughout tasks and perhaps some abstraction would be in order.

Perhaps we could pass a formatting STDOUT or similar into the tasks so that they can be given a string and an abstract 'class' for it, and the way it looks is then done by the formatter.

This would lead nicely to the fudge_server possibly being able to format into HTML for example

Coverage testing issue when you have a multiple of 10 rspec tests.

If you have rspec tests which total a multiple of 10, then the coverage checker will not work if the tests are passing:

be fudge build
Running build default
Running task rspec {:coverage=>100}
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
......................................................................

Finished in 0.06867 seconds
70 examples, 0 failures

Randomized with seed 4963

Coverage report generated for RSpec to /Volumes/Macintosh HD 2/development/my_repo/coverage. 384 / 700 LOC (54.86%) covered.

Skipping callbacks...
Build SUCCEEDED!

The reason being that the regex to check the coverage, matches on 0 examples 0 failures which of course 70 examples 0 failures also matches!

Issue seems to be with this line: https://github.com/Sage/fudge/blob/master/lib/fudge/tasks/rspec.rb#L11

# Preconditions to check for coverage, that if not met make the test pass
# for example, if no tests exist, no need to fail
def pre_conditions_regex
    /(0 examples, 0 failures)/ # no tests exist
end

I believe this can be fixed by adjusting the regex as follows:

def pre_conditions_regex
    /^(0 examples, 0 failures)/ # no tests exist
end

Fudge build fails with no tests

If you don't have any test, the fudge build fails.

No examples found.


Finished in 0.00006 seconds
0 examples, 0 failures
Output didn't match (?-mix:(?-mix:((\d+\.\d+)%\) covered))|(?-mix:^(0 examples, 0 failures))).
Skipping callbacks...
Build FAILED!

The issue seems to be on the pre_conditions_regex method:
https://github.com/Sage/fudge/blob/master/lib/fudge/tasks/rspec.rb#L12

def pre_conditions_regex
    /^(0 examples, 0 failures)/ # no tests exist
end

The output string for no tests is something like this:
"No examples found.\n\n\nFinished in 0.00005 seconds\n\e[32m0 examples, 0 failures\e[0m\n"

The line don't start with "0 examples…" but with the console color code "\e[32m" so we could change the method to

def pre_conditions_regex
    /[^\d](0 examples, 0 failures)/ # no tests exist
end

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

flog hangs with multiple :exclude directories

The :flog directive works with a single :exclude option but not more than one. If more than one :exclude option is specified for :flog the process will hang.

This works with :flay, but not with :flog.

  • This works:
    task :flay, :exclude => ['spec/','db/']
  • These also work:
task :flog, :exclude => 'spec/', :max => 20, :methods => true
task :flog, :exclude => 'db/', :max => 20, :methods => true
  • This doesn't work:
    task :flog, :exclude => ['spec/','db/'], :max => 20, :methods => true

Documentation for running custom build name

Fudge readme is good on describing how you can define a build with a custom name (as opposed to default), but I don't think it actually says anywhere how to run a non-default build...

Add autoloading

I thought getting rid of autoloading would lead to neater code, turns out it doesn't - requires all over the shop.

Lets get autoloading back.

Provide a way to specify per-directory metrics for flog, flay, etc.

Repos with multiple subdirectories need to have different thresholds for errors. In the example below one sub-gem is very small and has very low flog/flay scores, but another is very large and has much higher scores.

task_group :analysis do
  in_directory 'meta_addresses' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/'
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :max => 20, :average => 5, :methods => true
  end
  in_directory 'meta_banks' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/', :max => 172
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :max => 74.9, :average => 9.1, :methods => true
  end
  in_directory 'meta_invoices' do
    task :flay, :exclude => '^\.\/(db|factories|spec)\/', :max => 2117
    task :flog, :exclude => '^\.\/(db|factories|spec)\/', :max => 46.8, :average => 6.5, :methods => true
  end
end

We need higher granularity, and these metrics probably don't all belong in the Fudgefile in the root directory.

Proposal: in each subdirectory, include fudge_settings.yml:

flay:
  max: 2117
flog:
  max: 46.8
  average: 6.5

Fudge can read these files when operating inside a given directory (with in_directory or each_directory) and provide those settings to the specified task when called.

Spin out none-core tasks to own gems

Fudge should supply:

  • shell
  • compound
  • bundle_aware
  • subprocess

Everything else should be a separate gem:

  • fudge-rspec
  • fudge-yard
  • fudge-cane
  • fudge-flog
  • fugde-flay

this would allow #47 to be created as fudge-rubocop and then people can pick and choose which analysis they want to include.

Get rid of unnessecary exceptions

Though this was a good idea at the time, but exceptions whouldn't be used for command line stuff like no arg given and no command found. Just print a message and print the usage.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.