jsoverson / grunt-preprocess Goto Github PK

View Code? Open in Web Editor NEW

294.0 17.0 41.0 77 KB

Preprocess files based off environment configuration

License: Other

JavaScript 95.65% CoffeeScript 2.59% HTML 1.75%

grunt-preprocess's Introduction

grunt-preprocess

Grunt task around preprocess npm module

What does it look like?

<head>
  <title>Your App</title>

  <!-- @if NODE_ENV='production' -->
  <script src="some/production/lib/like/analytics.js"></script>
  <!-- @endif -->

</head>
<body>
  <!-- @ifdef DEBUG -->
  <h1>Debugging mode - <!-- @echo RELEASE_TAG --> </h1>
  <!-- @endif -->
  <p>
  <!-- @include welcome_message.txt -->
  </p>
</body>

var configValue = '/* @echo FOO */' || 'default value';

// @ifdef DEBUG
someDebuggingCall()
// @endif

See preprocess documentation for more information

Getting Started

Install this grunt plugin next to your project's Gruntfile with: npm install --save-dev grunt-preprocess

Then add this line to your project's Gruntfile:

grunt.loadNpmTasks('grunt-preprocess');

Options

context

Type: Object Default: {NODE_ENV: 'development'}

The additional context on top of ENV that should be passed to templates. If NODE_ENV is not set, the task sets it to development by default.

inline

Type: Boolean Default: undefined

Required to enable overwriting of source files

srcDir

Type: String Default: <path to source file to be processed>

The directory where to look for files included via @include variants and @extend.

srcEol

Type: String Default: EOL of source file or os.EOL if source file contains multiple different or no EOLs.

The end of line (EOL) character to use for the preprocessed result. May be one of:

\r\n - Windows
\n - Linux/OSX/Unix
\r - legacy Mac

type

Type: String Default: file extension of the file to be processed

The syntax type of source file to preprocess. See preprocess() description for a list of all supported file types.

Example Usage

preprocess : {
  options: {
    context : {
      DEBUG: true
    }
  },
  html : {
    src : 'test/test.html',
    dest : 'test/test.processed.html'
  },
  js : {
    src : 'test/test.js',
    dest : 'test/test.processed.js'
  },
  multifile : {
    files : {
      'test/test.processed.html' : 'test/test.html',
      'test/test.processed.js'   : 'test/test.js'
    }
  },
  inline : {
    src : [ 'processed/**/*.js' ],
    options: {
      inline : true,
      context : {
        DEBUG: false
      }
    }
  },
  all_from_dir: {
    src: '**/*.tmpl',
    ext: '.html',
    cwd: 'src',
    dest: 'build',
    expand: true
  }
}

Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using grunt.

Release History

5.1.0
- Grunt peer dependency tagged >= 0.4.0, dependency updates
- added explicit dependency on lodash
- added logging for preprocess errors (@marcominetti, #53)
5.0.1 fixed processing of mutifile-style tasks for multiple different file extensions or different containing directories (#50)
5.0.0 bumped preprocess dep to 3.0.2, implemented backward-compatible mapping of old to new options and pass-through for new options (#34, #39, #48)
4.2.0 bumped preprocess dep to 2.3.1, bumped dev dependencies
4.1.0 bumped preprocess dep to 2.1.0
4.0.0 Switched order of context assignment, small change but necessitated major version
3.0.1 Fixed issue arising from undefined options (#19)
3.0.0 Updated dependencies, added merge from global options context to subtask context (#13)
2.3.0 Updated preprocess, changes default handling to html
2.2.0 Delegating to grunt's file.read/write for consistent usage within grunt (e.g. deep writes)
2.1.0 updated preprocess dependency
2.0.0 updated for grunt 0.4.0, moved context override to context option
1.3.0 Moved logic to 'preprocess' npm module
1.2.1 Added @include to include external files
1.2.0 Added @include to include external files
1.1.0 Added ability to process multiple destinations in a files block
1.0.0 Changed syntax, added directives
0.4.0 Added support for inline JS directives
0.3.0 Added insert, extended context to all environment
0.2.0 Added simple directive syntax
0.1.0 Initial release

License

Written by Jarrod Overson

Licensed under the Apache 2.0 license.

grunt-preprocess's People

Contributors

Stargazers

Watchers

grunt-preprocess's Issues

Support for lodash grunt templates in context and options

Seems the lodash templates are not resolved...

@echo '-' not working

Hi,

It seems that there is an issue when echoing a string containing '-. My code :

    <!-- @if DEMO -->
    <!-- @echo STATICURL -->
    <!-- @echo '<link href="https://s3' -->
    <!-- @echo '<link href="https://s3-eu' -->
    <!-- @echo '-' -->
    <!-- @echo 'other test' -->
    <!-- @endif -->

Output :

    <link href="https://s3
    <!-- @echo '<link href="https://s3-eu' -->
    <!-- @echo '-' -->
    other test

I think this comes from the exclusion of the '-' char in preprocess/regexprules.js

echo : "^@echo[ \t]*([^\n-]*)[ \t]*$",

can't remove

cfg:

preprocess : {
      options: {
        inline: true,
        context : {
        }
      },
      prod: {
        src: 'index.html',
        dest: 'index2.html'
      }
    }

input:

<!doctype html>
<html>
<body>
<!-- @ifndef DEBUG --> 
<script>
  console.group('init');
  console.time('load script');
</script>
<!-- @endif -->
<script>
  // @ifndef DEBUG
  console.timeEnd('load script');
  console.time('load sea script');
  // @endif
</script>
</body>
</html>

output:

<!doctype html>
<html>
<body>

<script>
  console.group('init');
  console.time('load script');
</script>
<script>
  // @ifndef DEBUG
  console.timeEnd('load script');
  console.time('load sea script');
  // @endif
</script>
</body>
</html>

Support for 'grunt server' when using yeoman

Yeoman and grunt newbie here..

I had the problem of having different URL:s for stage and production (and some other stage/prod related things) and grunt-preprocess did exactly what I wanted to do - great :)

However, I quickly ran into the problem that 'grunt server' does not preprocess files. I looked at the Gruntfile, but it seems that I need to serve the entire app from .tmp or similar. It seems that https://github.com/livedata/grunt-devcode is doing this, but at the same time grunt-devcode is too limited for me.

Am I missing something? It seems that without this usecase, yeoman is not supported?

Cannot read property 'context' of undefined

If options are not set, then the following warning is thrown:

Warning: Cannot read property 'context' of undefined Use --force to continue.

Because:

var options = this.options();
var origOptions = grunt.config('preprocess').options;
var context = _.extend({}, process.env, options.context || {}, origOptions.context || {});

Therefore:

var options = this.options() || {};
var origOptions = grunt.config('preprocess').options || {};
var context = _.extend({}, process.env, options.context || {}, origOptions.context || {});

is what I propose...

Please bump to preprocess 3.0.0 to allow recursive tags

Nested tasks

I'm using this task (which is awesome, btw) but I've run into a bit of a snag. This works fine:

preprocess: {
    dev: {
        src: 'grand_cherokee/index-grunt.html',
        dest: 'grand_cherokee/index-processed.html',
        options: {
            context: {
                DEBUG: true
            }
        }
    },
    production: {
        src: 'grand_cherokee/index-grunt.html',
        dest: 'grand_cherokee/index-processed.html'
    }
}

however, when I wrap that in another object, like so, it doesn't work:

preprocess: {
    gc: {
        dev: {
            src: 'grand_cherokee/index-grunt.html',
            dest: 'grand_cherokee/index-processed.html',
            options: {
                context: {
                    DEBUG: true
                }
            }
        },
        production: {
            src: 'grand_cherokee/index-grunt.html',
            dest: 'grand_cherokee/index-processed.html'
        }
    }
}

The reason for wrapping it is to be able to run this on multiple directories separately within the same project. Am I missing something? Or is there a better way to do this?

Lacks on documentation.

Hello.
We got some problems due on lacks on documentation.
Here down I'll list you all the problems that takes me 5 hours of swears:

on HTML side the rule doesn't work.
Instead the one wrapped with spaces it works well.
on HTML the OR,AND condition (on if statement) is never mentioned.

Please take a look at doc and fix it as well.
Davide

Can I use preprocess for targeting an app?

Hi;

I have a question. I want to use same codebase (AngularJS+nodeJS app) and target for 2 web apps. Only some css files, some controllers will be changed. Is it the right place to do such things?

Is there a way to do wildcard file processing without inline

I want to do the multi file but without listing each file (Basically process all files in the app folder and it's sub-folders and move the resulting files to the target directory maintaining the original directory structure). Anyway to achieve that? Something like:

multifile : {
src: 'app/*/;,
dest; 'target/'
}

Cheers

Target folders are not created

Hi, if you've got a tree of html files and the target subfolders don't exist you get the following issue:

Warning: ENOENT, no such file or directory '<>l' Use --force to continue.

Sensitive to whitespace

I noticed that inline HTML is sort of sensitive to whitespace. Like

<!-- @if DEBUG -->
<script src="bower_components/react/JSXTransformer.js"></script>
<script type="text/jsx" src="scripts/jsx/temp.jsx"></script>
<!-- @endif -->

will work.

But the below will not.

<!-- @if DEBUG-->
<script src="bower_components/react/JSXTransformer.js"></script>
<script type="text/jsx" src="scripts/jsx/temp.jsx"></script>
<!-- @endif-->

This can be particularly painful if using an HTML templating engine like Jade (in which I would write // @if DEBUG and // @endif) in an IDE which trims whitespace in the end.

Doesn't preprocess

grep version node_modules/grunt-preprocess/package.json
"version": "3.0.1",

grep @if source/client/index.html
<!-- @if ENV='dev'>
<!-- @if ENV='rel'>
<!-- @if ENV='dev'>
<!-- @if ENV='rel'

grep -A 8 preprocess: Gruntfile.coffee
preprocess:
source:
files: [
expand: true
cwd: "#{config.hierarchy.source}"
src: '*/.html'
dest: "#{config.hierarchy.build}"
filter: 'isFile'
]

ENV='rel' grunt preprocess

Local Npm module "grunt-cli" not found. Is it installed?

Running "preprocess:source" (preprocess) task

Done, without errors.

grep "@if ENV='rel'" build/client/index.html
<!-- @if ENV='rel'>
<!-- @if ENV='rel'>

Add 'type' option for multifiles

Currently you can specify 'type' option for the preprocess action, but as far as I can tell you can't specify this individually for each file in a multifile declaration.

It may be useful to be able to specify the type per file. Obviously a workaround is a different action for each type and only include the files for that type, although the action declarations could increase in number dramatically if there are many environments being configured.

Are prod:dev subkeys supported?

I'm used to running a grunt task subdivided into prod and dev subkeys, but I'm trying it with preprocess and it is not working (well it is, it's finding the correct options key and using the prod or dev keys, but it is not finding the files in the html key).

Any ideas?

preprocess: {
        prod: {
            options: {
                context: {
                    BUILDTYPE: 'production',
                    BUILDVERSION: '<%= pkg.clientversion %>',
                    BUILDDATE: '<%= grunt.template.today("yyyymmdd HH:MM:ss") %>'
                }
            },
            html: {
                src: '<%= pkg.srcDir %>/index.html',
                dest: '<%= pkg.dstDir %>/index.html'
            }
        },
        dev: {
            options: {
                context: {
                    BUILDTYPE: 'development',
                    BUILDVERSION: 'Development-build',
                    BUILDDATE: '<%= grunt.template.today("yyyymmdd HH:MM:ss") %>'
                }
            },
            html: {
                src: '<%= pkg.srcDir %>/index.html',
                dest: '<%= pkg.dstDir %>/index.html'
            }
        },
    },

Great project

Hello, I was working on something similar to this a long while back called samson. I went in the wrong the direction and lost interest in the project, but when I found this, it reinvigorated me.

The core of my preprocessor was just using the template engine doT, but what I wanted was a lot of functionality regarding hooks and ignores. I never actually got to the ignores, but I also didn't ever integrate with grunt which has a lot of functionality I was trying to build out.

Overall what I was trying to allow things such as:

conditionals, iteration, etc.
default defines for filename and dirname for the input and output
default defines for timestamp or date
convenience method to pass all script arguments to the process
convenience method to pass any arbitrary json object to the process
allowing direct access to javascript of course
allowing to include files relatively from the file being processed

Of course last but not least, allowing to preprocess any text file with a comment format. The cli tool I was going for was suppose to be simple to just specify a directory and mimi-magic + mimi-match would do the rest, checking mimes from file contents rather than extensions, and ignoring and just copying over any binary or unsupported files.

Really interested in this and the parent "preprocess" project of course.

Where do you intend to take these projects, what are the plans/next steps?

Parsing of inline <script>

Hi,

I wonder if it's my fault or if it's currently not supported:

I have an *.html document with some inline < script >, and I tried to remove a block:

//@ifdef DEBUG
< some js here >
//@endif

but it just doesn't work. Is js-parsing deactivated when parsing *.html-files? Are you planning to support inlined scripts?

Kind regards
Marc

More file types support

Actually, only .html and .js are supported. It would be very useful to add support for the following file types:
.php, .phtml, .cshtml, .less, .sass, .css

Ifdefs not removing code

Hi,

I am having difficulty getting ifdefs to work.

Here is my gruntfile

module.exports = function(grunt) {
// Configuratuion goes here

grunt.initConfig({

  preprocess : {

    options: {
      context : {
        DEBUG: false
      }
    },

    html : {
      src : 'dev/index.html',
      dest : 'dev/index.processed.html'
    }

  }
});

grunt.loadNpmTasks('grunt-preprocess');
grunt.registerTask('default', ['preprocess']);
}

and this is my html

<!DOCTYPE html>
<html>
<head>
</head>
<body>

<div id="wrap">

<!-- @ifdef DEBUG -->
  <h1>Test Page</h1>
<!-- @endif -->

<!-- @exclude -->
<header>You're on dev!</header>
<!-- @endexclude -->

<p>Test Pragraph</p>

<div id="output"></div>

</div>

</body>
</html>

But when I run grunt I get this

<!DOCTYPE html>
<html>
<head>
</head>
<body>

<div id="wrap">


  <h1>Test Page</h1>




<p>Test Pragraph</p>

<div id="output"></div>

</div>

</body>
</html>

Is there another step that I need to do in order to set the DEBUG variable so it will strip out the code between the ifdefs?

Merge Task Context into Target Context

It would be nice to have a default context via the task-level context object. This way, default context options do not have to be duplicated in each target, but can also be overridden, if desired. In the following example, the defaultText context option would be supplied to both targets:

preprocess: {
    options: {
        context: {
            defaultText: 'Add me to both scenarios.'
        }
    },
    one: {
        options: {
            context: {
                type: 'one'
            }
        }
    },
    two: {
        options: {
            context: {
                type: 'one'
            }
        }
    }
}

To achieve this currently, your targets would each need to declare the defaultText option, like so:

preprocess: {
    one: {
        options: {
            context: {
                type: 'one',
                defaultText: 'Echo me in both scenarios.'
            }
        }
    },
    two: {
        options: {
            context: {
                type: 'one',
                defaultText: 'Echo me in both scenarios.'
            }
        }
    }
}

I did a quick, successful test by replacing the assignment of context on line 27 with the following:

var context = _.extend( {}, defaultEnv, process.env, grunt.config( 'preprocess.options.context' ), options.context  || {} );

Update preprocess to include fix: default to 'html' for unknown file extensions

Please update preprocess version to include the commit default to 'html' for unknown file extensions.

Thanks.

is there way to preprocess css files?

I tried the sample code on preprocess module, but it is not working, have you tried to use it with HTML or CSS files?

CSS sample:

body {
/* @if NODE_ENV=='development' */
  background-color: red;
/* @endif */
}

Grunt Not Responding during preprocess single larger JavaScript file with Debug Source Map

My Web App only have one JavaScript file with Debug Source Map, the file size is about 3.4 MB, but the grunt-preprocess will be caused grunt not responding and took about 100% CPU.

Processing multiple files with a wildcard.

New feature request. I would like to be able to specify source as a wildcard, and destination as a directory. I have many html files (and JS files and CSS files) that I need to process for several environments over and over. Currently it appears that you can only specify a single file per line or per preprocess task. Unless I am mistaken, it seems that only inline will let you do wildcard preprocessing.

preprocess: {
html: {
src: './src/*/.html',
dest: './processed'
}
}

Could u add @else directive?

I tryed to use grunt-preprocess like this:

but it's not work for me.

I could use /* @if NODE_ENV='production' / / @if NODE_ENV='devel' */ construction,
but it looks silly.

Is it hard to add @else support?

Consider a different syntax

Hi!
I was looking just for this!

However the syntax is a bit non-intuitive/standard for pre-processors.
Would you consider something more similar to C pre-processors syntax such as in jsmacro?
https://github.com/smartt/jsmacro

Thank you!!

N00b question: How to set process.env?

Hi,

Sorry if I missed this in your docs (or the preprocess docs).

What's the recommended way to set the states (DEBUG/production/other)?

Found this tip on stack:

export NODE_ENV=development

But when echo NODE_ENV nothing's set.

Optimally, I'd like to set my ENV from my Gruntfile.js. Is this possible? I didn't see an example in your repo's Gruntfile.

Sorry if this is a dumb question. :(

Thanks!

src file is changed, not only dest

Here's what I've defined, as I want to reload certain scripts for MOBILE. However also the original app/index.html keeps changing.
I tried to remove the inline and give a dest, but didn't work -- what am I missing? :)

preprocess: {
      mobile: {
        src: '<%= yeoman.app %>/index.html',
        options: {
          inline: true,
          context: {
            MOBILE: true
          }
        }
      },
      web: {
        src: '<%= yeoman.app %>/index.html',
        options: {
          inline: true,
          context: {
            WEB: true
          }
        }
      }
    },

HTML:

    <!-- @ifdef MOBILE -->
      <script src="cordova.js"></script>
    <!-- @endif -->

Add a way to compare booleans

The "@ifdef VAL" works fine if you're passing in VAL = true to the context options.

What if you want to strip out some code if VAL = false?

Having an example or support for this would be great. Ex: "@ifndef VAL" or "@if VAL = 'false'"

any way to set dest directory?

Is it possible to preprocess all files from src to a dest directory? currently i have to do a copy task in order to make this happen.

optional tag for @include

It would be cool if there was some sort of optional tag for @include's, where if it can't find the template, it will pass gracefully.

Something like:

<!-- @include favicons.template.html !optional -->

Support for grunt targets

Hello,

I have the following task described in my gruntfile.js:

    preprocess: {
        html: {
          src: 'src/index.html',
          dest: 'dist/index.html'
        },
        options: {
          context: {
            DEBUG: false
          }
        }
    }

It works great (using grunt preprocess), but if I want to use grunt targets:

    preprocess: {
      dev: {
        html: {
          src: 'src/index.html',
          dest: 'dist/index.html'
        },
        options: {
          context: {
            DEBUG: false
          }
        }
      }
    }

It does not work anymore (using grunt preprocess:dev). Is this expected behaviour ? Or maybe I did something wrong

Create a standalone lib?

I'd love to see the core preprocessor as a stand-alone lib, where the grunt task exists as just a very lightweight wrapper around it.

Also, have you seen this?

https://github.com/mcoolin/Node-JavaScript-Preprocessor

Allow the file type to be specified in options

Instead of only assuming the file type can be derived from the file extension (https://github.com/jsoverson/grunt-preprocess/blob/master/tasks/preprocess.js#L49) please allow the file type to be specified in the options, for example:

preprocess: {
  html: {
    src: 'dist/layout.html.php',
    options: {
      type: 'html',
      inline: true,
      context: {
        DEBUG: false,
      }
    },
  }
},

This would allow html in .php files, or allow for files of a known type but with non-standard extensions (.ctp, .php5, .txt, etc.).

npm publish access

@jsoverson can you grant me npm publish access for this repo, too? Thanks!

Source pattern with no extension will not traverse all files

preprocess:
    build:
        options:
            inline: true
            context:
                ENV: 'dev'
        src: ['build/**/*']

Will give a file not found error

However, if I do src: ['build/**/*.js'] then it's all good.

Not sure if this is a bug or if it just needs to specified in the documentation.

Multifile broken with 5.0.0

Multifile doesn't work with 5.0.0: the only file that is processes is the first in the object.

Here is the code I've used:

module.exports = function(grunt) {

  // Project configuration.
  grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    preprocess : {
          options: {
            context : {
              DEBUG: true
            }
          },
          multifile : {
            files : {
              'test/test.processed.html' : 'test/test.html',
               'test/test.processed.js'   : 'test/test.js'


            }
          }
        }
  });

  grunt.loadNpmTasks('grunt-preprocess');

  // Default task(s).
  grunt.registerTask('default', ['preprocess']);

};

On v. 4.2.0 seems to work well instead

grunt ENV config

I was looking through the source and saw this line:

var context = _.extend({},defaultEnv,process.env)

Though defaultEnv doesn't seem exposed in any way. I would love to be able to override process.env variables with properties set within my grunt file, or even define my own context, maybe something like:

var context = this.data.context || _.extend({},defaultEnv,process.env)
if (this.data.env) _.extend(context, this.data.env);

Using grunt these of course can be expanded json file directives, and things get much more flexible.

coffee script?

what if I want to do preprocess for coffee script? does not seem to work. I have to persist the comment when converting coffee to js and to preprocess, then take out the comment. It would be great if preprocess and just work with coffee script

Grunt 0.4 Release

I'm posting this issue to let you know that we will be publishing Grunt 0.4 on Monday, February 18th.

If your plugin is not already Grunt 0.4 compatible, would you please consider updating it? For an overview of what's changed, please see our migration guide.

If you'd like to develop against the final version of Grunt before Monday, please specify "grunt": "0.4.0rc8" as a devDependency in your project. After Monday's release, you'll be able to use "grunt": "~0.4.0" to actually publish your plugin. If you depend on any plugins from the grunt-contrib series, please see our list of release candidates for compatible versions. All of these will be updated to final status when Grunt 0.4 is published.

Also, in an effort to reduce duplication of effort and fragmentation in the developer community, could you review the grunt-contrib series of plugins to see if any of your functionality overlaps significantly with them? Grunt-contrib is community maintained with 40+ contributors—we'd love to discuss any additions you'd like to make.

Finally, we're working on a new task format that doesn't depend on Grunt: it's called node-task. Once this is complete, there will be one more conversion, and then we'll never ask you to upgrade your plugins to support our changes again. Until that happens, thanks for bearing with us!

If you have any questions about how to proceed, please respond here, or join us in #grunt on irc.freenode.net.

Thanks, we really appreciate your work!

preprocess inside <script> tag on an HTML

Actually this code inside an index.html file is not working for me:

<div>text, tags, and moar moar things</div>
<script>
// @echo SOME_KEY
moarJsCode();
</script>

with this configuration on the Gruntfile:

preprocess: {
  options: {
    context: { SOME_KEY: 'var foo="bar"' }
  }
}

am I missing something? is it possible to parse js inside a tag in an html?