Giter Site home page Giter Site logo

howardroark / pollinate Goto Github PK

View Code? Open in Web Editor NEW
223.0 7.0 16.0 1.72 MB

Template your base files and generate new projects from Git(Hub).

License: The Unlicense

JavaScript 100.00%
generator scaffolding templates schema parsing project boilerplate skeleton cli git

pollinate's Introduction

Pollinate

Generate a new project directly from Git(Hub) using a simple schema.

npm version Build Status Dependency Status

What?

It is a command that takes a templated tree of files and generates them for new projects using data defined by a simple schema. The data can define an output name, files to discard, files to parse with the data, and files to move or rename. The template can supply the default data, and that data can be extended for each project. You can throw in any other data you'd like to be passed to the template context as well.

All templates are parsed with Nunjucks aka Jinja and Twig.

Why?

When starting new projects the quickest way is often to just copy the last project and fiddle with it until it works. This can introduce many unwanted issues, like having one client's name appear in place of the other's. This project's goal is to offer an elegant and efficient way of working with a base set of files that can be understood by looking at a single example.

Install

$ npm install -g pollinate

An example

$ pollinate howardroark/webapp --name newproject --image alpine --description="A thing that does something."

Skip to more examples...

The GitHub sourced template
.
├── PROJECT-README
├── README.md
├── Dockerfile
├── project-name
└── template.json
template.json (optional)
{
  // Core schema
  "name": "webapp",
  "parse": [
    "PROJECT-README",
    "Dockerfile"
  ],
  "discard": [
    "README.md",
    "template.json"
  ],
  "move": [
    { "PROJECT-README": "README.md" },
    { "project-name": "{{ name }}.txt" }
  ],
  // Custom defaults
  "image": "debian",
  "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit."
}

You can omit any or all of discard, parse and move.

PROJECT-README
# {{ name }}

{{ description }}
Dockerfile
FROM {{ image }}
The project data
{
  "name": "newproject",
  "image": "alpine",
  "description": "A thing that does something."
}
The data after extending and parsing
{
  "name": "newproject",
  "parse": [
    "Dockerfile",
    "PROJECT-README"
  ],
  "discard": [
    "README.md",
    "template.json"
  ],
  "move": [
    { "PROJECT-README": "README.md" },
    { "project-name": "newproject.txt" }
  ],
  "image": "alpine",
  "description": "A thing that does something."
}
The result
.
└── newproject
   ├── README.md
   ├── newproject.txt
   └── Dockerfile
README.md
# newproject

A thing that does something.
Dockerfile
FROM alpine

More options

You can specify template files as a local directory (.git will be removed)

$ pollinate ./template --name newproject --image ubuntu

You can use any Git url (.git will be removed)

$ pollinate https://github.com/howardroark/webapp.git --name newproject --image ubuntu

You can pass project data as a file

$ pollinate howardroark/webapp data.json

You can pass project data as a JSON string

$ pollinate howardroark/webapp '{"name":"newproject","image":"alpine","description":"A thing that does a thing."}'

You can pass project data as a JSON endpoint

$ pollinate howardroark/webapp https://example.com/json/data

You can generate from the default data in the template

$ pollinate howardroark/webapp

You can override data as CLI options

$ pollinate howardroark/webapp data.json --name=alternate --image=ubuntu

You can specify a command to run on completion

{
  "complete": "git init {{ name }}"
}

You can supply user specific data each time with a ~/.pollen defaults file

{
  "api_key":"secret"
}

You can preserve the commit history from the skeleton project with the --keep-history CLI option or:

{
  keepHistory: true
}

Filters

You can supply custom Nunjucks filter functions (files must be included within template)

{
  "filters": {
    "markdown": "filters/markdown.js"
  }
}
filters/markdown.js
var markdownParser = function() { ... }

module.exports = function(markdownText) {
  var html = markdownParser(markdownText)
  return '<div class="markdown">'+html+'</div>'
}

Parse

All parse paths are first passed to globby

{
  "parse": ["*"]
}
{
  "parse": [
    "*",
    "!templates"
  ]
}

Merge

You can specify .json files to merge

package.json

{
  "name": "@howardroark/webapp",
  "version": "1.0.0",
  "description": "project for testing pollinate with merge",
  "main": "index.js",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/howardroark/webapp.git"
  },
  "author": "Andy Edwards",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/howardroark/webapp/issues"
  },
  "homepage": "https://github.com/howardroark/webapp#readme",
  "dependencies": {
    "lodash": "^4.17.4"
  }
}

PROJECT-package.json

{
  "name": "@{{ organization }}/{{ name }}",
  "version": "1.0.0",
  "description": "{{ description }}",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/{{ organization }}/{{ name }}.git"
  },
  "author": "{{ author }}",
  "bugs": {
    "url": "https://github.com/{{ organization }}/{{ name }}/issues"
  },
  "homepage": "https://github.com/{{ organization }}/{{ name }}#readme",
}

template.json

{
  "name": "webapp",
  "description": "project for testing pollinate with merge",
  "organization": "howardroark",
  "author": "Andy Edwards",
  "parse": [
    "PROJECT-package.json"
  ],
  "merge": [
    ["package.json", "PROJECT-package.json"]
  ],
  "discard": [
    "PROJECT-package.json"
  ]
}

command

pollinate howardroark/webapp#merge-test --name myapp --description 'my new app' --organization myorg --author Me

This will overwrite package.json with the contents of package.json and PROJECT-package.json merged with lodash.merge. This is useful when you want to keep template variables out of package.json, since they would cause certain npm commands to fail.

Resulting package.json

{
  "name": "@myorg/myapp",
  "version": "1.0.0",
  "description": "my new app",
  "main": "index.js",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/myorg/myapp.git"
  },
  "author": "Me",
  "license": "ISC",
  "bugs": {
    "url": "https://github.com/myorg/myapp/issues"
  },
  "homepage": "https://github.com/myorg/myapp#readme",
  "dependencies": {
    "lodash": "^4.17.4"
  }
}

Thanks

  • @binhood for the fantastic work on the logo!
  • @jedwards1211 for the handy object merge option :)
  • @ben657 for refactoring a bunch of stuff :o

pollinate's People

Contributors

ben657 avatar howardroark avatar jedwards1211 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

pollinate's Issues

Does not automatically create nested folders in `move`

This is my move section:

{"Production\\ProsperoTemplate\\src\\ProsperoTemplate\\ProsperoTemplate.Web.xproj":"Production\\ProsperoTemplate\\src\\ProsperoTemplate\\{{ name }}.Web.xproj"},
{"Production\\ProsperoTemplate\\src\\ProsperoTemplate.Core\\ProsperoTemplate.Core.xproj":"Production\\ProsperoTemplate\\src\\ProsperoTemplate.Core\\{{ name }}.xproj"},

{"Production\\ProsperoTemplate\\ProsperoTemplate.sln":"Production\\ProsperoTemplate\\{{ name }}.sln"},

{"Production\\ProsperoTemplate\\src\\ProsperoTemplate\\":"Production\\ProsperoTemplate\\src\\{{ name }}\\"},
{"Production\\ProsperoTemplate\\src\\ProsperoTemplate.Core\\":"Production\\ProsperoTemplate\\src\\{{ name }}.Core\\"},

{"Production\\ProsperoTemplate\\":"Production\\{{ name }}\\"}

I have structure like this:

Production\ProsperoTemplate\src\ProsperoTemplate\
Production\ProsperoTemplate\src\ProsperoTemplate.Core\

When running the command pollinate ./ProsperoTemplate --name=Sample

I expect the structure to end up:

Production\Sample\src\Sample\
Production\Sample\src\Sample.Core\

However, when running once, I get:

Production\ProsperoTemplate\Sample\
Production\ProsperoTemplate\Sample.Core\

I delete entire Sample\ directory and rerun the command, and end up with

Production\Sample\src\Sample\
Production\Sample\src\ProsperoTemplate.Core\

I suspect there's a concurrency issue somewhere, although I can't tell why just by looking at the source, seems a fairly standard synchronous loop, so I assume the mv is non-blocking...

Add tests

Pretty self explanatory. A test for each example in the README should do the trick.

Offer optional hashsum check

Some setups may want to offer additional security for build systems. As a start a CLI hashsum option could be checked against the corresponding github tar.gz.

pollinate username/template --hashsum=MD5

Offer an `init` command

Offer a pollinate init newtemplate command as a way to auto generate a template or modify an existing tree of files.

Allow specifying target directory as cli input

As of now the output directory is equal to name. It would be nice if the tool acted a bit more like the git cli and let you specify the target as an optional second input...

$ pollinate howardroark/webapp newproject
$ pollinate howardroark/webapp .
$ pollinate howardroark/webapp newproject data.json

If the second input is not a file/json/url then assume it is a target and then check the 3rd input.

Silent errors?

Hi! I would love to use pollinate. However, if I run pollinate wunderflats/javascript-boilerplate '{"packageName": "hello"}', it does nothing.

~/javascript-boilerplate $ pollinate wunderflats/javascript-boilerplate '{"packageName": "hello"}'
~/javascript-boilerplate $ ls
~/javascript-boilerplate $
$ node -v
v5.3.0
$ npm -v
3.5.2

Parse all files except specific files/directories

I'm trying to get Pollinate to work with a WordPress theme that HAS twig files within it that I don't want parsed by Pollinate, they are parsed as part of WordPress, in my project those files are in web/app/themes/template-theme/templates

I tried having the parse array something like this:

   "parse": [
    "*",
    "!web/app/themes/template-theme/templates"
  ],

but that doesn't seem to parse anything... is that a feature that I'm just not entering correctly - or something that would be a feature request?

Make the example in the README work.

The README outlines a basic example that should work for the beta release.

  • Implement duojs/package to download repo files to tmp location
  • Implement and apply discard operation to tmp files
  • Parse all files marked parse nunjucks and pass the context object
  • Implement and apply move operation to tmp files
  • Move tmp files to the current directory and name the folder as the name property

Parse explicit directories

When I try to add a directory to the parse array it doesn't seem to work at all, is this a feature request or something that I am doing wrong... e.g. for a repo I am trying to add all files within the config directory, I have tried:

 "parse": [
    "Capfile",
    "bower.json",
    "composer.json",
    "Gruntfile.js",
    "package.json",
    "config/*"
  ],

and

 "parse": [
    "Capfile",
    "bower.json",
    "composer.json",
    "Gruntfile.js",
    "package.json",
    "config"
  ],

and when I run pollinate mygitnamesapce/myproject when I run it, it does nothing... not even output the json that it typically does.

BTW, thanks for this project it's awesome!

Add "link" to the schema.

It would be really great to be able to specify symlinks upon generation. Could work just like move but instead be link.

"link": [
    { "origin-path": "linked/path" }
],

linked/path should end up having a relative path symlink back to origin-path

See PR #55

What is the best development virtualization option to use? Vagrant or Docker?

I tend to like Vagrant because it is easy to install, and so is VirtualBox. Docker is quick but seems fidgety when trying to develop with it, plus you have to use VirtualBox anyway.

In terms of OS... to me it is more about how you "provision" them. What OS do people enjoy working with? Performance and blah, blah aside.

(Yes I have OCD with issue titles, but it's important! Kind of...)

Enable custom nunjucks `filters` as a {"filter name":"module path"} schema object property.

First off, great little tool!

Would be nice to be able to do some parsing of the passed in options though, things like capitalising a name and having that available to templates too. Maybe give a js file which could export a function to do some work on the data.

e.g.

module.exports = function(data) {
  let capitalName = data.name[0].toUpperCase() + data.name.slice(1);
  data.capitalName = capitalName;
}

Then {{ capitalName }} would be available in templates.

Reduce file related dependencies

This project has too many dependencies that do different things in different ways with files. Would be nice to be able to reduce this number. Ideally it does not involve adding more code to this project, but refactoring is totally fine.

add option to not delete commit history

I typically keep a ref to the skeleton repo around, because I'm constantly improving my own skeletons, and I often pull the improvements into projects based off of them.

separate "command" key from data key

Since all data and command are top-level, it's hard to distinguish data key (like "name", "description", etc.) from command ("parse", "discard", "move").

Also, in the current state, each new command is a backward incompatible change because you never know if someone was not using your new command name as a key for data.

My proposition would be to create a key "data" that would contain all the data you want to pollinate in your template, so the separation between data and command is clearer,
for backward compatibility sake, both using data or putting data in the top level could work for now.

An alternative solution would be to prefix all command with a character like _ and to refuse data key starting with the same character, thus again separating the domain space of the keys.

As an aside, it would also be nice to have in the readme the complete list of command keyword near the top,

This is a very nice project, hope you can keep it up!

Allow setting default organization.

It would be cool to set a default org to default to so that you could just specify repo name.

$ pollinate test-flower

vs

$ pollinate codingcoop/test-flower

Complete command should accept an array

Be great if the complete command accepted a string[]. Ultimately, I want to do this:

{
  "complete": [
    "cd {{ name or org }}",
    "git init",
    "npm install",
    "git add .",
    "git commit -m 'initial commit'",
    "git remote add origin https://github.com/{{ org or name }}/{{ name or org }}.git",
    "git push -u origin master",
    "code ."
  ]
}

Related: #13

Introduce an optional "context" property.

Some people may not want the base schema to be passed to their templates. It would be good to allow for a context property that is passed to the templates instead. Extending name into this context would be helpful as well.

Support for any 'platform'

Both bitbucket and gitlab also offer the user/repo pattern. It would be cool if pollinate could be configured to point to another platform with the same pattern.

Add `merge` option to schema

The idea here is to add a new "step" which merges the data of json (or other) object files. The first file in each array being the file to write to.

{
  merge: [
    ['package.json', 'PROJECT-package.json']
  ]
}

replace deprecated package

Installing pollinate currently show the following warning for deprecated package:

npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated [email protected]: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated [email protected]: this library is no longer supported
npm WARN deprecated [email protected]: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated [email protected]: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated [email protected]: request has been deprecated, see https://github.com/request/request/issues/3142

Offer the option for a home folder defaults file.

It would be nice to be able to extend the data with default user preferences like ids and keys. Pollinate could check for a file in the user home directory and extend that before the the cli specified source.

Offer an API

Right now this is command line centric, but could easily offer a simple API to fit into other node based build processes.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.