Make Grommet-Toolbox a generator app?

[grommet]

Make Grommet-Toolbox a generator app?

[michaelgilley]

For noting purposes, by using node scripts we could also take an advantage of concurrent/parallel tasks w/ mutexes to speed up process times.

[alansouzati]

The first pass for this is available in the pull request.

grommet/grommet-cli#10

@michaelgilley I really would like to hear your feedback on it. There are two new tasks check and pack. The details for those tasks are available in the pull request description

[alansouzati]

Yeah!! @ericsoderberghp just merged grommet-cli#10. But I would love your feedback about these two new tasks.

[alansouzati]

Thanks for your question. I've been evaluating this request for a while now. But I still see value in this project as it deals with copying assets, packing contents with its dependencies, and testing.

I know you get all that with few additional dependencies in your project but I see the toolbox as an important piece of getting started quickly and also providing an easy way to update things in the future.

If it is an "app generator" where the configuration will live inside npm scripts, it will make future updates much harder as we will have no control over this file as soon as someone generates the app the first time. It is sort of saying, you generate once and then you are on your own...

I'm probably not doing a good job with semantic versioning though, from Jest 15 to Jest 16 I should have released a breaking change and I didn't. So I'm sorry for that.

I'm curious to understand better which features you see in create-react-app that is not easily available in grommet-toolbox?

We think this would cut down on the required issue->PR->review->release loop that must take place every time a feature or proprietary change is needed.

I think the version upgrade for libraries will need to happen regardless. I just need to do a better job of keep the semantic versioning right.

[michaelgilley]

Hey @alansouzati, the idea here is to still use the bootstrapping that grommet-toolbox (GT) provides but without the lingering dependency. There's a lot to this so I'll try to be concise...

Generator apps like create-react-app and the entire Yeoman ecosystem are popular because they bootstrap tech stacks so that getting up and started is fast BUT they do set you off on your own. I tend to see this as a plus, actually. If you own the generated code you can control it as the precise requirements of your project evolve, which will inevitably differ from someone else's needs.

As it is, if a new feature is needed (think extra reporter that you want to run on top of a GT testing task) one would have to either: a) overwrite the gulp task defined by GT or b) submit a PR that might not be accepted and wait for the release that may or may not be timely given the loose dependency with Grommet. We've had to do both by either adding features as needed that were sometimes short-lived or overwriting tests, for instance, when we moved from tape to Ava (again, short-lived). For other users who don't know or care about these specific requirements they may have the displeasure of having GT update and pull the rug out beneath them (what if I wanted to continue using tape?).

If GT were a generator app each consumer would own the generated code in her code base and thus could make changes as she saw fit while also benefitting from the fast startup time. When updates to the generator come out those users would then just read the change logs and make the appropriate changes as desired. This is how updates with most generators are. I also suppose patch scripts could be written to easily update users if their configurations hadn't changed.

Of course, as yeoman demonstrates, GT could still be just as gulp-based, which in the end would make it even easier to update. No need for npm scripts only.

So yes, updates are more manual with a generator app BUT it's the price paid for more control given to the user. Actually, these are very close models yet different enough I think they warrant discussion.

[alansouzati]

I think this is a good discussion to have, I feel your pain with the update process and wanting to use different things at a given point in time, like the example you gave on Tape.

Well, GT is very close create-react-app. I just created one sample app with that and my package.json looks like this:

{
  "name": "testing",
  "version": "0.1.0",
  "private": true,
  "devDependencies": {
    "react-scripts": "0.7.0"
  },
  "dependencies": {
    "react": "^15.3.2",
    "react-dom": "^15.3.2"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  }
}

As of this moment I see react-scripts as our grommet-toolbox dependency. And this good in my opinion. Imagine if you are starting a new project and you don't have a lot of experience. If we add all the dependencies immediately I don't think it will help users.

If you know what you want I would say it is easier to configure the things you don't like, than for a new UI developer to understand everything that is being added and what he needs to remove/add to get going. Again, if we expose all the configuration inside toolbox it will be very hard, if not impossible, to send fixes and improvements for the folks that are not necessarily savvy with all this configuration.

Don't get me wrong, I'm glad you are bringing this up, it makes me think ways to improve GT in a way that will benefit everyone. That said, I think we need the eject functionality that is present in the create-react-app, which is an awesome project BTW.

When I generated a create-react-app I saw something like this:

npm run eject
Removes this tool and copies build dependencies, configuration files
and scripts into the app directory. If you do this, you can’t go back!

Well this is pretty much it, if you expose the config, there is no coming back and then you are on your own. But folks that decide to run eject, they probably know what they are doing.

Thoughts?

[michaelgilley]

Yes, that was my thoughts after using create-react-app as well. An eject method might be useful so that users can completely unhook from GT if needed. From that point on, they could follow more detailed changlelogs in order to keep up with the project, if desired.

This also brings up another related topic: whether GT should be a CLI package rather than a collection of gulp tasks that rely on the user having gulp already included as a dependency. I think the benefit of doing something like this would be even less configuration on the part of the user. For instance, gulp would be installed behind the scenes and an init script could produce the grommet-toolbox.config script for the user given some prompt inputs. I think in the end it would make the bootstrapping process that much easier. Another benefit would be an abstraction of how tasks were done so not everything needs to be kept within gulp. Sync tasks that rely on their own cli packages (like Jest or Webpack) could simply be run as npm scripts while copying, generating, and other buffer-based tasks could live in gulp, where gulp's streaming really excels. One more advantage might be the ease of an updater command that could try to update a user's grommet-toolbox config file or something like yarn's self-update...

What do you think about this?

[michaelgilley]

Speaking of which, how does this intersect with Grommet-cli?

[alansouzati]

I see grommet-cli as an interface to generate projects with a given template. For example, I'm soon going to work on a blog engine for Grommet, I thought about using the cli to provide a template for such an app. I haven't thought of it as something more powerful like a developer environment. Or using it to seed the toolbox configuration (good idea BTW).

The more I investigate create-react-app, the more I feel we can try and move away from gulp. I don't like the idea of some of the scripts using it, and some not.

Also, after running the eject command, I feel it added too many things into my project. For example the generated config folder feels super heavy to me.

That said, if I were to start GT 2.0.0 I would take the following path:

1. Join grommet-cli and grommet-toolbox and make it one single tool to run things (both generate projects and dev env).
2. Move away from Gulp entirely and use Node scripts to run things (not NPM scripts).
3. Do not offer eject capabilities but make things configurable through a config file (like the grommet-toolbox.config.js).

I have a lot of ideas for this possible new project. So let me know your thoughts and if you agree with this plan I can start talking with Grommet folks to allocate time for me to work on this. I'm excited :)

If you want to participate in the development process I can involve you, so let me know...

[michaelgilley]

Yes, gulp is a a good tool for piping multiple files through a line of processors but I don't like it being used as a all-inclusive cli tool. Projects that don't depend on buffered file contents shouldn't be coerced to run inside of a gulp task, IMHO. That being said, even gulp can be easily swapped out for node scripts that use a streaming interface.

Speaking of this, I agree that using node scripts would be far superior to relying on shell via npm. Windows has proven to be a headache with such implementations.

There is a lot to what create-react-app injects into your project on eject, but I don't fault the project. I think it's just a mirror of the craziness that is modern UI javascript development right now. It's also pretty clear that they don't expect you to need to eject. Anyway, something to keep in mind is that gulp overtook grunt like it did because (besides its ability to handle files more efficiently) it embraced a code-over-configuration methodology instead of grunt's heavy configuration-based one. I think more experienced developers appreciated this because it empowered them to make changes to their build process architecture as needed whereas configurations can only offer minor changes to an opinionated sequence.

Of course config options are a good way to go for bootstrapping a project and for less-experienced devs but I would argue that the need for manipulating how something gets done would still be present. This could be done by either expecting users to overwrite a command or substitute their own version of the node script. Perhaps there could also be the ability to eject only portions of the toolbox instead of the whole thing? E.g., how could I change, for example, webpack@2 to be used instead of 1 since that would require more than a simple config argument?

Re your roadmap:

1. Love the idea of merging the projects to create a dev/building env that contains a configurable project scaffolder.
2. 👍 on using Node scripts over larger abstractions
3. Config via a file or cli arguments is a win but I still think users need the ability to gain full control if they want/need to.

I do think this a good route to go down and given some more honing it would make for a robust project. I'd love to help out! :)

[RyanCCollins]

@alansouzati this roadmap looks great! I am really excited to see it progress.

Is there anything that I can do to help at this stage? Do you need an extra set of eyes on #10?

[alansouzati]

I added a video on how to use the new CLI, check it out:

https://vimeo.com/199912095