swipely/pipely

Visualize pipeline definitions for AWS Data Pipeline

pipely's Issues

no warning when project gem has un-added files

When using the pipeline-as-a-gem deployment strategy, the project's gem is built using git-ls. If you have added files to the project, but not yet added them to the git repo, then the deployed gem will be lacking these files. This can be very confusing to diagnose, as all local tests on your development machines will find the file.

Bundler's gem deployment Rake tasks get around this by refusing to release a gem if there are uncommitted changes in the working directory.[1] We should consider whether a similar strategy would be worth adding to pipely, to prevent this confusing type of failure.

Alternatively, the overhead of committing changes before each pipeline deployment may introduce too much friction, and we might consider whether we can avoid the use of git-ls when building the gem.

[1] https://github.com/bundler/bundler/blob/6725c27bdd31c60f5d768360debc508912529f97/lib/bundler/gem_helper.rb#L119-L129

"definitions" directory not automatically created

The graph:* rake tasks ensure that the project's graphs directory exists, by making the "graphs" directory task a prerequisite for graphs:full

pipely/lib/pipely/tasks/graph.rb

Lines 35 to 39 in 425fde8

	# create the `path` directory if it doesn't exist
	directory path
	namespace name do
	task :full => path do |_, task_args|

The definition rake task should do likewise. It does have a "definitions" directory task, but it is not correctly assigned as a prerequisite for the task that generates definitions.

pipely/lib/pipely/tasks/definition.rb

Line 37 in 425fde8

directory path

documentation

This gem looks really cool. I'd love to play around with it and even contribute, but could I trouble one of you to beef up the readme and/or make a couple wiki pages for getting started? Without spending a lot of time digging into the code, I can't tell what is supposed to work and what is still in progress (also with pipely-generator). The specs looks like they haven't quite kept up. Thanks!!!

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.