pazdera/gitwalk

Bulk processing of git repositories

gitwalk's Issues

Create repositories

Hello,

Perhaps I've missed something, but can I create repositories using this? I'm looking for a bulk build script, but I cannot see one, and I figured if gitwalk could do it, it would be almost perfect! :)

Cheers.

Install fail

▶ npm install -g gitwalk
npm WARN deprecated [email protected]: 'github' has been renamed to '@octokit/rest' (https://git.io/vNB11)
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
/usr/local/bin/gitwalk -> /usr/local/lib/node_modules/gitwalk/out/bin/gitwalk.js

> [email protected] install /usr/local/lib/node_modules/gitwalk/node_modules/nodegit
> node lifecycleScripts/install

/usr/local/lib/node_modules/gitwalk/node_modules/promisify-node/utils/args.js:9
  var args = func.toString().match(/function\s.*?\(([^)]*)\)/)[1];
                                                              ^

TypeError: Cannot read property '1' of null
    at module.exports (/usr/local/lib/node_modules/gitwalk/node_modules/promisify-node/utils/args.js:9:63)
    at /usr/local/lib/node_modules/gitwalk/node_modules/promisify-node/index.js:79:29
    at Array.filter (<anonymous>)
    at processExports (/usr/local/lib/node_modules/gitwalk/node_modules/promisify-node/index.js:60:6)
    at module.exports (/usr/local/lib/node_modules/gitwalk/node_modules/promisify-node/index.js:112:10)
    at Object.<anonymous> (/usr/local/lib/node_modules/gitwalk/node_modules/nodegit/generate/scripts/generateNativeCode.js:3:13)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
    at Module.load (internal/modules/cjs/loader.js:977:32)
    at Function.Module._load (internal/modules/cjs/loader.js:877:14)

gitwalk clones local repositories

The README says:

You can match repositories on your file system using glob. Note that gitwalk will clone the repository
even if it's stored locally on your computer. One can never be too careful.

Sorry, but I strongly disagree ;-) Firstly, I'd suggest that this is too careful:

• For gitwalk processors which only read repos and don't write to them, there is zero risk.
• The "original" repos (i.e. the ones which already existed on the local file system) are still git repos, so even if the changes by gitwalk were wrong, you can simply use normal git commands to revert them. That safety net is the whole point of git :-) And anyway, if the user is deliberately invoking changes on the repo then they are surely already aware of the risks. They could already do the same thing manually, one repo at a time.

If there were no downsides to being careful, I'd concede that OK, no harm done. But unfortunately this approach has several big disadvantages to the user experience:

• For gitwalk processors which write to repos, this makes those processors highly inconvenient: the writes happen on the gitwalk-owned clones, so if you wanted the changes in your original repos, you'd then have to manually propagate the changes across. It would actually be less work to construct a shell for loop to work directly on the original repos, which defeats the point of gitwalk.
• It's a needless waste of disk space, especially since gitwalk builds one clone per branch of each original repo.
• The first time gitwalk runs on these local repos, it will be annoyingly slow due to this cloning.
• For subsequent times gitwalk has to fetch updates from the original repos, which is also a loss in performance, and has another unfortunate side-effect:
• If you run a processor which writes to the repos (e.g. via sed -i) and then another one which reads (e.g. grep) then the changes you just made vanish. This violates the Principle of Least Astonishment.

If I've misunderstood, please correct me. But otherwise please consider changing the design of this. Thanks!

Installation doesn't work due to nodegit issue

I installed via npm install -g gitwalk as directed, but running gitwalk produces

module.js:472
    throw err;
    ^

Error: Cannot find module '../build/Debug/nodegit.node'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/home/adam/.npm-packages/lib/node_modules/gitwalk/node_modules/nodegit/lib/nodegit.js:16:12)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)

This seems to be the exact issue described by nodegit/nodegit#137, and to some extent nodegit/nodegit#521. According to a comment by @maxkorp, it seems that the nodegit folks were intending to make this error message less cryptic, but didn't get round to it yet. Anyway I found that I could fix this via:

cd ~/.npm-packages/lib/node_modules/gitwalk/node_modules/nodegit
npm install

This forced a build from source, and now it's working.

Alternative for GitLab?

A lovely tool. Sadly it works only with GitHub. Anyone who knows an alternative for GitLab or is able to update this tool?