jacebrowning/gitman

Language-agnostic dependency manager using Git.

Overview

Gitman is a language-agnostic dependency manager using Git. It aims to serve as a submodules replacement and provides advanced options for managing versions of nested Git repositories.

Setup

Requirements

• Python 3.8+
• Git 2.8+ (with stored credentials)

Installation

Install this tool globally with pipx (or pip):

$ pipx install gitman

or add it to your Poetry project:

$ poetry add gitman

Configuration

Generate a sample config file:

$ gitman init

or manually create one (gitman.yml or .gitman.yml) in the root of your working tree:

location: vendor/gitman

sources:
  - repo: "https://github.com/kstenerud/iOS-Universal-Framework"
    name: framework
    rev: Mk5-end-of-life
  - repo: "https://github.com/jonreid/XcodeCoverage"
    name: coverage
    links:
      - target: Tools/XcodeCoverage
  - repo: "https://github.com/dxa4481/truffleHog"
    name: trufflehog
    rev: master
    scripts:
      - chmod a+x truffleHog/truffleHog.py
  - repo: "https://github.com/FortAwesome/Font-Awesome"
    name: fontawesome
    rev: master
    sparse_paths:
      - "webfonts/*"
  - repo: "https://github.com/google/material-design-icons"
    name: material-design-icons
    rev: master

groups:
  - name: code
    members:
      - framework
      - trufflehog
  - name: resources
    members:
      - fontawesome
      - material-design-icons

default_group: code

Ignore the dependency storage location:

$ echo vendor/gitman >> .gitignore

Usage

See the available commands:

$ gitman --help

Updating Dependencies

Get the latest versions of all dependencies:

$ gitman update

which will essentially:

1. Create a working tree at <root>/<location>/<name>
2. Fetch from repo and checkout the specified rev
3. Symbolically link each <location>/<name> from <root>/<link> (if specified)
4. Repeat for all nested working trees containing a config file
5. Record the actual commit SHAs that were checked out (with --lock option)
6. Run optional post-install scripts for each dependency

where rev can be:

• all or part of a commit SHA: 123def
• a tag: v1.0
• a branch: main
• a rev-parse date: 'main@{2015-06-18 10:30:59}'

Alternatively, get the latest versions of specific dependencies:

$ gitman update framework

or named groups:

$ gitman update resources

Restoring Previous Versions

Display the versions that are currently installed:

$ gitman list

Reinstall these specific versions at a later time:

$ gitman install

Deleting Dependencies

Remove all installed dependencies:

$ gitman uninstall

Resources

• Source code
• Issue tracker
• Release history

gitman's Issues

Automatically ignore the sources location

Potentially this is part of $ gdm init and/or a warning when running $ gdm install if the directory is not ignored.

Consider renaming PyPI to 'git-dependency-manager'

This would increase visibility in searches and reduce confusion around https://en.wikipedia.org/wiki/GNOME_Display_Manager.

I would still like the CLI to be gdm and git-deps (and perhaps git-dependency-manager as well).

Display 'git status' output when '--force' is required

Document command-line interface

http://git-dependency-manager.info/interfaces/cli/

Add a website

Install as a Git plugin

If this can't be done automatically by setuptools, just add an --install-git-plugin [path] argument. The plugin should be available as $ git deps. A --force option should be required if the plugin file already exists.

Add 'gdm edit' to launch the config file

Add a 'lock' command

Make:

gdm update --no-lock
gdm lock [foo bar]

equivalent to the current:

gdm update [foo bar]

Add better Git credential manager documentation

Create a separate page on the website to describe stored credentials with tips for dealing with TFA (http://olivierlacan.com/posts/why-is-git-https-not-working-on-github/).

Add a command to display revs

It would be useful in scripting scenarios to be able to run something like:

$ gitman --get-rev <name>
v1.0

to write the rev value to standard out.

`gdm install` hangs

Running it my machine just hangs while cloning repos. You should add a -v | --verbose flag to show std_out/std_err

Support individual source selection

If I have two entries "a" and "b" in gdm.yml under sources, and I run gdm update, I will get the latest rev of both "a" and "b".

I would like to be able to call gdm update a and only update "a", leaving "b" untouched.

Could also work for gdm install

Replace website with mkdocs

http://www.mkdocs.org/

Include:

• Add link to full docs from the readme
• Git plugin
• Scenarios from http://jacebrowning.info/2015/08/07/replacing-submodules-with-gdm/
  • Replace submodules
  • Related features
  • Using links
• Add more fields from: http://www.mkdocs.org/user-guide/configuration/

Add a way to install a specific dependency version from the CLI

Something like:

• gitman install foo@0bcd85a
• gitman install foo -r 0bcd85a
• gitman install foo:0bcd85a
• gitman install foo --at 0bcd85a

Allow checkout using rev-parse

http://stackoverflow.com/a/6990682/429533

Currently, this doesn't work:

$ gdm install
...
fatal: Invalid refspec 'develop@{2015-06-07 18:30:00}'

Split up install and update

Where, update has the current behavior and install will use the last known SHA.

Store version in the config to report incompatibility

In gitman.yml:

...
version: 0.3.1

If there is an incompatibility in:

• major: require update
• minor/micro: warn about update

Require an option to lock sources

I'd like the user to be more explicit about when they want gdm.yml to change.

Make $ gdm update behave like $ gdm update --no-lock by default and add $ gdm update --lock as a new option.

Once a source as been locked, it's version will continue to be recorded when $ gdm update is run.

`gdm install` shouldn't bump version numbers

After running gdm install I noticed that the version changed even though the dependency was already installed. Seems like only upgrade should do this.

Document branch tracking use case

http://git-dependency-manager.info/use-cases/branches/

"gdm install" fails with "remote origin already exists" error

gdm install always fails with fatal: remote origin already exists error.

$ gdm --version
GDM v0.6

$ cat gdm.yml
location: deps
sources:
- dir: <MY-DEPENDENCY>
  link: lib
  repo: <MY-DEPENDENCY-GIT-URL>
  rev: master

$ gdm install
Installing dependencies...

$ mkdir -p /home/admin/tmp/deps
$ cd /home/admin/tmp/deps

  $ git clone <MY-DEPENDENCY-GIT-URL>
  $ cd <MY-DEPENDENCY>
  $ git remote add origin <MY-DEPENDENCY-GIT-URL>

  IN: '/home/admin/tmp/deps/<MY-DEPENDENCY>'

  RAN: '/usr/bin/git remote add origin <MY-DEPENDENCY-GIT-URL>'

  STDOUT:


  STDERR:
fatal: remote origin already exists.

The gdm install command adds the remote origin. In cases where the remote origin is already present, gdm tries (ignores on failures) to remove the remote and adds it back. The actual command to remote an origin is git remote rm origin, but gdm uses git remote remove origin.

I guess this should have been introduced in commit SHA: 27e5d8b where git fetch was replaced with git remote commands.

Let me know if you need more information.

I will send in a patch (via pull request) with 'remove' changed to 'rm', shortly. I don't see the above error with this patch and gdm removes/adds remotes as expected.

Document feature linking use case

http://git-dependency-manager.info/use-cases/linked-features/

Support caching repositories to save time/space

http://randyfay.com/content/reference-cache-repositories-speed-clones-git-clone-reference

Only fetch if needed

When tracking a SHA, only run git fetch if the ref is missing locally.

Display shell commands on 'list'

Rather than just display long lines of information, show the commands of cding into each directory and asking Git for the SHA.

Add a '--recursive' option to 'update'

And make non-recursive the default. In most cases, only update the immediate dependencies is desired.

Add a Homebrew installation option

http://jimkubicek.com/blog/2015/02/14/creating-a-homebrew-formula-for-a-python-project/

Add infinite recursion detection

For simplicity of implementation, the --depth=<N> option could simply have a default of 5 or so.

Automatically set upstream branches

Create an uninstall command

This will simply delete the top location directory.

Return error instead of locking <unknown>

When git fails to pull something, it acts weird.

Display shell commands on 'uninstall'

This will mirror the behavior of 'install' and emphasis that GDM isn't doing anything special that couldn't be done normally with shell commands.

Add to StackShare

http://stackshare.io/submit

Document Git plugin

http://git-dependency-manager.info/interfaces/plugin/

Checkout branches when possible

Even when tracking a SHA, first try checking out the branch to see if the current head matches the desired ref. This will leave the nested repository in a better state for ongoing development.

Local variable 'path' referenced before assignment

gdm uninstall --quiet
Traceback (most recent call last):
  File "/usr/local/bin/gdm", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.4/site-packages/gdm/cli.py", line 98, in main
    success = function(**kwargs)
  File "/usr/local/lib/python3.4/site-packages/gdm/commands.py", line 82, in delete
    for _ in config.get_deps(allow_dirty=force):
  File "/usr/local/lib/python3.4/site-packages/gdm/config.py", line 211, in get_deps
    yield source.identify(allow_dirty=allow_dirty)
  File "/usr/local/lib/python3.4/site-packages/gdm/config.py", line 109, in identify
    return path, "<missing>", "<unknown>"
UnboundLocalError: local variable 'path' referenced before assignment

Support multiple links

Change the link property to links (and provide migration) to support creating multiple symlinks into different subdirectories of the dependency.

---

• Wait for #146

'--quiet' is not respected, shell commands still display

'--quiet' should suppress warnings

$ gdm install --quiet
WARNING: added missing nested key 'link'...

Simplify config format

Move from:

location: gdm_sources
sources:
- dir: api
  link: ''
  repo: https://github.com/example/api
  rev: develop
sources_locked:
- dir: api
  link: ''
  repo: https://github.com/example/api
  rev: b2730855c9efaaa7448b25b82e5a4363785c83ed

to

location: gdm_sources
sources:
- name: api
  repository:
    url: https://github.com/example/api
    rev:
      default: develop
      locked: b2730855c9efaaa7448b25b82e5a4363785c83ed
  link: foo/bar

where rev_locked is optional (along with link).

Document the reasons alternatives are insufficient

Support link creation

Add a '--depth' option to limit nesting traversal

Examples:

• depth == None (default): normal behavior
• depth == 1: Only get immediate dependencies
• depth == 2: Get immediate dependencies and each dependency's immediate dependencies

Add a '--no-clean' option

The default is to clean all untracked/ignored files before checkout. This option would disable that.

With '--verbose', show the commits that were updated

Make '--no-clean' the default

And add a '--clean' option

Require '--force' with 'uninstall' when changes

Add a '--no-lock' option to 'update'

In cases were 'update' is the only command ever used, we don't need GDM writing changes to lock source versions.

Only lock previously locked sources

If a project has two dependencies, foo and bar, but only foo has a locked version, $ gdm update should update both to the latest, but only record the version of foo.

Require '--force' to clean/reset changes?