timabell/gitopolis

Manage multiple git repositories

Gitopolis

Manage multiple git repositories with ease.

• 🤓 -> Run any shell or git command on multiple git repositories.
• 🤓 -> Re-clone all your repos on new machines.
• 🤓 -> Limit actions to custom tags.
• 🤓 -> Easy to remember and use command list (add / exec / clone / tag).
• 🤓 -> A-GPL v3 licensed labour of love ❤️.

Installation

1. Grab the latest release,
2. unzip it
3. put the binary somewhere in your PATH.

I suggest adding a shorter shell alias to save typing. Perhaps gm for git many or gop.

Usage

Initial setup

cd ~/repos/
gitopolis add *

Running shell / git commands in many repos

gitopolis exec -- git pull

Tagging

gitopolis tag some_tag repo1 repo2
gitopolis exec -t some_tag -- git pull

Re-cloning repos on a new machine

mkdir ~/repos/ && cd ~/repos/

wget https://gist.githubusercontent.com/timabell/87add070a8a44db4985586efe380757d/raw/08be5b3c38190eeed4fda0060818fa39f3c67ee3/.gitopolis.toml

gitopolis clone

Using shell aliases

You can't currently use shell aliases because commands are executed directly rather than passing through a shell such as bash/zsh/fish. See #13.

As a workaround you can create git aliases that run arbitrary shell commands. See #80 (comment)

Using chained commands

You can't chain commands together with && or || like this gitopolis exec -- git branch && git pull because the shell (bash etc) will parse the && before it ever gets to gitopolis.

As a workaround you can create git aliases that run arbitrary shell commands. See #80 (comment)

State file

Creates a single simple .gitopolis.toml file that you can edit, read, share with others and copy to other machines.

[[repos]]
path = "gitopolis"
tags = ["tim"]
[repos.remotes.origin]
name = "origin"
url = "[email protected]:timabell/gitopolis.git"

[[repos]]
path = "schema-explorer"
tags = ["tim", "databases"]
[repos.remotes.origin]
name = "origin"
url = "[email protected]:timabell/schema-explorer.git"

[[repos]]
path = "database-diagram-scm"
tags = ["databases"]
[repos.remotes.origin]
name = "origin"
url = "[email protected]:timabell/database-diagram-scm.git"

View as gist.

TOML is a well-supported config markup with parsers for many programming languages.

---

---

The name

Think a metropolis of git repos.

It's a lot to type as a name, but it's nice and unique, and if you use it a lot I suggest you create a shell alias to something shorter.

Why did I create this?

• Wanted to learn more Rust.
• Had a client with many microservices and teams.
• Tried gita but found command layout hard to remember, and didn't like having to install python.
• To help others with their microservices.

Social

Help others discover gitopolis:

Twitter

Tweet about gitopolis

Product Hunt

Vote for gitopolis:

Contributing

Suggestions welcome, particularly adding your experience, problems and ideas to the issues list.

I'm happy for people to open issues that are actually just questions and support queries.

Rough internal design and ambitions can be found at Design.md.

PRs are appreciated but bear in mind I have my own plans and this is a side project for me to learn rust in a useful way, so worth talking to me before investing too much time in anything that might not fit yet. I hope to make this smoother with better CI tests etc. Start by opening an issue with your thoughts, or ping me some other way (I'm easy to find for better or worse).

Builds

• - Continuous integration build.
• - Release build - generates binaries for download from tagged builds.

gitopolis's Issues

Unable to upgrade crates with cargo-edit on rust 1.77.1

See killercup/cargo-edit#411 (comment)

 tim@fox:~/repo/gitopolis  (main*)
$ ./upgrade-crates.sh 
#!/bin/sh -v
set -e # exit on error
cargo update
    Updating crates.io index
cargo install cargo-edit
    Updating crates.io index
     Ignored package `cargo-edit v0.12.2` is already installed, use --force to override
cargo upgrade --incompatible # from cargo-edit
Error: The repo at path /home/tim/.asdf/installs/rust/1.77.1/registry/index/github.com-1ecc6299db9ec823 is unusable due to having an invalid HEAD reference: reference 'refs/heads/master' not found; class=Reference (4); code=NotFound (-3)

multiple tags as arg to list/clone/exec (and or or?)

support for multiple/chained commands?

i'm trying to run

gitopolis exec -- git branch && git pull

it iterates through each repo and executes the git branch part just fine.

but on the last repo in the list, it gives a:

fatal: not a git repository (or any of the parent directories): .git

ci & release jobs not supposed to be running in parallel

read / restore multiple remotes for repo

read & store remote url from repo

• trait interface for talking to git so it can be mocked
• additional test coverage
• read & store url

A way of excluding repos from commands

"Maybe add a way to disable a repo without removing it? So if you need to run a command in a subset you don't have to remember and re-add the one you removed manually."

shell alias support

 tim@max:~/repo                                                                                   
$ gitopolis exec ll                                                                               
🏢 gitopolis> ll                                                                                  
thread 'main' panicked at 'Error running exec ll: Os { code: 2, kind: NotFound, message: "No such 
file or directory" }', src/exec.rs:18:10                                                          
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace                     
 tim@max:~/repo                                                                                   
$ ll                                                                                              
total 348K                                                                                        
drwxr-xr-x   3 tim tim 4.0K Oct  9  2021 aaclock                
...

https://github.com/timabell/dotmatrix/blob/592130dd6aea627076f65d392ca5f24f69059c6c/.aliases#L1

Race condition in end to end tests (random failures)

Running the tests repeatedly throws up strange failures that look like the filesystem isn't in the state it should be.

e.g.

working folder: /tmp/.tmpLTY0QB/2dQgODf0

Unexpected success
command=`"/home/tim/repo/gitopolis/target/debug/gitopolis" "list"`
code=0
stdout="some_git_folder\n"
stderr=""

thread 'list_empty_exit_code_2' panicked at 'Unexpected success
command=`"/home/tim/repo/gitopolis/target/debug/gitopolis" "list"`
code=0
stdout="some_git_folder\n"
stderr=""
', /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:507:5
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/panicking.rs:64:14
   2: core::panicking::panic_display
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/panicking.rs:135:5
   3: assert_cmd::assert::AssertError::panic
             at /home/tim/.asdf/installs/rust/1.67.1/registry/src/github.com-1ecc6299db9ec823/assert_cmd-2.0.10/src/assert.rs:1033:9
   4: core::ops::function::FnOnce::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:507:5
   5: core::result::Result<T,E>::unwrap_or_else
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/result.rs:1504:23
   6: assert_cmd::assert::Assert::failure
             at /home/tim/.asdf/installs/rust/1.67.1/registry/src/github.com-1ecc6299db9ec823/assert_cmd-2.0.10/src/assert.rs:187:9
   7: end_to_end_tests::list_empty_exit_code_2
             at /home/tim/repo/gitopolis/tests/end_to_end_tests.rs:22:2
   8: end_to_end_tests::list_empty_exit_code_2::{{closure}}
             at /home/tim/repo/gitopolis/tests/end_to_end_tests.rs:20:29
   9: core::ops::function::FnOnce::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:507:5
  10: core::ops::function::FnOnce::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:507:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

try out faux for mocking https://docs.rs/faux/

`gitopolis add *` only works on bash, not cmd or powerhell

see PR #121

A bug in as much as the readme says it should work :-)

Quotes missing in display of exec

Quoted arguments are displayed without the quotes when shown for each repo

switch to `toml_edit` parser

https://github.com/ordian/toml_edit

might allow me to get the inline-table format I wanted, and end the battle to get toml-rs to produce the toml layout I would actually like.

https://twitter.com/tshepang_dev/status/1537174750836965376

warning: variant `MacOSX` is never constructed

   Compiling gitopolis v0.0.0-git (/home/tim/repo/gitopolis)                                                           
warning: variant `MacOSX` is never constructed                                                                         
   --> tests/end_to_end_tests.rs:786:2                                                                                 
    |                                                                                                                  
785 | enum OperatingSystem {                                                                                           
    |      --------------- variant in this enum                                                                        
786 |     MacOSX,                                                                                                      
    |     ^^^^^^                                                                                                       
    |                                                                                                                  
    = note: `#[warn(dead_code)]` on by default                                                                         
                                                                                                                       
warning: `gitopolis` (test "end_to_end_tests") generated 1 warning                                                     
    Finished `test` profile [unoptimized + debuginfo] target(s) in 2m 47s                                              
     Running unittests src/lib.rs (target/debug/deps/gitopolis-fa1e7cbf0f05fcfc)                                       
                                                                                                                       
running 2 tests                                                                                                        

Hitting github rate limits - connection reset

This tool is so good and fast (thanks rust), that looping through 15+ repos results in github dropping the connection and the commands failing with "connection reset" (econnreset? don't have it handy).

As a workaround I've got a shell script on my path called git-slow-fetch that looks something like

#/bin/sh -v
set -e
git fetch
echo "pausing to avoid rate limit"
sleep 1.5

and then I run

gitopolis exec -- git slow-fetch

But perhaps gitopolis should be smarter about this somehow, or allow a delay flag?

If anyone else has issues with this please 👍 this issue so I have an idea how many people are affected.

Separate `.gitopolis.toml` location from current working directory to allow running from anywhere

The .gitopolis file is currently generated relative to the binary location. This setup works for users who download the application, place it alongside their Git repositories, and manually update it. However, most developers prefer installing tools via package managers for ease of installation and updates—Homebrew on macOS, winget on Windows, and various package managers like apt or yum on Linux.

The main obstacle preventing Gitopolis from being installed via these package managers is the config file location, which is currently referenced from a non-managed directory. Since Gitopolis should be accessible from anywhere within the PATH, it requires a standardized config file location across all supported systems.

Therefore, I propose standardizing the config file location across the supported platforms. For Windows, I would need some suggestions as I am not very familiar with it:

System	Location
MacOS	~/.config/gitopolis.toml
Linux	~/.config/gitopolis.toml
Windows	%LOCALAPPDATA%\gitopolis.toml

This change would prepare Gitopolis for inclusion in package managers.

Hello

Cool pet project. What an obstacle do you have?

OSX Support

It looks like Gitopolis compiles on OSX but has some tests failing and missing CI stages to produce a workable build for OSX. Im guessing this has not been looked at before. Ill see if i can help with it.

github actions: require ci to pass before building tags

tend to push main and tags at once, chain the ci somehow to prevent shipping a broken build

streamed output (exec + clone)

Use libgit instead of sh commands

Probably not important, but currently just calling git command and capturing output in order to grab urls from config.

https://github.com/rust-lang/git2-rs - got a compile error including this:

tim@max:~/repo/gitopolis(url*)
$ cb
    Updating crates.io index
error: failed to select a version for the requirement `libgit2-sys = "^0.13.4"`
candidate versions found which didn't match: 0.13.2+1.4.2, 0.13.1+1.4.2, 0.13.0+1.4.1, ...
location searched: crates.io index
required by package `git2 v0.14.4`
    ... which satisfies dependency `git2 = "^0.14.4"` of package `gitopolis v0.1.0 (/home/tim/repo/gitopolis)`

finish off command coverage in end_to_end_tests

• clone --tags
• dry for composability and readability

wip in branch tag-args-typo

skip exec gracefully when folder not found

currently panics when one of the folders is missing, would be better to print a message and move on

particularly problematic if clone --tag was used to clone a subset

• update the clone test to not filter on tag when running exec this is done

Inline toml tables for repos tidier state storage

I designed the storage as

[[repos]]
path = "schema-explorer"
remotes = { origin = "https://github.com/timabell/schema-explorer.git" }
tags = ["tim"]

but toml-rs doesn't currently support generating that style for the remotes - see toml-rs/toml-rs#265