Running appraisal specs with Bundler >= 2.4.0 broken

[tisba]

Running appraisal specs with Bundler >= 2.4.0 broken

[tisba]

This is what I have so far.

Minimal acceptance test:

require 'spec_helper'

describe 'Debugging Bundler Issues' do
  it 'does something' do
    build_git_gems %w(omelette)

    build_gemfile <<-Gemfile
      source 'https://rubygems.org'
      gem "omelette", git: "../gems/omelette"
    Gemfile

    run 'bundle install --local --verbose'
  end
end

Trigger the error via bundle _2.5.6_ exec rspec ./spec/acceptance/bundler_debug_spec.rb.

When you then go into tmp/stage and run bundle install --local --verbose you'll get the same error, but also see stderr from bundler:

Retrying `git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63` due to error (2/4): Bundler::Source::Git::GitCommandError Git error: command `git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63` in directory /Users/basti/.gem/ruby/3.3.0/cache/bundler/git/omelette-0232b7510a79059d96d8bcbfb4a82ad436173b63 has failed.
fatal: '/gems/omelette' does not appear to be a git repository
fatal: Could not read from remote repository.
[…]

BINGO!

Here is what happens:

• Bundler is doing git clone --bare --no-hardlinks --quiet --no-tags --depth 1 --single-branch -- file://../gems/omelette […], to clone the git repo of the gem to the bundler cache using a shallow clone.
• BUT the file:// scheme does not permit relative paths (see RFC8089 "The path component represents the absolute path to the file in the file system." and RFC1738).
• The underlying problem is hidden because AcceptanceTestHelpers#run is unfortunately using Kernel#` which only captures standard out. In case of errors when invoking bundler, this is quite useless.

What does that mean?

tl;dr: I think Bundler does not support local git repositories to be defined via Gemfile (or it just broken). The docs are pretty clear though: For gems out of local git repositories you are supposed to do bundle config set local.GEM_NAME /path/to/local/git/repository.

I'll try to dig through the Bundler changelog and PRs to see if this was intentional or not. I suspect that this was an unintended side-effect introduced by rubygems/rubygems#4475. As it was never documented/recommended to use gems from local git repositories in a different way.

IMO, btw: It should either work, or Bundler should raise an error, that local git repos are not supported, instead of trying to feed an invalid file:// URI to git.

[nickcharlton]

I think it's a bundler incompatibility that you're seeing. I haven't been able to pin down exactly what it is, but I was working on this last week.

I can run bundle exec rspec on Ruby 3.3.0 and with:

$ bundle exec bundler --version
Bundler version 2.3.7

…and the tests pass, but different versions cause problems.

In addition, last week I was playing around with a branch I'd worked on ages ago for testing out GitHub Actions and seen the same.

Could you try replicating with one of the combinations (see the GHA branch), and see if that helps you?

[tisba]

That was a good hint, thanks. I was a little lost and couldn't really find a good starting point. I did some very corse search and found 2.3.27 to be working. I poked at some 2.4.x and 2.5.x versions, but was seeing the same error. I can try some more smart bisecting in the next days, if that helps. My guess is that there broken something between 2.3.27 (laest 2.3.x release) and 2.4.0.

Looking at https://github.com/rubygems/rubygems/blob/master/bundler/CHANGELOG.md#240-december-24-2022, I kind of have the suspicion that rubygems/rubygems#4475 might be the reason. Bundler::Source::Git has been changed quite a bit, where the error message is coming from.

🤔

[tisba]

Forgot to mention: I tested bundler versions with this: bundle _2.4.0_ update --bundler && bundle && rspec ./spec/acceptance/appraisals_file_bundler_dsl_compatibility_spec.rb. Since bundler also manages itself, debugging things like that became rather easy :)

[nickcharlton]

Ah, hah, nice! It'd be great if you could spend the time tracking down what might have changed.

Whilst you're doing that, I'd be keep to hear if you have any ideas of what we could do to make these things a bit easier to debug, too.

[nickcharlton]

Oh wow! Great work. Thanks for digging into it.