binary C extension not built?

[noraj]

binary C extension not built?

[noraj]

Ok found the issue, it related to ArchLinux build.

There is a patch to disable to copy of the .so from ext to lib: https://gitlab.archlinux.org/archlinux/packaging/packages/rubygems/-/blob/main/rubygems_stop_so_duplication.patch?ref_type=heads

Upstream issue https://bugs.archlinux.org/task/70959

[boazsegev]

I see no error while install, is the C building failing silently?

Most probably, as I cannot see any reason for the error. The CI on Ruby 3.0 fails with a bundler error stating:

ERROR:  Error installing /home/runner/work/iodine/iodine/pkg/iodine-0.7.56.gem:
	ERROR: Failed to build gem native extension.

    No such file or directory @ dir_s_mkdir - /home/runner/work/iodine/iodine/vendor/bundle/ruby/3.0.0/gems/iodine-0.7.56/ext/iodine/.gem.20230707-1798-pdmzvu

I am not sure why your version fails silently, but there's a bundler issue somewhere that I haven't looked into working around quite yet.

Once I upgraded to Ruby 3.1.3p185, my build issue was resolved. I don't know if that will solve your issue.

Sorry about that.

[noraj]

ArchLinux version of ruby has been quite outdated for a long time https://archlinux.org/packages/extra/x86_64/ruby/, they don't seem to care much about it.

If I want to use a more recent version of Ruby I usually use asdf-vm on my dev machines. But on my desktop I wanted to avoid installing dev tools for virtual environments and use only the system ruby.

Based on what you told me, installing asdf-ruby + ruby 3.2+ will probably workaround this compilation issue that appears only on 3.0-3.1.3.

[noraj]

gem install iodine --no-document --backtrace -V 9 | tee /tmp/iodine-install.log on ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]

iodine-install.log ❌

gem install iodine --no-document --backtrace -V 9 | tee /tmp/iodine-install-3.2.log on ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]

iodine-install-3.2.log ✅

[janbiedermann]

@noraj in both logs it says "successfully installed", whats the issue?

[janbiedermann]

In addition, in both logs it links iodine.so correctly and installs it without error.

Now, if iodine.so doesn't get loaded, that may be possible because of a name clash in the load path, where it finds iodine.rb first and thus doesn't load iodine.so. Not sure if that is the case here, but that happened to me once, thats why (@boazsegev) i renamed iodine.so to iodine_ext.so in isomorfeus-iodine.

@noraj you can verify that by patching the gem in its path, possibly copying the built extension in its place to iodine_ext.so and tinkering a bit in irb to see if it gets loaded.

[noraj]

@noraj in both logs it says "successfully installed", whats the issue?

The issue is what is explained in the original message (#148 (comment)), the gem is installed successfully but the iodine C binary is not built.

You can see both log file have very different size.

➜ lsd -lh iodine-install-3.2.log iodine-install.log
.rw-r--r-- noraj noraj 194 KB Sun Sep  3 18:17:48 2023  iodine-install-3.2.log
.rw-r--r-- noraj noraj  29 KB Sun Sep  3 17:52:08 2023  iodine-install.log

➜ wc iodine-install-3.2.log iodine-install.log     
  3749  20837 198640 iodine-install-3.2.log
   483   2146  29918 iodine-install.log

If you diff both, you'll see there are tons of stuff being build for 3.2 that is not for 3.0.

➜ diff iodine-install.log iodine-install-3.2.log | bat

[janbiedermann]

@noraj i see that, if i inspect the files, looking fo "compiling " i see 32 hits in both of them, i see the *.c *.h files are copied correctly.
The difference in log file size is either because of different compiler versions being used or different build options, one printing more warnings, one printing less warnings. I can see no difference in compiled files and i can find no errors. Its built correctly in both cases.

[janbiedermann]

So the issue seems to be, that it cannot find the built extension, see above.

[janbiedermann]

And sure, you cannot use the gem built for 3.0 with 3.2 or the other way around, if you switch rubies, are the gem paths updated accordingly? Otherwise load will fail.

[noraj]

So it's something from the ruby environment.

3.0 being archlinux system ruby with gem installed as user.
3.2 being a asdf-vm shim env.

[janbiedermann]

Then check $LOAD_PATH, see if /home/noraj/.local/share/gem/ruby/3.0.0/gems/
is in there on archlinux

[janbiedermann]

Oh yes, it tries to load from it, so i'm back on this possibly being a name clash, as described above (@boazsegev)

[boazsegev]

Hi @janbiedermann and @noraj ,

Thank you both for hunkering down on this one. I wasn't sure what the root cause was until @janbiedermann brought it up.

I released a patch that names the library file to iodine_ext, avoiding the possible filename conflict.

This patch doesn't solve the CI tests failing on Ruby 3+, but they might solve this issue.

Thanks again and please keep me posted so I know if this help.

Cheers,
Bo.

[noraj]

Can someone try with a suer install?

gem install --user-install iodine or this in /etc/gemrc

# Read about the gemrc format at http://guides.rubygems.org/command-reference/#gem-environment

# --user-install is used to install to $HOME/.gem/ by default since we want to separate
#                system (package manager) installed gems and gem installed gems
gem: --user-install

In your .bashrc|.zshrc:

export GEM_HOME="$(gem env user_gemhome)"
export PATH="$PATH:$GEM_HOME/bin"

Maybe it's a path issue when you use --user-install

[janbiedermann]

@noraj but it tries to load from the right path
it says "no such file or directory"
please do
ls -l /home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine
ls -l /home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/

[noraj]

~ 
➜ ls -l /home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/
ls: cannot access '/home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/': No such file or directory

~ 
➜  ls -l /home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine
ls: cannot access '/home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine': No such file or directory

~ 
➜ ls -l /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine/     
total 32
-rw-r--r-- 1 noraj noraj 2226 sept.  4 11:40 connection.rb
-rw-r--r-- 1 noraj noraj 2012 sept.  4 11:40 json.rb
-rw-r--r-- 1 noraj noraj 5309 sept.  4 11:40 mustache.rb
-rw-r--r-- 1 noraj noraj 3585 sept.  4 11:40 pubsub.rb
-rw-r--r-- 1 noraj noraj 2234 sept.  4 11:40 rack_utils.rb
-rw-r--r-- 1 noraj noraj  670 sept.  4 11:40 tls.rb
-rw-r--r-- 1 noraj noraj   46 sept.  4 11:40 version.rb

➜ /home/noraj/.gem/ruby/3.0.0/bin/iodine
/home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine.rb:6:in `require_relative': cannot load such file -- /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine/iodine_ext (LoadError)
        from /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine.rb:6:in `<top (required)>'
        from <internal:/usr/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from <internal:/usr/lib/ruby/3.0.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
        from /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/exe/iodine:4:in `<top (required)>'
        from /home/noraj/.gem/ruby/3.0.0/bin/iodine:25:in `load'
        from /home/noraj/.gem/ruby/3.0.0/bin/iodine:25:in `<main>'

[janbiedermann]

Hm, it installs from
/home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.56/ext/iodine
it loads from:
/home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine

and at same time it concludes:
/home/noraj/.local/share/gem/ruby/3.0.0/gems/iodine-0.7.57/lib/ : No such file or directory?

I can only conclude, that your system ruby is broken or your filesystem or there is a symlink somwhere thats broken or something like that.

My recommendation:

• uninstall the system ruby
• install rvm from rvm.io
• use rvm to manage rubies on your system

[noraj]

It's maybe confusing but /home/noraj/.local/share/gem/ruby/3.0.0/ was on another machine (original post). On my previous message the user gem home is /home/noraj/.gem/ruby/3.0.0/. and you can see iodine is trying to load from /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine/iodine_ext so the right path, the issue is there is no iodine_ext when I check ls -l /home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/lib/iodine/ . The original message was the same issue on another machine where the user gem home was /home/noraj/.local/share/gem/ruby/3.0.0. I don't think my system ruby is broken, I have user other native extension gem. And yeah I already worked around it by using asdf-vm (rvm is dangerous) and installing ruby 3.2. I'll try installing a ruby 3.0 on asdf to see if I have the same issue.

[noraj]

Also the C extension is built:

➜ find /home/noraj/.gem/ruby/3.0.0 -iname 'iodine_ext*'
/home/noraj/.gem/ruby/3.0.0/extensions/x86_64-linux/3.0.0/iodine-0.7.57/iodine/iodine_ext.so
/home/noraj/.gem/ruby/3.0.0/gems/iodine-0.7.57/ext/iodine/iodine_ext.so

The require tries gems/iodine-0.7.57/lib/iodine/iodine_ext instead of gems/iodine-0.7.57/ext/iodine/iodine_ext

[noraj]

On asdf ruby 3.2

➜ find /home/noraj/.asdf/installs/ruby/3.2.0/ -iname 'iodine_ext*'
/home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/extensions/x86_64-linux/3.2.0/iodine-0.7.57/iodine/iodine_ext.so
/home/noraj/.asdf/installs/ruby/3.2.0/lib/ruby/gems/3.2.0/gems/iodine-0.7.57/lib/iodine/iodine_ext.so

So in asdf ruby 3.2 tt's put in lib while in system ruby with user install 3.0 it's put in ext. Now I'll try asdf ruby 3.0 to know if the issue is caused by the user install or ruby 3.0 vs 3.2 or something else (weird default at compile time for AL ruby).

[janbiedermann]

install should copy the *.so as final step from ext to lib

[noraj]

Interestingly with asdf ruby 3.1.0 there is the extension on both lib and ext:

➜ find /home/noraj/.asdf/installs/ruby/3.1.0/ -iname 'iodine_ext*'
/home/noraj/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/extensions/x86_64-linux/3.1.0/iodine-0.7.57/iodine/iodine_ext.so
/home/noraj/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/iodine-0.7.57/ext/iodine/iodine_ext.so
/home/noraj/.asdf/installs/ruby/3.1.0/lib/ruby/gems/3.1.0/gems/iodine-0.7.57/lib/iodine/iodine_ext.so

[janbiedermann]

Yes, thats correct, thats how it should be

[janbiedermann]

(@boazsegev glad you renamed the ext, although thats not the problem here, i remember, that i had to make that change, when i build iodine withing the ruby tree for debugging)

[noraj]

Ruby source	Ruby version	ext build	work	install as
ASDF	3.2.0	lib	✅	system
ASDF	3.1.0	lib & ext	✅	system
ASDF	3.0.6	lib & ext	✅	system
ArchLinux	3.0.6p216	ext	❌	user
ASDF	3.0.6	lib & ext	✅	user

So now I can conclude this is not an issue with ruby 3.0 or 3.1, not an issue with the system/user install, but seems to be an issue with ArchLinux ruby. I'll check the probably apply a wrong patch or bad compilation flags.