OS X 64-bit compatibility: Cannot send a file descriptor: Invalid argument (22)

[phusion]

OS X 64-bit compatibility: Cannot send a file descriptor: Invalid argument (22)

[FooBarWidget]

From manfred on April 17, 2008 09:03:32

Ah, so it seems sendmsg is returning an EINVAL in MessageChannel.writeFileDescriptor. Which means "The sum
of the iov_len values overflows an ssize_t." on BSD. When I remove setting of the msg_iov, I get "Message too
long".

I'll keep searching and see if I can find a way to solve it. Any help would be appreciated (:

[FooBarWidget]

From honglilai on April 17, 2008 14:26:44

Are you using the default Apache provided by OS X? It's known to be broken - in fact,
the installer warns the user about this - and one should use the one from MacPorts
instead.

[FooBarWidget]

From manfred on April 17, 2008 14:47:29

Yeah, I'm using the default Apache. I just think it might be worth the effort to find out why it's broken. I will try
to compile Passenger against a manually installed Apache to see if that solves the problem.

[FooBarWidget]

From honglilai on April 17, 2008 15:05:12

We've tried to. Though after several days worth of futile attempts, we gave up and
figured that it's more productive to ask people to install Apache from MacPorts. Our
beta testers agreed.

If someone ever finds out why OS X's Apache is broken, we'd really like to know.

Status: Invalid
Labels: OpSys-OSX

[FooBarWidget]

From manfred on April 18, 2008 01:48:32

With a little testing we found out it's a 64-bit problem and is specific to using sendmsg on 64-bit BSD systems.
Fortunately Apache is a Universal binary, so the workaround for now is starting Apache in 32-bit mode: sudo
arch -i386 /usr/sbin/httpd.

I'm going to try to find some time this weekend to write a workaround for the broken sendmsg on BSD.

[FooBarWidget]

From honglilai on April 18, 2008 05:26:55

Status: Accepted
Labels: -Priority-Medium Priority-High Portability

[FooBarWidget]

From honglilai on April 19, 2008 06:04:06

Summary: OS X 64-bit compatibility: Cannot send a file descriptor: Invalid argument (22)

[FooBarWidget]

From honglilai on April 20, 2008 02:50:58

Status: Fixed
Labels: Milestone-1.0.2

[FooBarWidget]

From timcharper on May 02, 2008 07:11:29

I'm also getting this on a 64 bit mac (OSX Server 10.5). Is the workaround still needed?

[FooBarWidget]

From timcharper on May 02, 2008 07:11:42

Using passender 1.0.4

[FooBarWidget]

From timcharper on May 02, 2008 07:14:03

OK - just confirmed that the workaround fixed my problem. I stopped apache, then started it with:

sudo arch -i386 /usr/sbin/httpd

brilliant!

Tim

[FooBarWidget]

From manfred on May 02, 2008 07:14:21

No, this should be fixed >= 1.0.2. Make sure you update the apache config to point to the correct .so and
spawner after update.

[FooBarWidget]

From timcharper on May 02, 2008 10:26:30

I never did install a previous vesion (1.0.4 was it). Here's my error log:

.....
[Fri May 02 10:22:31 2008] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Fri May 02 10:22:31 2008] [notice] Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7l Phusion_Passenger/1.0.4 configured -- resuming normal operations
[45342:ApplicationPoolClientServer.h:394] Cannot send a file descriptor: Invalid argument (22) --- aborting!
[Fri May 02 10:22:31 2008] [notice] seg fault or similar nasty error detected in the parent process
httpd: apr_sockaddr_info_get() failed for cheesegrater.leadmediapartners.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
[Fri May 02 10:22:42 2008] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Fri May 02 10:22:42 2008] [notice] Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7l Phusion_Passenger/1.0.4 configured -- resuming normal operations
[45507:ApplicationPoolClientServer.h:394] Cannot send a file descriptor: Invalid argument (22) --- aborting!
[Fri May 02 10:22:42 2008] [notice] seg fault or similar nasty error detected in the parent process

I'm going to try to recompile w/a different architecture. Running "sudo arch -i386 /usr/sbin/httpd" works for a while, but seams to break when process respawns. Will post more details
as I can clearly articulate them

[FooBarWidget]

From timcharper on May 02, 2008 11:02:13

OK, so here's my theory so far.

I am running a 64 bit machine (a Mac Pro 2x2.66GHz Dual-Core Intel Xeon)

cheesegrater:~ root# arch
i386
cheesegrater:~ root# gem install passenger
ERROR: While executing gem ... (Gem::RemoteFetcher::FetchError)
Gem::RemoteFetcher::FetchError: bad response Not Found 404 reading http://gems.rubyforge.org/gems/passenger-1.0.4.gem cheesegrater:~ root# gem install passenger
Building native extensions. This could take a while...
Successfully installed passenger-1.0.4
1 gem installed
Installing ri documentation for passenger-1.0.4...
Installing RDoc documentation for passenger-1.0.4...
cheesegrater:~ root# passenger-install-apache2-module
Welcome to the Passenger Apache 2 module installer.

This installer will guide you through the entire installation process. It
shouldn't take more than 3 minutes in total.

Here's what you can expect from the installation process:

1. The Apache 2 module will be installed for you.
2. You'll learn how to configure Apache.
3. You'll learn how to deploy a Ruby on Rails application.

Don't worry if anything goes wrong. This installer will advise you on how to
solve any problems.

Press Enter to continue, or Ctrl-C to abort.

Checking for required software...

• GNU C++ compiler... found at /usr/bin/g++
• Ruby development headers... found
• OpenSSL support for Ruby... found
• RubyGems... found
• Rake... found at /usr/bin/rake
• Apache 2... found at /usr/sbin/httpd
• Apache 2 development headers... found at /usr/sbin/apxs
• Apache Portable Runtime (APR) development headers... found at /Developer/SDKs/MacOSX10.5.sdk/usr/bin/apr-1-config

* fastthread... found

---

Compiling and installing Apache 2 module...
cd /Library/Ruby/Gems/1.8/gems/passenger-1.0.4
rake clean apache2
(in /Library/Ruby/Gems/1.8/gems/passenger-1.0.4)

In ext/apache2:

rm -rf Utils.o Logging.o Configuration.o Hooks.o mod_passenger.o mod_passenger.so

In ext/passenger:

make clean
rm -f Makefile

In ext/boost/src:

rm -f libboost_thread.a *.o

In test:

rm -f Apache2ModuleTests *.o

In benchmark:

rm -f DummyRequestHandler

In ext/boost/src:

g++ -O2 -fPIC -I../.. -D_REENTRANT -DNDEBUG -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c *.cpp
ar cru libboost_thread.a *.o
ranlib libboost_thread.a
ranlib: for architecture: ppc file: libboost_thread.a(tss_dll.o) has no symbols
ranlib: for architecture: ppc file: libboost_thread.a(tss_hooks.o) has no symbols
ranlib: for architecture: ppc file: libboost_thread.a(tss_null.o) has no symbols
ranlib: for architecture: ppc file: libboost_thread.a(tss_pe.o) has no symbols
ranlib: for architecture: ppc64 file: libboost_thread.a(tss_dll.o) has no symbols
ranlib: for architecture: ppc64 file: libboost_thread.a(tss_hooks.o) has no symbols
ranlib: for architecture: ppc64 file: libboost_thread.a(tss_null.o) has no symbols
ranlib: for architecture: ppc64 file: libboost_thread.a(tss_pe.o) has no symbols
ranlib: for architecture: i386 file: libboost_thread.a(tss_dll.o) has no symbols
ranlib: for architecture: i386 file: libboost_thread.a(tss_hooks.o) has no symbols
ranlib: for architecture: i386 file: libboost_thread.a(tss_null.o) has no symbols
ranlib: for architecture: i386 file: libboost_thread.a(tss_pe.o) has no symbols
ranlib: for architecture: x86_64 file: libboost_thread.a(tss_dll.o) has no symbols
ranlib: for architecture: x86_64 file: libboost_thread.a(tss_hooks.o) has no symbols
ranlib: for architecture: x86_64 file: libboost_thread.a(tss_null.o) has no symbols
ranlib: for architecture: x86_64 file: libboost_thread.a(tss_pe.o) has no symbols

In ext/apache2:

g++ -I.. -fPIC -g -DPASSENGER_DEBUG -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I/usr/include/apr-1 -I/usr/include/apache2 -D_REENTRANT -Wall -g -
I/usr/local/include -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c mod_passenger.c

In ext/apache2:

g++ -I.. -fPIC -g -DPASSENGER_DEBUG -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I/usr/include/apr-1 -I/usr/include/apache2 -D_REENTRANT -Wall -g -
I/usr/local/include -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c Utils.cpp

In ext/apache2:

g++ -I.. -fPIC -g -DPASSENGER_DEBUG -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I/usr/include/apr-1 -I/usr/include/apache2 -D_REENTRANT -Wall -g -
I/usr/local/include -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c Logging.cpp

In ext/apache2:

g++ -I.. -fPIC -g -DPASSENGER_DEBUG -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I/usr/include/apr-1 -I/usr/include/apache2 -D_REENTRANT -Wall -g -
I/usr/local/include -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c Configuration.cpp

In ext/apache2:

g++ -I.. -fPIC -g -DPASSENGER_DEBUG -DDARWIN -DSIGPROCMASK_SETS_THREAD_MASK -no-cpp-precomp -I/usr/include/apr-1 -I/usr/include/apache2 -D_REENTRANT -Wall -g -
I/usr/local/include -arch ppc7400 -arch ppc64 -arch i386 -arch x86_64 -c Hooks.cpp

In ext/apache2:

g++ -flat_namespace -bundle -undefined dynamic_lookup Utils.o Logging.o Configuration.o Hooks.o mod_passenger.o -fPIC -o mod_passenger.so -arch ppc7400 -arch ppc64 -arch i386 -arch
x86_64 -lstdc++ -lpthread ../boost/src/libboost_thread.a -L/usr/lib -lapr-1

In ext/passenger:

ruby extconf.rb
creating Makefile

In ext/passenger:

make
gcc -I. -I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin9.0 -
I/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/universal-darwin9.0 -I. -fno-common -arch ppc -arch i386 -Os -pipe -fno-common -c native_support.c
cc -arch ppc -arch i386 -pipe -bundle -undefined dynamic_lookup -o native_support.bundle native_support.o -L"." -L"/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib" -L. -
arch ppc -arch i386 -lruby -lpthread -ldl -lm

---

The Apache 2 module was successfully installed.

Please edit your Apache configuration file, and add these lines:

LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-1.0.4/ext/apache2/mod_passenger.so
RailsSpawnServer /Library/Ruby/Gems/1.8/gems/passenger-1.0.4/bin/passenger-spawn-server
RailsRuby /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

After you restart Apache, you are ready to deploy any number of Ruby on Rails
applications on Apache, without any further Ruby on Rails-specific
configuration!

Press ENTER to continue.

---

Deploying a Ruby on Rails application: an example

Suppose you have a Ruby on Rails application in /somewhere. Add a virtual host
to your Apache configuration file, and set its DocumentRoot to
/somewhere/public, like this:

<VirtualHost *:80>
ServerName www.yourhost...

[FooBarWidget]

From timcharper on May 02, 2008 11:02:13

....com
DocumentRoot /somewhere/public

And that's it! You may also want to check the Users Guide for security and
optimization tips and other useful information:

/Library/Ruby/Gems/1.8/gems/passenger-1.0.4/doc/Users guide.html

Enjoy Passenger, a product of Phusion (www.phusion.nl) :-) http://www.modrails.com/ ## here is my module config file ##
cheesegrater:~ root# cat /etc/apache2/conf.d/mod_rails.conf

LoadModule passenger_module /Library/Ruby/Gems/1.8/gems/passenger-1.0.4/ext/apache2/mod_passenger.so
RailsSpawnServer /Library/Ruby/Gems/1.8/gems/passenger-1.0.4/bin/passenger-spawn-server
RailsRuby /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby

cheesegrater:~ root# arch -x86_64 /usr/sbin/httpd
httpd: apr_sockaddr_info_get() failed for cheesegrater.leadmediapartners.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName

hit mod_rails app - failure

cheesegrater:~ root# tail -n 5 /var/log/apache2/error_log
[Fri May 02 10:50:37 2008] [notice] seg fault or similar nasty error detected in the parent process
[Fri May 02 10:57:09 2008] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Fri May 02 10:57:09 2008] [notice] Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7l Phusion_Passenger/1.0.4 configured -- resuming normal operations
[49303:ApplicationPoolClientServer.h:394] Cannot send a file descriptor: Invalid argument (22) --- aborting!
[Fri May 02 10:57:09 2008] [notice] seg fault or similar nasty error detected in the parent process
cheesegrater:~ root# date
Fri May 2 10:57:58 MST 2008
cheesegrater:~ root# killall httpd
cheesegrater:~ root# arch -i386 /usr/sbin/httpd
httpd: apr_sockaddr_info_get() failed for cheesegrater.leadmediapartners.com
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName

hit mod_rails app - it worked!

cheesegrater:~ root# tail -n 5 /var/log/apache2/error_log
[Fri May 02 10:57:09 2008] [notice] Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7l Phusion_Passenger/1.0.4 configured -- resuming normal operations
[49303:ApplicationPoolClientServer.h:394] Cannot send a file descriptor: Invalid argument (22) --- aborting!
[Fri May 02 10:57:09 2008] [notice] seg fault or similar nasty error detected in the parent process
[Fri May 02 10:58:16 2008] [warn] pid file /var/run/httpd.pid overwritten -- Unclean shutdown of previous Apache run?
[Fri May 02 10:58:16 2008] [notice] Apache/2.2.6 (Unix) mod_ssl/2.2.6 OpenSSL/0.9.7l Phusion_Passenger/1.0.4 configured -- resuming normal operations
cheesegrater:~ root# tail -n 5 /var/log/apache2/access_log
::1 - - [02/May/2008:10:21:06 -0700] "GET /images/rails.png HTTP/1.1" 200 1787
::1 - - [02/May/2008:10:21:11 -0700] "GET /test_controller HTTP/1.1" 200 11592
::1 - - [02/May/2008:10:44:01 -0700] "GET / HTTP/1.1" 200 7557
::1 - - [02/May/2008:10:44:26 -0700] "GET / HTTP/1.0" 200 7557
::1 - - [02/May/2008:10:58:17 -0700] "GET / HTTP/1.1" 200 7557

[FooBarWidget]

From timcharper on May 02, 2008 11:17:49

Something else interesting:

I forced the passenger module to compile as x64_86 by changing line 166 in platform_info.rb:

    def self.determine_multi_arch_flags
            if RUBY_PLATFORM =~ /darwin/ && !HTTPD.nil?
                    architectures = []
                    `file "#{HTTPD}"`.split("\n").grep(/for architecture/).each do |line|
                            line =~ /for architecture (.*?)\)/
                            architectures << "-arch #{$1}"
                    end
                    return "-arch x86_64" # architectures.join(' ')
            else
                    return ""
            end
    end

Starting apache as x86_64 booted, but seg-faulted. Starting apache as i386 wouldn't even start, complaining:
cheesegrater:passenger-1.0.4 root# arch -i386 /usr/sbin/httpd
httpd: Syntax error on line 1450 of /private/etc/apache2/httpd.conf: Syntax error on line 1 of
/etc/apache2/conf.d/mod_rails.conf: Cannot load /Library/Ruby/Gems/1.8/gems/passenger-
1.0.4/ext/apache2/mod_passenger.so into server: dlopen(/Library/Ruby/Gems/1.8/gems/passenger-
1.0.4/ext/apache2/mod_passenger.so, 10): no suitable image found. Did
find:\n\t/Library/Ruby/Gems/1.8/gems/passenger-1.0.4/ext/apache2/mod_passenger.so: mach-o, but
wrong architecture

.....

Doing the reverse (forcing passenger to compile 32 bit only) yield the opposite results, but of course running
apache as i386 works, whereas running as x86_64 complains about the passenger module being the wrong
architecture.

cheesegrater:passenger-1.0.4 root# apachectl -v
Server version: Apache/2.2.6 (Unix)
Server built: Sep 23 2007 18:07:19

[FooBarWidget]

From timcharper on May 02, 2008 11:33:04

OK: I just installed passenger-1.0.3, and everything is working just fine. It appears to be a regression with
passenger-1.0.4.

I suspect this is the change that broke OSX again: http://github.com/FooBarWidget/passenger/commit/af4a8e13209f31dfd59914261266a4427a1942c6 thanks!
Tim

[FooBarWidget]

From bill.horsman on May 08, 2008 01:56:11

For the record, I experienced the same issue on a MacBook Air running Mac OS 10.5.2
and Passenger 1.0.4. When I installed Passenger 1.0.4 it worked fine. Thanks for the
workaround, Tim.

• Bill

[FooBarWidget]

From honglilai on May 08, 2008 02:00:01

This issue has been solved in 1.0.5.

[FooBarWidget]

From bill.horsman on May 08, 2008 02:15:26

honglilai: great.

Of course, my comment above should have read "When I installed Passenger 1.0.3 it
worked fine."

[FooBarWidget]

From [email protected] on June 15, 2008 01:31:59

I've just started using Passenger 1.0.5 - received this:

[Sun Jun 15 02:53:46 2008] [warn] child process 23969 still did not exit, sending a SIGTERM
[Sun Jun 15 02:53:47 2008] [notice] SIGHUP received. Attempting to restart
[29275:ApplicationPoolClientServer.h:394] Cannot send a file descriptor: Broken pipe (32) --- aborting!
[Sun Jun 15 02:53:47 2008] [notice] seg fault or similar nasty error detected in the parent process

uname -a

Linux servername 2.6.9-67.0.4.ELsmp #1 SMP Fri Jan 18 05:00:58 EST 2008 i686 i686 i386 GNU/Linux

httpd -v
Server version: Apache/2.0.52
Server built: Jan 7 2008 10:03:05

ldd /opt/ruby/current/lib/ruby/gems/1.8/gems/passenger-1.0.5/ext/apache2/mod_passenger.so
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00c04000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x001fa000)
libapr-0.so.0 => /usr/lib/libapr-0.so.0 (0x00ba0000)
libm.so.6 => /lib/tls/libm.so.6 (0x00e98000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00aef000)
libc.so.6 => /lib/tls/libc.so.6 (0x0020c000)
/lib/ld-linux.so.2 (0x00b5e000)
librt.so.1 => /lib/tls/librt.so.1 (0x00111000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00125000)
libdl.so.2 => /lib/libdl.so.2 (0x007b7000)

The whole httpd process went down and needed manual restart.

This was within 24 hours of starting to use Passenger on a lightly trafficked site (10 hits / minute on average for the period in question)

[FooBarWidget]

From honglilai on June 15, 2008 01:52:33

Could you try version 2.0 RC 1? That should be much more stable than the 1.0.x series.

[FooBarWidget]

From joshua.nichols on June 15, 2008 11:35:56

I'm seeing this problem with 2.0 rc1 as well.

[FooBarWidget]

From honglilai on June 15, 2008 12:14:45

What is the error message when on 2.0 RC 1?

[FooBarWidget]

From balupton on January 05, 2011 20:12:28

I'm getting this error in the latest version, on OSX 10.6 x64, with Zend Server CE 5.0.4 PHP5.3 release http://www.zend.com/en/products/server-ce/downloads

[FooBarWidget]

From balupton on January 05, 2011 20:12:50

Here is the exact error when starting apache:

httpd: Syntax error on line 422 of /usr/local/zend/apache2/conf/httpd.conf: Cannot load /Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so into server: dlopen(/Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so, 10): no suitable image found. Did find:\n\t/Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so: mach-o, but wrong architecture
/usr/local/zend/bin/apachectl stop [FAILED]
httpd: Syntax error on line 422 of /usr/local/zend/apache2/conf/httpd.conf: Cannot load /Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so into server: dlopen(/Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so, 10): no suitable image found. Did find:\n\t/Library/Ruby/Gems/1.8/gems/passenger-3.0.1/ext/apache2/mod_passenger.so: mach-o, but wrong architecture
/usr/local/zend/bin/apachectl start [FAILED]