Install doesn't quite work for python on Linuxbrew

[grpc]

Install doesn't quite work for python on Linuxbrew

[dhermes]

In total I'm needing to do (on GCE)

$ sudo apt-get update
$ sudo apt-get install -y python-dev python-pip m4 ruby git
$ sudo pip install --upgrade pip virtualenv  # pip in debian repo is WAAAY old
$ git clone https://github.com/Homebrew/linuxbrew.git ~/.linuxbrew
$ export PATH="$HOME/.linuxbrew/bin:$PATH"
$ export MANPATH="$HOME/.linuxbrew/share/man:$MANPATH"
$ export INFOPATH="$HOME/.linuxbrew/share/info:$INFOPATH"
$ virtualenv venv
$ source venv/bin/activate
$ curl -fsSL https://goo.gl/getgrpc | bash -s python

[stanley-cheung]

Hi,
I am able to reproduce the issue. m4 is missing. And it causes some dependency to fail to build.
But m4 is listed as one of the dependency for linuxbrew itself so we didn't list it explicitly in our README.

Here's the minimal list of command I needed to make this work on GCE

sudo apt-get update
sudo apt-get install -y git python-dev python-pip m4 ruby
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/linuxbrew/go/install)" </dev/null
export PATH="$HOME/.linuxbrew/bin:$PATH"
export MANPATH="$HOME/.linuxbrew/share/man:$MANPATH"
export INFOPATH="$HOME/.linuxbrew/share/info:$INFOPATH"
sudo pip install virtualenv
virtualenv venv
source venv/bin/activate
curl -fsSL https://goo.gl/getgrpc | bash -s python

[dhermes]

Indeed :)

AFAICT the only difference is you installing homebrew with a script instead of a git checkout and then

sudo pip install --upgrade pip virtualenv  # pip in debian repo is WAAAY old
# vs.
sudo pip install virtualenv

[stanley-cheung]

I added the m4 dependency explicitly to the google-protobuf formula anyway. Tested that, and can remove the apt-get m4 in my command set.

[dhermes]

Great!

Do you guys have a tracking issue for getting https://github.com/google/protobuf to build in a more reasonable way (i.e. via pip install protobuf)?

[stanley-cheung]

Actually, looks like PR #33 doesn't actually work. There apparently doesn't exist a formula called "m4". For some reason my local testing works but after this gets merged to master it doesn't work.

Will look more into this

[dhermes]

@stanley-cheung m4 is in homebrew/dupes, so you need to tap it. (UPDATE: I see you already noted this in #34)

But you should ditch brew on Linux and m4 ships with OS X, so it's moot-ish

[dhermes]

@stanley-cheung While doing some homebrew sleuthing I discovered that dupes is tapped AFTER unzip is installed. This causes it to fail, which in turn causes grpc/grpc/google-protobuf to fail.

UPDATE: Address in #35

[stanley-cheung]

Are we all good here?

[dhermes]

As far as installing with Linuxbrew, the answer is "kind of" yes. But the requirement that pip install work without root will throw people off.

The instructions/docs about using a virtualenv and installing prerequisites likely need some work too.

[stanley-cheung]

@dhermes Please take a look at #41. Thanks!

[dhermes]

@stanley-cheung I just tried to install on a fresh GCE instance with Ubuntu 14.04 and it failed.

First zlib was a dependency of openssl that was not available (it's in homebrew-dupes).

Then, glibtoolize was not available to autoconf (I did not resolve this and just moved on).

[stanley-cheung]

hm I can reproduce this

==> Installing google-protobuf
==> Downloading https://github.com/google/protobuf/archive/v3.0.0-alpha-3.tar.gz
==> Downloading from https://codeload.github.com/google/protobuf/tar.gz/v3.0.0-alpha-3
######################################################################## 100.0%
==> ./autogen.sh
           s/RuntimeLibrary="4"/RuntimeLibrary="2"/g; gtest/msvc/gtest-md.vcproj gtest/msvc/gtest.vcproj gtest/msvc/gtest_main-md.vcproj gtest/msvc/gtest_main.vcproj gtest/msvc/gtest_prod_test-md.vcproj gtest/msvc/gtest_prod_test.vcproj gtest/msvc/gtest_unittest-md.vcproj gtest/msvc/gtest_unittest.vcproj
+ autoreconf -f -i -Wall,no-obsolete
Can't exec "glibtoolize": No such file or directory at /home/stanleycheung/.linuxbrew/Cellar/autoconf/2.69/share/autoconf/Autom4te/FileUtils.pm line 345, <GEN6> line 6.
autoreconf: failed to run glibtoolize: No such file or directory
autoreconf: glibtoolize is needed because this package uses Libtool

I am almost certain this is something new. I did the exact same steps last week and it worked then. Will investigate more.

[stanley-cheung]

These 2 recent linuxbrew issues/PRs may have broken our formulas:

• Linuxbrew/legacy-linuxbrew#447, openssl now depends on zlib
• Linuxbrew/legacy-linuxbrew#445, libtool now doesn't build their binaries as glibtoolize, but autoconf is still expecting it

[dhermes]

(Added after original posting): Is it worth fighting linuxbrew when it isn't a standard path for

• Linux users
• Homebrew maintainers

---

@stanley-cheung I'm firmly in the camp that homebrew should not be used on Linux and should only be used for dependencies on OS X.

Once support for NPN is in, the argument for using homebrew for the shadowed OpenSSL is no longer necessary, yes?

I realize this repo is called grpc/homebrew-grpc. Is there some other place to file an issue about dropping linuxbrew?

[stanley-cheung]

@tbetbetbe Any comment?

[stanley-cheung]

Status update:

• The issue reported in this comment has been resolved after I submitted a PR to upstream linuxbrew, and another PR #42 in this repo. Please verify.
• I have tested a fresh install of gRPC Python and PHP on Mac and Linux. They were successful
• Turns out installing homebrew means checking out from upstream master. This means our formula could be susceptible to unstable changes. Homebrew does not seem to have releases.
• But in any case, homebrew/linuxbrew will be just a stop-gap intermediate solution for us. Currently the reason why we have homebrew/linuxbrew as a packaging solution for now is because of our dependency on the latest version of openssl (for ALPN support). gRPC is working on removing that dependency. And then in the long run, we would like to be able to package our code as a debian package in the long run. By then we probably don't need linuxbrew anymore.

[dhermes]

Thanks. Is there a tracking bug for the smooth downgrade from ALPN to NPN? Let's hope "we probably don't need linuxbrew anymore." becomes "we don't need linuxbrew anymore."

RE: "Homebrew does not seem to have releases." I think even if it did, it would pull down new formulae via brew update just like apt-get update would.

[stanley-cheung]

https://github.com/grpc/grpc/releases/tag/release-0_10_0

ALPN is no longer a mandatory feature, meaning that OpenSSL 1.0.1 is now the lowest required version.

I also think that a grpc debian package is currently being worked on.

[dhermes]

w00t!

[stanley-cheung]

gRPC can now be installed via apt-get install libgrpc-dev via the Debian unstable repository. An RPM package will be worked on soon. We should no longer need linuxbrew. Can we close this now? :)

[dhermes]

I want to actually go through the process of installing via apt-get before closing out.

Reading http://askubuntu.com/questions/116257/adding-debian-sid-as-package-repository makes me think we should have a designated PPA rather than asking people (potentially too trusting people) to add unstable to sources.

[dhermes]

So just trying to install the deb (without subjecting a system to all the unstable packages) is a no-op:

From https://packages.debian.org/unstable/libgrpc-dev

wget http://http.us.debian.org/debian/pool/main/g/grpc/libgrpc-dev_0.10.0-1_amd64.deb
sudo dpkg -i libgrpc-dev_0.10.0-1_amd64.deb

fails because there is a missing dep of libgrpc0, but then

wget http://http.us.debian.org/debian/pool/main/g/grpc/libgrpc0_0.10.0-1_amd64.deb
sudo dpkg -i libgrpc0_0.10.0-1_amd64.deb

gets even hairier because of all the things it depends on (like an upstream openssl):

https://packages.debian.org/unstable/libgrpc0

(Wasn't the whole point of ALPN / NPN fix to not have to rely on upstream libssl?)

[stanley-cheung]

Update:

• We are unlikely to go down the path of making a PPA
• The libgrpc-dev package is now in Debian testing
• and is likely to make it into jessie backports in the near future [Edit: already did!]

[stanley-cheung]

The libgrpc-dev 0.10 package is now in jessie-backports. (Docker also has similar installation story) Now that we have done this one time, future releases (e.g. beta 0.11, etc) will follow a pattern of making it into unstable, then testing, then jessie-backports.

So can we close this now? :)

[dhermes]

Not really.

1. When will you ever just have apt-get install libgrpc / libgrpc-dev? Editing the sources.list and asking users to bring in backports may not work for everyone
2. What about wheezy-backports?

[tbetbetbe]

I'm closing this

• we no longer rely on linuxbrew
• we support install on jessie via jessie-backports

When will will have ever just have apt-get install libgrpc-dev ? Editing the sources.list and asking users to bring in backports may not work for everyone
This is actually normal for debian users. A debian stable release occurs once every 18 months; the rest of the time new packages are major releases are added to the xxx-backports, as that's tested to be compatible with the last stable release.

What about wheezy-backports ?
Lets consolidate this discussion on grpc/grpc/#662, this thread is no longer homebrew related

[tbetbetbe]

Closing, linuxbrew is no longer required on debian linux