Build gRPC for 0.10

[samjabrahams]

Build gRPC for 0.10

[samjabrahams]

Awesome stuff @dwightcrow. I gave this some time earlier today: like you, I was able to get protoc-gen-grpc-java built, but Bazel failed building. I'm going to give it another shot tomorrow, double checking that all of the Protobuf/GRPC versions match up properly.

[samjabrahams]

Just wanted to update: I wasn't able to get it built on Friday, and, unfortunately, I've been traveling since Saturday. I won't be able to give this another go until I get back next Tuesday. Hopefully one of us will be able to crack this nut soon!

[jalajc]

facing the same issue : gRPC Java plugin not found in. for Bazel build from source

while protobuf compile from source was smooth. I did makeand make check with -j 6 and j 4 respectively (-j 6on make check failed) to see substantial performance improvement.

[samjabrahams]

Right now, I believe what we need is to create the protoc-gen-grpc-java plugin from this section of the gRPC-java repository, and rename/move the output binary to third_party/grpc/protoc-gen-grpc-java-0.15.0-linux-arm32.exe

When attempting to just use the gradle.build file there, we end up with the error no tool chain for platform linux_arm-v7. I don't have much experience with Gradle, so I'm working through this to see how to register the RPi system to follow 32-bit build instructions.

[dwightcrow]

Would love it if you make progress on this - am attempting to build bazel 0.3.0 on the TX1 and am hitting same issue requiring protoc-gen-grpc-java-0.15.0-linux-arm32.exe as well. Are there other workarounds possible? Must admit I am not too familiar with protobuf or gRPC

[samjabrahams]

I think the main thing to do right now is to figure out how to adjust this grpc-java Gradle build script so that it doesn't fail on ARM devices. That should output the binaries that we need. Unfortunately, I don't have much experience with Gradle, and I haven't had much time to look into this yet. It may take me a few weeks before I can really set time aside to focus on this :/

[dwightcrow]

Got it. This is probably beyond my skill level to fix but will at least
give it a once over!

Found these threads. Do they make much sense to you as to what exact
changes required?
https://groups.google.com/forum/#!topic/grpc-io/DpsdI6kSxJI/discussion
https://groups.google.com/forum/#!topic/grpc-io/lEwfOdDgdZU/discussion

Seems like maybe just need to edit the platforms / components here to
include 'arm' (conceivably 'aarch64' for me, I'm working on TX1):
https://github.com/grpc/grpc-java/blob/v0.15.0/compiler/build.gradle#L55

Have you tried any of the above yet?

On Wed, Sep 28, 2016 at 11:43 PM, Sam Abrahams [email protected]
wrote:

I think the main thing to do right now is to figure out how to adjust this grpc-java
Gradle build script
https://github.com/grpc/grpc-java/blob/master/compiler/build.gradle so
that it doesn't fail on ARM devices. That should output the binaries that
we need. Unfortunately, I don't have much experience with Gradle, and I
haven't had much time to look into this yet. It may take me a few weeks
before I can really set time aside to focus on this :/

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#39 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAI43-Ws7SHpbR1aJhY-lf_rBmZ70odpks5qu14QgaJpZM4J8bJ2
.

[samjabrahams]

I hadn't seen the Google Groups discussion- nice find. I didn't think that we would have to do much finagling; I just wasn't sure how to properly modify the Gradle script (I played around with it a few days ago without much luck), but those links look promising.

I'll try to pick this up for an hour tomorrow to see what comes of it- keep me posted on your progress on the TX1.

[dwightcrow]

OK - have gotten gRPC to compile but fails ../gradlew test and bazel is still stuck, so second opinion/ideas welcome.

Following diff to build.gradle produces successful compile in grpc (branch v0.13.2, though have tried 0.15.0 as well)

diff --git a/compiler/build.gradle b/compiler/build.gradle
index 7e116d4..4183a59 100644
--- a/compiler/build.gradle
+++ b/compiler/build.gradle
@@ -47,6 +47,9 @@ model {
       }
     }
     gcc(Gcc) {
+       target("linux_aarch64"){
+           cppCompiler.executable = "/usr/bin/gcc"
+       }
     }
     clang(Clang) {
     }
@@ -59,11 +62,15 @@ model {
     x86_64 {
       architecture "x86_64"
     }
+    linux_aarch64 {
+      architecture "arm64"
+      operatingSystem "linux"
+    }
   }

   components {
     java_plugin(NativeExecutableSpec) {
-      if (arch in ['x86_32', 'x86_64']) {
+      if (arch in ['x86_32', 'x86_64', 'arm64']) {
         // If arch is not within the defined platforms, we do not specify the
         // targetPlatform so that Gradle will choose what is appropriate.
         targetPlatform arch

With
cp ~/grpc-java/compiler/build/exe/java_plugin/protoc-gen-grpc-java ~/bazel/third_party/grpc/protoc-gen-grpc-java-0.13.2-linux-arm32.exe

And in bazel scripts/bootstrap/compile.sh

diff --git a/scripts/bootstrap/compile.sh b/scripts/bootstrap/compile.sh
index 6faa330..e63e462 100755
--- a/scripts/bootstrap/compile.sh
+++ b/scripts/bootstrap/compile.sh
@@ -45,6 +45,7 @@ linux)
   else
     if [ "${MACHINE_IS_ARM}" = 'yes' ]; then
       PROTOC=${PROTOC:-third_party/protobuf/protoc-linux-arm32.exe}
+      GRPC_JAVA_PLUGIN=${GRPC_JAVA_PLUGIN:-third_party/grpc/protoc-gen-grpc-java-0.13.2-linux-arm32.exe}
     else
       PROTOC=${PROTOC:-third_party/protobuf/protoc-linux-x86_32.exe}
       GRPC_JAVA_PLUGIN=${GRPC_JAVA_PLUGIN:-third_party/grpc/protoc-gen-grpc-java-0.13.2-linux-x86_32.exe}

However, upon cd ~/bazel && ./compile.sh I get symbol not found errors such as error: cannot find symbol public static io.grpc.ServiceDescriptor getServiceDescriptor() - looks like big chunks of code are different/missing? See attached file for exact output on bazel compile errors.

error.txt

Also notably, ../gradlew test on arm32.exe binary also fails with error Could not find protoc-gen-javalite-linux-aarch_64.exe (com.google.protobuf:protoc-gen-javalite:3.0.0).

Most of above changes are reference off exhausted-engy's reply on this thread if that provides any clues. Didn't understand "/usr/bin/protoc was a custom compiled version as described in the grpc-java docs" but think everything else done same. Any thoughts most welcome!

[jalajc]

I did compile bazel today to get 1 error and 27 warnings...

/tmp/bazel_20mIUQtO/src/com/google/devtools/build/xcode/plmerge/proto/PlMergeProtos.java:1089: error: cannot find symbol
        .serializeStringMapTo(
        ^
  symbol:   method serializeStringMapTo(CodedOutputStream,MapField<String,String>,MapEntry<String,String>,int)
  location: class GeneratedMessageV3
src/main/java/com/google/devtools/build/lib/runtime/OomSignalHandler.java:33: warning: Signal is internal proprietary API and may be removed in a future release
  private static final Signal SIGUSR2 = (OS.getCurrent() != OS.WINDOWS) ? new Signal("USR2") : null;
                       ^
src/main/java/com/google/devtools/build/lib/runtime/OomSignalHandler.java:33: warning: Signal is internal proprietary API and may be removed in a future release
  private static final Signal SIGUSR2 = (OS.getCurrent() != OS.WINDOWS) ? new Signal("USR2") : null;
                                                                              ^
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error
27 warnings

I followed some relevant hints for armv7l (the 32 bit stuffs) from https://github.com/neo-titans/odroid/blob/master/build_tensorflow/bazel.47be2a4.patch

[jalajc]

@samjabrahams @dwightcrow I could go little ahead today, no previous errors... and the next stage of Building Bazel with Bazel started
Unfortunately when I was keeping my fingers crossed this strange error encountered...

🍃  Building Bazel with Bazel.
.WARNING: /tmp/bazel_hqMK1GWF/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_hqMK1GWF/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
ERROR: /home/pi/tf/bazel/third_party/protobuf/3.0.0/BUILD:22:12: Configurable attribute "srcs" doesn't match this configuration (would a default condition help?).
Conditions checked:
 //third_party:darwin
 //third_party:freebsd
 //third_party:k8
 //third_party:piii
 //third_party:s390x
 //third_party:windows_clang
 //third_party:windows_mingw
 //third_party:windows_msys64
 //third_party:windows_msys64_mingw64.
ERROR: Analysis of target '//src:bazel' failed; build aborted.
INFO: Elapsed time: 19.632s

ERROR: Could not build Bazel

This seems to be fixable easily... will keep posting!

[jalajc]

@samjabrahams @dwightcrow Finally I could build bazel

INFO: Elapsed time: 1368.058s, Critical Path: 1335.05s
WARNING: /tmp/bazel_glZVdCKI/out/external/bazel_tools/WORKSPACE:1: Workspace name in /tmp/bazel_glZVdCKI/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.

Build successful! Binary is here: /home/pi/tf/bazel/output/bazel

Enabling the verbose mode really helped (compile.sh)
Some caveats:

• Enabling the verbose mode really helped (compile.sh)
• Executed compile.sh with sudo as I observed at-least in my configuration some of the generated/binaries were not executable by normal login even though admin/owner (alternatively, chmod +x *.exe etc)
• gRPC version dependency and bazel version dependency on protobuf is different. I opted for intersection of dependency - protoc beta 3
• hints for armv7l (the 32 bit stuffs) from patch helped. I did applied changes manually on the respective files in patch for arm32 as well equivalent changes of arm64
• in case of error **symbol not found** in bazel I cherry picked the file from bazel hash: 47be2a40c601b5e4737f7a6825fad7e7f6ce0302 build (so essentially one can checkout that hash and apply patch referred above)

Next, I am not building the complete tensorflow, but some models (thats what I need) presumably unlike you folks.

[samjabrahams]

Version 0.10 binaries have been released.

[HanDaSeul]

@samjabrahams
I just tried to install tensorflow on my raspberry pi 2 and followed your guide here (https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md)
But, after I inserted code sudo ./compile.sh , it has problem with error at [1342/1396].
So I tried changing new version of bazel, tried without swap and change limitation of memory 100M to 1024M(that was able to install opencv) then changed -J-Xmx500M to -J-Xmx2048M. Even I tried switching oracle java(sudo apt-get install build-essential openjdk-8-jdk python zip), but that stuck 'building bazel from stratch..' and it has lasted 5hours.
What should I do for fix that problems?

I use Raspberry pi 2, 16GB SD card, noobs1.4.1(I guess that noobs version caused error but I afraid when I try to change new version of noobs, then my Raspberry pi 2 does not support newer one after 1.4.1 and I wonder if can upgrade newer one without removing all data that I installed in my SD card data)