C++ 40.14%C 9.58%Makefile 1.01%Java 16.78%Python 26.58%Shell 0.12%R 5.80%
deepwater's Introduction
Deep Water
What it is
Native implementation of Deep Learning models for GPU-optimized backends (MXNet, Caffe, TensorFlow, etc.)
State-of-the-art Deep Learning models trained from the H2O Platform
Train user-defined or pre-defined Deep Learning models for image/text/H2OFrame classification from Flow, R, Python, Java, Scala or REST API
Behaves just like any other H2O model (Flow, cross-validation, early stopping, hyper-parameter search, etc.)
Deep Water is a legacy project (as of December 2017), which means that it is no longer under active development. The H2O.ai team has no current plans to add new features, however, contributions from the community (in the form of pull requests) are welcome.
We have a pre-built image for Amazon Web Services's EC2 environment:
AMI ID: ami-97591381
AMI Name: h2o-deepwater-ami-latest
AWS Region: US East (N. Virginia)
Recommended instance type: p2.xlarge
The AMI image contains the Docker Image described below. Once started,
login to the shell prompt. It's a good idea to update the docker image
with docker pull opsh2oai/h2o-deepwater to ensure that you have the
most recent version. Then start the docker image, either with the
provided shell script or with nvidia-docker run -it --net host opsh2oai/h2o-deepwater.
Start H2O with java -Xmx30g -jar /opt/h2o.jar &. Connect to port 54321.
Start Jupyter with jupyter notebook --allow-root --ip=* &.
Connect to the link shown, with your IP exchanged for localhost.
Pre-Release Docker Image
We have a GPU-enabled Docker image and one the CPU only. Both are available on
Docker Hub.
Docker >= 1.9 (official docker-engine, docker-ce or docker-ee only)
NVIDIA GPU with Architecture > Fermi (2.1) and Compute Capability >= 3.5
NVIDIA drivers >= 340.29 with binary nvidia-modprobe
Download and run the H2O Docker image
nvidia-docker run -it --rm --net host -v $PWD:/host opsh2oai/h2o-deepwater
You now get a prompt in the image: # . The directory you started from is avaiable as /host
Start H2O with java -jar /opt/h2o.jar
Python, R and Jupyter Notebooks are available
exit or ctrl-d closes the image
CPU-only Docker Image
To use the CPU-enabled Docker image you just need to have Docker installed.
Note that this image is significantly slower than the GPU image, which is
why we don't recommend it.
Download and run the H2O Docker image:
On Linux: docker run -it --rm --net host -v $PWD:/host opsh2oai/h2o-deepwater-cpu
On MacOS: docker run -it --rm -p 54321:54321 -p 8080:8080 -v $PWD:/host opsh2oai/h2o-deepwater-cpu
You now get a prompt in the image: # . The directory you started from is avaiable as /host
From the top-level of the deepwater repository, do
./gradlew build -x test
This will create the following file: build/libs/deepwater-all.jar
5. Add DeepWater support to H2O-3
You need to check out the h2o-3.
Copy the freshly created jar file build/libs/deepwater-all.jar from the previous step to H2O-3's library h2o-3/lib/deepwater-all.jar (create the directory if it's not there) and you're done!
Build H2O-3 as usual:
./gradlew build -x test
This H2O version will now have GPU Deep Learning support!
To use the GPU, please make sure to set your path to your CUDA installation:
If you want to build your own MXNet models from Python or R, install the MXNet wheel (which was built together with MXNet above):
sudo easy_install deepwater/thirdparty/mxnet/python/dist/mxnet-0.7.0-py2.7.egg
R CMD INSTALL deepwater/thirdparty/mxnet/mxnet_0.7.tar.gz
Releasing
The release process bundles all defined submodules and push them into Maven central via Sonatype repository provider.
The released artifacts are Java 6 compatible.
Performing the actions "Close" and "Release" for the ai.h2o staging area
Note: Be careful because the area can contain more artifacts from different H2O projects.
Note: The release process creates two new commits and a new tag with the release version. However, the process does not push it to a remote repository, so it is necessary to perform a remote update manually using git push --tags or update the gradle/release.gradle settings and remove the --dry-run option from the pushOptions field.
Using the install instructions the latest Standalone version is 3.15.0.391 but the Jupyter version seems to be 3.13.0.369. So when trying to connect to the backend from Jupyter I am getting the following error.
H2OConnectionError: Version mismatch. H2O is version 3.15.0.391, but the h2o-python package is version 3.13.0.369. Install the matching h2o-Python version from - http://h2o-release.s3.amazonaws.com/h2o/(HEAD detached at ba3505b10)/391/index.html.
The link mentioned in the error message is not valid. What would the correct links be for versions that will connect with each other.
Environment
**OS platform, distribution and version : Redhat 7.5 ALT ppc64le
Python version (optional): python 3.6 from DriverlessAI
CUDA/cuDNN version: CUDA 9.2 cuDNN 7.1
GPU model (optional): V100
CPU model: POWER9
RAM available: 512GB
R version : 3.4.1
Tensorflow version : 1.8.0 (built from source)
I am trying to use h2o.deepwater included in DriverlessAI in python and R environment, instead of the web GUI of DAI. Plus, I'd like to use tensorflow as backend.
For this, I set the environment variables to use the python from DriverlessAI.
However, h2o.deepwater produces an error or "Unable to initialize the native Deep Learning backend: No backend found. Cannot build a Deep Water model."
Below is the error message related to running h2o.deepwater in R with backend of tensorflow.
$ cat t4.R
'# Package Load
library(reticulate)
use_python("/opt/h2oai/dai/python/bin/python")
library(Metrics)
library(h2o)
h2o.init(max_mem_size = "500g")
'# Data Load
df <- read.csv('/data/rpjt/R_script/user/yslee/powerplant_output.csv')
'# Randomly sample 80% of the rows for the training set
set.seed(1)
train_idx <- sample(1:nrow(df), 0.8*nrow(df))
'# h2o Dataset
train <- df[train_idx,]
test <- df[-train_idx,]
train <- as.h2o(train,col.types=c("string"))
test <- as.h2o(test,col.types=c("string"))
'# h2o.deepwater model
gpu_dl <- h2o.deepwater(x = c("TemperatureCelcius","ExhaustVacuumHg","AmbientPressureMillibar","RelativeHumidity"),
y = "HourlyEnergyOutputMW",
training_frame = train,
backend = "tensorflow",
hidden = 10,
standardize =T,
activation = "Tanh",
seed = 1234)
h2o.performance(gpu_dl, newdata = test)
$ Rscript t4.R
...
R is connected to the H2O cluster:
H2O cluster uptime: 16 minutes 30 seconds
H2O cluster timezone: Asia/Seoul
H2O data parsing timezone: UTC
H2O cluster version: 3.20.0.2
H2O cluster version age: 1 month and 22 days
H2O cluster name: dai
H2O cluster total nodes: 1
H2O cluster total memory: 227.37 GB
H2O cluster total cores: 128
H2O cluster allowed cores: 128
H2O cluster healthy: TRUE
H2O Connection ip: localhost
H2O Connection port: 54321
H2O Connection proxy: NA
H2O Internal Security: FALSE
H2O API Extensions: Algos, MLI, MLI-Driver, AutoML, Core V3, Core V4
R Version: R version 3.4.1 (2017-06-30)
java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: No backend found. Cannot build a Deep Water model.
java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: No backend found. Cannot build a Deep Water model.
at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:267)
at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:214)
at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:227)
at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:131)
at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:118)
at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:214)
at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:111)
at water.H2O$H2OCountedCompleter.compute(H2O.java:1260)
at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Error: java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: No backend found. Cannot build a Deep Water model.
Execution halted
install.pacakges("tensorflow") and library(tensorflow) worked fine in R,
$ ls -l /usr/local/lib64/R/library/tensorflow
total 12
-rw-rw-r-- 1 root root 2456 Aug 7 17:45 DESCRIPTION
drwxrwxr-x 5 root root 112 Aug 7 17:45 examples
drwxrwxr-x 2 root root 125 Aug 7 17:45 help
drwxrwxr-x 2 root root 39 Aug 7 17:45 html
-rw-rw-r-- 1 root root 1095 Aug 7 17:45 INDEX
drwxrwxr-x 2 root root 113 Aug 7 17:45 Meta
-rw-rw-r-- 1 root root 2713 Aug 7 17:45 NAMESPACE
drwxrwxr-x 2 root root 84 Aug 7 17:45 R
Also, tensorflow is installed in python from DriverlessAI.
installing source package ‘h2o’ ...
** R
Error in .install_package_code_files(".", instdir) :
files in '/tmp/Rtmpsd7UU3/R.INSTALL15c48573961e9/h2o/R' missing from 'Collate' field:
xgboost.R
ERROR: unable to collate and parse R files for package ‘h2o’
removing ‘/usr/lib64/R/library/h2o’
restoring previous ‘/usr/lib64/R/library/h2o’
This seems to be a description error. Any thoughts?
Hi, I am wondering if it's possible to provide any documentation or example code about using word2vec and cnn on text classification in H2O DeepWaterR version ?
By the way, is there any detailed tutorials or example about how to use the deep water in R ?
I have built latest h2o version with deepwater and I'm getting error after running h2o.deepwater with TensorFlow backend.
This is my code: model_tf<-h2o.deepwater(x=2:ncol(train), y=1, backend = "tensorflow", training_frame = train )
This is error:
java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: resource mlp_1093x1x1_32.meta not found.
java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: resource mlp_1093x1x1_32.meta not found.
at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:259)
at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:206)
at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:227)
at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:131)
at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:118)
at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:173)
at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:111)
at water.H2O$H2OCountedCompleter.compute(H2O.java:1240)
at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Error: java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: resource mlp_1093x1x1_32.meta not found.
The same code but with "mxnet" backend is working well.
I refer to "TensorFlow Bindings for H2O.ai"
I execute " ./gradlew clean tensorflowCompile",but always has error,likes ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': Error downloading [http://bazel-mirror.storage.googleapis.com/github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz, https://github.com/bazelbuild/rules_closure/archive/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz] to /root/.cache/bazel/_bazel_root/c0282f68c3c9fe7828209fa3ece89ea6/external/io_bazel_rules_closure/5ca1dab6df9ad02050f7ba4e816407f88690cf7d.tar.gz: Checksum was 5afc2087ab53b160fb58fde30339a2c2826c1a171404b7b8ff7227d5ebc8225c but wanted 60fc6977908f999b23ca65698c2bb70213403824a84f7904310b6000d78be9ce. :deepwater-tensorflow:tensorflowCompile FAILED
errors with checksum
how can I do ,fix this problem
how can I checksum disabled
Caused by: java.lang.NullPointerException
at java.awt.color.ICC_Profile.activateDeferredProfile(ICC_Profile.java:1090)
at java.awt.color.ICC_Profile$1.activate(ICC_Profile.java:744)
at sun.java2d.cmm.ProfileDeferralMgr.activateProfiles(ProfileDeferralMgr.java:95)
at java.awt.color.ICC_Profile.getInstance(ICC_Profile.java:777)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.setImageData(JPEGImageReader.java:657)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readImageHeader(Native Method)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readNativeHeader(JPEGImageReader.java:609)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.checkTablesOnly(JPEGImageReader.java:347)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.gotoImage(JPEGImageReader.java:481)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readHeader(JPEGImageReader.java:602)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.readInternal(JPEGImageReader.java:1059)
at com.sun.imageio.plugins.jpeg.JPEGImageReader.read(JPEGImageReader.java:1039)
at javax.imageio.ImageIO.read(ImageIO.java:1448)
at javax.imageio.ImageIO.read(ImageIO.java:1308)
at water.gpu.util.img2pixels(util.java:169)
at hex.deepwater.DeepWaterImageIterator$ImageConverter.compute2(DeepWaterImageIterator.java:102)
... 6 more
OS: ubuntu 14.04
I'm trying to install deepwater in my linux server, and I followed the tutorial step by step until Build TF Java bindings. Then I got some errors as below:
$ mvn -T 20 install --projects .,tensorflow
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:opencv:3.1.0-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/opencv/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:ffmpeg:jar:3.1.4-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:ffmpeg:3.1.4-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/ffmpeg/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:flycapture:2.9.3.43-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/flycapture/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:libdc1394:2.2.4-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/libdc1394/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:libfreenect:0.5.3-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/libfreenect/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:videoinput:0.200-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/videoinput/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:artoolkitplus:2.3.1-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/artoolkitplus/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:chilitags:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:chilitags:master-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/chilitags/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:flandmark:1.07-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/flandmark/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:hdf5:jar:1.10.0-patch1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:hdf5:1.10.0-patch1-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/hdf5/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:openblas:jar:0.2.19-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:openblas:0.2.19-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/openblas/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:fftw:jar:3.3.5-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:fftw:3.3.5-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/fftw/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:gsl:jar:2.2.1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:gsl:2.2.1-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/gsl/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:llvm:jar:3.9.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:llvm:3.9.0-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/llvm/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:leptonica:jar:1.73-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:leptonica:1.73-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/leptonica/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:tesseract:jar:3.04.01-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:tesseract:3.04.01-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/tesseract/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:caffe:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:caffe:master-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/caffe/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:cuda:jar:8.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:cuda:8.0-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/cuda/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:mxnet:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:mxnet:master-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/mxnet/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:tensorflow:jar:0.10.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:tensorflow:${tensorflow.version}-${project.parent.version}, /home/omm/deepwater/deepwater/thirdparty/javacpp-presets/tensorflow/pom.xml, line 14, column 12
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] Inspecting build with total of 21 modules...
[INFO] Not installing Nexus Staging features:
[INFO] * Preexisting staging related goal bindings found in 21 modules.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] JavaCPP Presets
[INFO] JavaCPP Presets for TensorFlow
[INFO] Building with 20 threads
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JavaCPP Presets 1.2.5-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] JavaCPP Presets ................................... FAILURE [0.111s]
[INFO] JavaCPP Presets for TensorFlow .................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.081s (Wall Clock)
[INFO] Finished at: Tue Jan 03 14:12:15 CST 2017
[INFO] Final Memory: 13M/217M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project javacpp-presets: Could not resolve dependencies for project org.bytedeco:javacpp-presets:pom:1.2.5-SNAPSHOT: Could not find artifact org.bytedeco:javacpp-presets:jar:linux-x86_64:1.2.5-SNAPSHOT -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
I don't know what's wrong, could you give me some advice to get rid of it?
How the statement 'Coming soon' about Caffe support should be interpreted?
Could you share more detailed plans or schedule for Caffe to be integrated with Deep Water?
Thanks,
Chris
When I am running mvn -e -X -T 20 install --projects .,tensorflow
[](org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (javacpp.cppbuild.install) on project tensorflow: Command execution failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:185)
at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:181)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed.
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:276)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
... 11 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine(ExecMojo.java:660)
at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:265)
... 13 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :tensorflow
))
Hi,
I followed the deeplearning_tensorflow_cat_dog_mouse_lenet notebook and tried to download the model as a pojo file.
model = H2ODeepWaterEstimator(epochs=50, network = "lenet", backend="tensorflow")
model.train(x=[0],y=1, training_frame=frame)
h2o.download_pojo(model)
And got the following error
H2OServerError Traceback (most recent call last)
<ipython-input-7-8b8a68a0b108> in <module>()
----> 1 h2o.download_pojo(model)
/usr/local/lib/python2.7/dist-packages/h2o/h2o.pyc in download_pojo(model, path, get_jar)
817
818 if path == "":
--> 819 java_code = api("GET /3/Models.java/%s" % model.model_id)
820 print(java_code)
821 return None
/usr/local/lib/python2.7/dist-packages/h2o/h2o.pyc in api(endpoint, data, json, filename, save_to)
82 # type checks are performed in H2OConnection class
83 _check_connection()
---> 84 return h2oconn.request(endpoint, data=data, json=json, filename=filename, save_to=save_to)
85
86
/usr/local/lib/python2.7/dist-packages/h2o/backend/connection.pyc in request(self, endpoint, data, json, filename, save_to)
261 auth=self._auth, verify=self._verify_ssl_cert, proxies=self._proxies)
262 self._log_end_transaction(start_time, resp)
--> 263 return self._process_response(resp, save_to)
264
265 except (requests.exceptions.ConnectionError, requests.exceptions.HTTPError) as e:
/usr/local/lib/python2.7/dist-packages/h2o/backend/connection.pyc in _process_response(response, save_to)
593 # Note that it is possible to receive valid H2OErrorV3 object in this case, however it merely means the server
594 # did not provide the correct status code.
--> 595 raise H2OServerError("HTTP %d %s:\n%r" % (status_code, response.reason, data))
596
597
H2OServerError: HTTP 500 Server Error:
u'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>\n<title>Error 500 Server Error</title>\n</head>\n<body>\n<h2>HTTP ERROR: 500</h2>\n<p>Problem accessing /3/Models.java/DeepWater_model_python_1493301774497_1. Reason:\n<pre> Server Error</pre></p>\n<hr /><i><small>Powered by Jetty://</small></i>\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n</body>\n</html>\n'
I'm using the deepwater AMI (ami-10bd9607, altough I know it's an an old AMI image #39 )
Does deepwater support the downloading of deep learning model as a pojo file?
Fresh install of H2O failing to init. Uninstalled and reinstalled several times. Same problem persists.
H2OConnectionError: Version mismatch. H2O is version 3.18.0.11, but the h2o-python package is version 3.20.0.8. Install the matching h2o-Python version from - http://h2o-release.s3.amazonaws.com/h2o/rel-wolpert/11/index.html.
Everything runs fine up until this step: mvn -T 20 install --projects .,tensorflow
Following the script's recommendation, I re-ran with debugging & error flags: mvn -e -X -T 20 install --projects .,tensorflow
The last command generated the error messages below. I looked into submitting an issue at the Javacpp-presets github page but it was not clear to me how to open a new issue there.
Thanks.
Javacpp-presets debugging messages:
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T10:41:47-06:00)
Maven home: /usr/local/Cellar/maven/3.3.9/libexec
Java version: 1.8.0_102, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.11.6", arch: "x86_64", family: "mac"
[DEBUG] Created new class realm maven.api
[DEBUG] Importing foreign packages into class realm maven.api
[DEBUG] Imported: javax.enterprise.inject.* < plexus.core
[DEBUG] Imported: javax.enterprise.util.* < plexus.core
[DEBUG] Imported: javax.inject.* < plexus.core
[DEBUG] Imported: org.apache.maven.* < plexus.core
[DEBUG] Imported: org.apache.maven.artifact < plexus.core
[DEBUG] Imported: org.apache.maven.classrealm < plexus.core
[DEBUG] Imported: org.apache.maven.cli < plexus.core
[DEBUG] Imported: org.apache.maven.configuration < plexus.core
[DEBUG] Imported: org.apache.maven.exception < plexus.core
[DEBUG] Imported: org.apache.maven.execution < plexus.core
[DEBUG] Imported: org.apache.maven.execution.scope < plexus.core
[DEBUG] Imported: org.apache.maven.lifecycle < plexus.core
[DEBUG] Imported: org.apache.maven.model < plexus.core
[DEBUG] Imported: org.apache.maven.monitor < plexus.core
[DEBUG] Imported: org.apache.maven.plugin < plexus.core
[DEBUG] Imported: org.apache.maven.profiles < plexus.core
[DEBUG] Imported: org.apache.maven.project < plexus.core
[DEBUG] Imported: org.apache.maven.reporting < plexus.core
[DEBUG] Imported: org.apache.maven.repository < plexus.core
[DEBUG] Imported: org.apache.maven.rtinfo < plexus.core
[DEBUG] Imported: org.apache.maven.settings < plexus.core
[DEBUG] Imported: org.apache.maven.toolchain < plexus.core
[DEBUG] Imported: org.apache.maven.usability < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.* < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.authentication < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.authorization < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.events < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.observers < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.proxy < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.repository < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.resource < plexus.core
[DEBUG] Imported: org.codehaus.classworlds < plexus.core
[DEBUG] Imported: org.codehaus.plexus.* < plexus.core
[DEBUG] Imported: org.codehaus.plexus.classworlds < plexus.core
[DEBUG] Imported: org.codehaus.plexus.component < plexus.core
[DEBUG] Imported: org.codehaus.plexus.configuration < plexus.core
[DEBUG] Imported: org.codehaus.plexus.container < plexus.core
[DEBUG] Imported: org.codehaus.plexus.context < plexus.core
[DEBUG] Imported: org.codehaus.plexus.lifecycle < plexus.core
[DEBUG] Imported: org.codehaus.plexus.logging < plexus.core
[DEBUG] Imported: org.codehaus.plexus.personality < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.Xpp3Dom < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlPullParser < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlPullParserException < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlSerializer < plexus.core
[DEBUG] Imported: org.eclipse.aether.* < plexus.core
[DEBUG] Imported: org.eclipse.aether.artifact < plexus.core
[DEBUG] Imported: org.eclipse.aether.collection < plexus.core
[DEBUG] Imported: org.eclipse.aether.deployment < plexus.core
[DEBUG] Imported: org.eclipse.aether.graph < plexus.core
[DEBUG] Imported: org.eclipse.aether.impl < plexus.core
[DEBUG] Imported: org.eclipse.aether.installation < plexus.core
[DEBUG] Imported: org.eclipse.aether.internal.impl < plexus.core
[DEBUG] Imported: org.eclipse.aether.metadata < plexus.core
[DEBUG] Imported: org.eclipse.aether.repository < plexus.core
[DEBUG] Imported: org.eclipse.aether.resolution < plexus.core
[DEBUG] Imported: org.eclipse.aether.spi < plexus.core
[DEBUG] Imported: org.eclipse.aether.transfer < plexus.core
[DEBUG] Imported: org.eclipse.aether.version < plexus.core
[DEBUG] Imported: org.slf4j.* < plexus.core
[DEBUG] Imported: org.slf4j.helpers.* < plexus.core
[DEBUG] Imported: org.slf4j.spi.* < plexus.core
[DEBUG] Populating class realm maven.api
[INFO] Error stacktraces are turned on.
[DEBUG] Reading global settings from /usr/local/Cellar/maven/3.3.9/libexec/conf/settings.xml
[DEBUG] Reading user settings from /Users/mohamed.badawy/.m2/settings.xml
[DEBUG] Reading global toolchains from /usr/local/Cellar/maven/3.3.9/libexec/conf/toolchains.xml
[DEBUG] Reading user toolchains from /Users/mohamed.badawy/.m2/toolchains.xml
[DEBUG] Using local repository at /Users/mohamed.badawy/.m2/repository
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /Users/mohamed.badawy/.m2/repository
[INFO] Scanning for projects...
[DEBUG] Dependency collection stats: {ConflictMarker.analyzeTime=1, ConflictMarker.markTime=1, ConflictMarker.nodeCount=84, ConflictIdSorter.graphTime=0, ConflictIdSorter.topsortTime=0, ConflictIdSorter.conflictIdCount=45, ConflictIdSorter.conflictIdCycleCount=0, ConflictResolver.totalTime=5, ConflictResolver.conflictItemCount=71, DefaultDependencyCollector.collectTime=277, DefaultDependencyCollector.transformTime=11}
[DEBUG] org.sonatype.plugins:nexus-staging-maven-plugin:jar:1.6.6:
[DEBUG] org.sonatype.nexus.maven:nexus-common:jar:1.6.6:compile
[DEBUG] org.sonatype.plexus:plexus-sec-dispatcher:jar:1.4:compile
[DEBUG] org.sonatype.plexus:plexus-cipher:jar:1.7:compile
[DEBUG] com.google.guava:guava:jar:14.0.1:compile
[DEBUG] org.sonatype.nexus:nexus-client-core:jar:2.9.1-02:compile
[DEBUG] org.sonatype.nexus.plugins:nexus-restlet1x-model:jar:2.9.1-02:compile
[DEBUG] org.apache.maven:maven-model:jar:3.0.4:compile
[DEBUG] org.slf4j:slf4j-api:jar:1.7.7:compile
[DEBUG] com.google.code.findbugs:jsr305:jar:2.0.1:compile
[DEBUG] com.intellij:annotations:jar:9.0.4:compile
[DEBUG] commons-io:commons-io:jar:2.4:compile
[DEBUG] com.thoughtworks.xstream:xstream:jar:1.4.7:compile
[DEBUG] xmlpull:xmlpull:jar:1.1.3.1:compile
[DEBUG] xpp3:xpp3_min:jar:1.1.4c:compile
[DEBUG] joda-time:joda-time:jar:2.2:compile
[DEBUG] commons-lang:commons-lang:jar:2.6:compile
[DEBUG] commons-beanutils:commons-beanutils-core:jar:1.8.3:compile
[DEBUG] org.sonatype.sisu.siesta:siesta-client:jar:1.7:compile
[DEBUG] org.sonatype.sisu.siesta:siesta-common:jar:1.7:compile
[DEBUG] javax.ws.rs:jsr311-api:jar:1.1.1:compile
[DEBUG] com.sun.jersey:jersey-core:jar:1.17.1:compile
[DEBUG] javax.validation:validation-api:jar:1.1.0.Final:compile
[DEBUG] com.sun.jersey:jersey-client:jar:1.17.1:compile
[DEBUG] com.sun.jersey.contribs:jersey-apache-client4:jar:1.17.1:compile
[DEBUG] org.sonatype.sisu.siesta:siesta-jackson:jar:1.7:compile
[DEBUG] com.fasterxml.jackson.core:jackson-annotations:jar:2.3.1:compile
[DEBUG] com.fasterxml.jackson.core:jackson-core:jar:2.3.1:compile
[DEBUG] com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:compile
[DEBUG] com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.3.1:compile
[DEBUG] com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.3.1:compile
[DEBUG] com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.3.1:compile
[DEBUG] org.apache.httpcomponents:httpclient:jar:4.3.5:compile
[DEBUG] commons-codec:commons-codec:jar:1.6:compile
[DEBUG] org.apache.httpcomponents:httpcore:jar:4.3.2:compile
[DEBUG] org.slf4j:jcl-over-slf4j:jar:1.7.7:compile
[DEBUG] javax.inject:javax.inject:jar:1:compile
[DEBUG] org.sonatype.spice.zapper:spice-zapper:jar:1.3:compile
[DEBUG] org.fusesource.hawtbuf:hawtbuf-proto:jar:1.9:compile
[DEBUG] org.fusesource.hawtbuf:hawtbuf:jar:1.9:compile
[DEBUG] org.codehaus.plexus:plexus-utils:jar:3.0.8:compile
[DEBUG] org.codehaus.plexus:plexus-interpolation:jar:1.15:compile
[DEBUG] org.sonatype.aether:aether-api:jar:1.13.1:compile
[DEBUG] ch.qos.logback:logback-core:jar:1.1.2:runtime
[DEBUG] ch.qos.logback:logback-classic:jar:1.1.2:runtime
[DEBUG] Created new class realm extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6
[DEBUG] Importing foreign packages into class realm extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6
[DEBUG] Imported: < maven.api
[DEBUG] Populating class realm extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6
[DEBUG] Included: org.sonatype.plugins:nexus-staging-maven-plugin:jar:1.6.6
[DEBUG] Included: org.sonatype.nexus.maven:nexus-common:jar:1.6.6
[DEBUG] Included: org.sonatype.plexus:plexus-sec-dispatcher:jar:1.4
[DEBUG] Included: org.sonatype.plexus:plexus-cipher:jar:1.7
[DEBUG] Included: com.google.guava:guava:jar:14.0.1
[DEBUG] Included: org.sonatype.nexus:nexus-client-core:jar:2.9.1-02
[DEBUG] Included: org.sonatype.nexus.plugins:nexus-restlet1x-model:jar:2.9.1-02
[DEBUG] Included: com.google.code.findbugs:jsr305:jar:2.0.1
[DEBUG] Included: com.intellij:annotations:jar:9.0.4
[DEBUG] Included: commons-io:commons-io:jar:2.4
[DEBUG] Included: com.thoughtworks.xstream:xstream:jar:1.4.7
[DEBUG] Included: xmlpull:xmlpull:jar:1.1.3.1
[DEBUG] Included: xpp3:xpp3_min:jar:1.1.4c
[DEBUG] Included: joda-time:joda-time:jar:2.2
[DEBUG] Included: commons-lang:commons-lang:jar:2.6
[DEBUG] Included: commons-beanutils:commons-beanutils-core:jar:1.8.3
[DEBUG] Included: org.sonatype.sisu.siesta:siesta-client:jar:1.7
[DEBUG] Included: org.sonatype.sisu.siesta:siesta-common:jar:1.7
[DEBUG] Included: javax.ws.rs:jsr311-api:jar:1.1.1
[DEBUG] Included: com.sun.jersey:jersey-core:jar:1.17.1
[DEBUG] Included: javax.validation:validation-api:jar:1.1.0.Final
[DEBUG] Included: com.sun.jersey:jersey-client:jar:1.17.1
[DEBUG] Included: com.sun.jersey.contribs:jersey-apache-client4:jar:1.17.1
[DEBUG] Included: org.sonatype.sisu.siesta:siesta-jackson:jar:1.7
[DEBUG] Included: com.fasterxml.jackson.core:jackson-annotations:jar:2.3.1
[DEBUG] Included: com.fasterxml.jackson.core:jackson-core:jar:2.3.1
[DEBUG] Included: com.fasterxml.jackson.core:jackson-databind:jar:2.3.1
[DEBUG] Included: com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.3.1
[DEBUG] Included: com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.3.1
[DEBUG] Included: com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.3.1
[DEBUG] Included: org.apache.httpcomponents:httpclient:jar:4.3.5
[DEBUG] Included: commons-codec:commons-codec:jar:1.6
[DEBUG] Included: org.apache.httpcomponents:httpcore:jar:4.3.2
[DEBUG] Included: org.slf4j:jcl-over-slf4j:jar:1.7.7
[DEBUG] Included: org.sonatype.spice.zapper:spice-zapper:jar:1.3
[DEBUG] Included: org.fusesource.hawtbuf:hawtbuf-proto:jar:1.9
[DEBUG] Included: org.fusesource.hawtbuf:hawtbuf:jar:1.9
[DEBUG] Included: org.codehaus.plexus:plexus-utils:jar:3.0.8
[DEBUG] Included: org.codehaus.plexus:plexus-interpolation:jar:1.15
[DEBUG] Included: ch.qos.logback:logback-core:jar:1.1.2
[DEBUG] Included: ch.qos.logback:logback-classic:jar:1.1.2
[DEBUG] Extension realms for project org.bytedeco:javacpp-presets:pom:1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Created new class realm project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT
[DEBUG] Populating class realm project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT
[DEBUG] Looking up lifecyle mappings for packaging pom from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:ffmpeg:jar:3.1.4-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:chilitags:jar:master-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:hdf5:jar:1.10.0-patch1-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:openblas:jar:0.2.19-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:fftw:jar:3.3.5-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:gsl:jar:2.2.1-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:llvm:jar:3.9.0-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:leptonica:jar:1.73-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:tesseract:jar:3.04.01-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:caffe:jar:master-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:cuda:jar:8.0-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:mxnet:jar:master-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[DEBUG] Extension realms for project org.bytedeco.javacpp-presets:tensorflow:jar:0.10.0-1.2.5-SNAPSHOT: [ClassRealm[extension>org.sonatype.plugins:nexus-staging-maven-plugin:1.6.6, parent: sun.misc.Launcher$AppClassLoader@55f96302]]
[DEBUG] Looking up lifecyle mappings for packaging jar from ClassRealm[project>org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:opencv:jar:3.1.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:opencv:3.1.0-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/opencv/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:ffmpeg:jar:3.1.4-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:ffmpeg:3.1.4-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/ffmpeg/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:flycapture:jar:2.9.3.43-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:flycapture:2.9.3.43-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/flycapture/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:libdc1394:jar:2.2.4-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:libdc1394:2.2.4-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/libdc1394/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:libfreenect:jar:0.5.3-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:libfreenect:0.5.3-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/libfreenect/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:videoinput:jar:0.200-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:videoinput:0.200-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/videoinput/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:artoolkitplus:jar:2.3.1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:artoolkitplus:2.3.1-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/artoolkitplus/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:chilitags:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:chilitags:master-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/chilitags/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:flandmark:jar:1.07-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:flandmark:1.07-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/flandmark/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:hdf5:jar:1.10.0-patch1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:hdf5:1.10.0-patch1-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/hdf5/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:openblas:jar:0.2.19-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:openblas:0.2.19-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/openblas/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:fftw:jar:3.3.5-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:fftw:3.3.5-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/fftw/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:gsl:jar:2.2.1-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:gsl:2.2.1-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/gsl/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:llvm:jar:3.9.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:llvm:3.9.0-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/llvm/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:leptonica:jar:1.73-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:leptonica:1.73-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/leptonica/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:tesseract:jar:3.04.01-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:tesseract:3.04.01-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/tesseract/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:caffe:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:caffe:master-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/caffe/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:cuda:jar:8.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:cuda:8.0-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/cuda/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:mxnet:jar:master-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:mxnet:master-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/mxnet/pom.xml, line 14, column 12
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.bytedeco.javacpp-presets:tensorflow:jar:0.10.0-1.2.5-SNAPSHOT
[WARNING] 'version' contains an expression but should be a constant. @ org.bytedeco.javacpp-presets:tensorflow:${tensorflow.version}-${project.parent.version}, /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/tensorflow/pom.xml, line 14, column 12
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] Inspecting build with total of 2 modules...
[INFO] Not installing Nexus Staging features:
[INFO] * Preexisting staging related goal bindings found in 2 modules.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] JavaCPP Presets
[INFO] JavaCPP Presets for TensorFlow
[DEBUG] === REACTOR BUILD PLAN ================================================
[DEBUG] Project: org.bytedeco:javacpp-presets:pom:1.2.5-SNAPSHOT
[DEBUG] Tasks: [install]
[DEBUG] Style: Regular
[DEBUG] -----------------------------------------------------------------------
[DEBUG] Project: org.bytedeco.javacpp-presets:tensorflow:jar:0.10.0-1.2.5-SNAPSHOT
[DEBUG] Tasks: [install]
[DEBUG] Style: Regular
[DEBUG] =======================================================================
[INFO]
[INFO] Using the MultiThreadedBuilder implementation with a thread count of 20
[DEBUG] Scheduling: MavenProject: org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT @ /Users/mohamed.badawy/Bin/deepwater/thirdparty/javacpp-presets/pom.xml
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building JavaCPP Presets 1.2.5-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[DEBUG] Lifecycle default -> [validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy]
[DEBUG] Lifecycle clean -> [pre-clean, clean, post-clean]
[DEBUG] Lifecycle site -> [pre-site, site, post-site, site-deploy]
[DEBUG] Lifecycle default -> [validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy]
[DEBUG] Lifecycle clean -> [pre-clean, clean, post-clean]
[DEBUG] Lifecycle site -> [pre-site, site, post-site, site-deploy]
[DEBUG] Lifecycle default -> [validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy]
[DEBUG] Lifecycle clean -> [pre-clean, clean, post-clean]
[DEBUG] Lifecycle site -> [pre-site, site, post-site, site-deploy]
[DEBUG] === PROJECT BUILD PLAN ================================================
[DEBUG] Project: org.bytedeco:javacpp-presets:1.2.5-SNAPSHOT
[DEBUG] Dependencies (collect): []
[DEBUG] Dependencies (resolve): [compile]
[DEBUG] Repositories (dependencies): [central (https://repo.maven.apache.org/maven2, default, releases)]
[DEBUG] Repositories (plugins) : [central (https://repo.maven.apache.org/maven2, default, releases)]
[DEBUG] -----------------------------------------------------------------------
[DEBUG] Goal: org.apache.maven.plugins:maven-javadoc-plugin:2.10.3:jar (attach-javadocs)
[DEBUG] Style: Regular
[DEBUG] Configuration:
After successfully building Deepwater, the documentation says: "Copy the freshly created jar file build/libs/deepwater-all.jar from the previous step to H2O-3's library h2o-3/lib/deepwater-all.jar."
In the H2O-3 repo, there is no lib dir. Am I misreading the instruction? Is this a typo? Has the location to place the jar file changed?
I'm trying to understand how Deep Water utilizes GPU memory. I ran the 'Deep Water Deep Features Similarity Cars Inception' notebook and noticed that nvtop showed
Device 0 [Tesla K80] PCIe GEN 3@16x RX: 0.000 kB/s TX: 0.000 kB/s
GPU 562MHz MEM 2505MHz TEMP 74M- FAN 0% POW 66 / 149 W
GPU-Util[||||||| 53%] MEM-Util[|||||4.4G/12.0G] Encoder[ 0%] Decoder[ 0%]
where GPU-Util fluctuated, but MEM-Util held constant at 4.4G until the notebook was closed.
Does Deep Water use the GPU to store the model and run it from there? I'm used to seeing full GPU usage with traditional basic tensorflow scripts.
I apologize in advance for any incorrect terminology, I'll amend this as needed.
I am using macOS and I only write python3 (when it comes to python) so I do not maintain any additional packages for the builti-in python27 in macOS.
After I build the deepwater-all.jar, I turned to build the h2o-3 according to the document. I am aware of the python dependencies in the h2o-3 project. So I change the pythonexe to point to my python3 (and also pip to my pip3) in build.gradle under project h2o-py in hope that only python3 will involve in the process. However, there is a problem with the project h2o-bindings cause the script bindings.py will import the requests package, which is not built-in for python27.
I don't have a way to workaround this by using python3 so I can only by-pass the build if I install the requests package for python27 in my system. (I don't even have a pip for python2 so I build it from source...)
Another error occurs at h2o-test-integ when running the testing script test_gbm_prostate.py where the package tabulate (and many others...) are required and again it uses python2 instead of python3.
Here is my question: Is python2 an explicit dependency for building deepwater and h2o-3? If not, then would it be good to let user choose which python to use for the whole build process? This will be especially helpful for mac user due to the multiple python path issue.
The thirdparty/javacpp-preset reference a fork instead of the original project of javacpp-preset. What's the matter is that the TensorFlow version is too old(v0.10), compared to original javacpp-preset, which is updated to TensorFlow v1.0.0-alpha.
Is there any problem to update TensorFlow Version? I have a look at the TensorFlow Demos MNIST and Lenet
I think it's a good idea to use Serializable and Deserialize for decoupling front-end from backend. So, I'd like to join in you to contribute and maintain the project. BTW, I'm an active contributor of TensorFlow and Kubernetes communities.
At a word, I plan to make a PR to update the TensorFlow version and the dependent branch of javacpp-preset.
I am using the AMI link provided in deepwater and trying to build the package based on it.
When I build h20-3 using ./gradlew build -x test
I get following error
`Traceback (most recent call last):
File "setup.py", line 6, in
import h2o
File "/home/ubuntu/h2o-3/h2o-py/h2o/init.py", line 11, in
from h2o.h2o import (connect, init, api, connection,
File "/home/ubuntu/h2o-3/h2o-py/h2o/h2o.py", line 14, in
from h2o.backend import H2OConnection
File "/home/ubuntu/h2o-3/h2o-py/h2o/backend/init.py", line 42, in
from .cluster import H2OCluster
File "/home/ubuntu/h2o-3/h2o-py/h2o/backend/cluster.py", line 10, in
from h2o.display import H2ODisplay
File "/home/ubuntu/h2o-3/h2o-py/h2o/display.py", line 10, in
import tabulate
ImportError: No module named tabulate
:h2o-py:buildDist FAILED
:h2o-py:buildDist took 1.47 secs
From the main project directory, I ran ./gradlew clean tensorflowCompile to build the so and jar TensorFlow files.
But gcc reported the errors below and stopped:
gcc: error: unrecognized command line option '-Wthread-safety'
gcc: error: unrecognized command line option '-Wself-assign'
Problem was fixed after I changed the bazel version to 0.5.4
So I supposed that, the Prerequisites of Tensorflow Bindings should be changed from
"Install Bazel 0.4.5 or later" to "Install Bazel 0.5.*"
PS:
Extra errors will be raised after I changed the bazel version to 0.6.0 and later, because set() was deprecated in 0.6.0 and later version
Whether Deepwater repo is cloned or zip-downloded, the ./thirdparty/javacpp folder is empty. I just noticed that someone has forked it, could it be that it has been force pushed? Could someone please advise/work on a fix? Thanks.
So I have installed CUDA, cudNN, tensorflow and h2o in R to build a GPU enabled deepwater neural net using Tensorflow as a backend (ubuntu 16.04 Nvidia docker).
Running Tensorflow with gpu support as follows works fine:
When "balance classes" is selected in a Deep Water model it breaks because it tries to select a sample out of the valid range of samples. A Java error in red in the H2O flow GUI can be seen with something like this:
0 <= 403123 <= 207534
I haven't been able to find any other condition to raise this error. This does not happen with the same configuration (dataset and settings) using Deep Learning models.
H2O Build git branch (HEAD detached at 7c6572f64)
H2O Build git hash 7c6572f642ec554fd89163f20a41d3b612e976f5
H2O Build git describe jenkins-master-3970
H2O Build project version 3.13.0.352
H2O Build age 1 month and 6 days
H2O Built by jenkins
H2O Built on 2017-07-28 01:05:29
H2O Internal Security Disabled
Flow version 0.7.2
The deepwater whl file says "py2.py3", but the mxnet and tensorflow links only link to py2 versions.
I've tried pip3 install mxnet, which installed "mxnet-0.11.0-py2.py3-none-manylinux1_x86_64.whl", but it still fails with "java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: /tmp/libmxnet.so: libcudart.so.8.0: cannot open shared object file: No such file or directory"
I don't know if this is because of mxnet 0.7 vs. mxnet 0.11, or python 2.7 vs. 3.5, or because of some other configuration step I've missed.
(BTW, I'm working through the deeplearning_mnist_introduction jupyter notebook. Mint 18.1, 64-bit, h2o is 3.13.0.369, H2O API Extensions says "XGBoost, Algos, AutoML, Core V3, Core V4". Python is 3.5.2.)
If I tried backend="tensorflow" I instead get:
java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: Python Tensorflow not installed on this machine. Please run 'pip install tensorflow[-gpu]' first.
This was after having run pip3 install tensorflow. "tensorflow-1.3.0-cp35-cp35m-manylinux1_x86_64.whl" was installed.
(Again, is the problem the version 1.3 vs. 1.1, or Python 2.7 vs. 3.5, or something else?)
BTW, identical results with and without gpu=False, for both mxnet and tensorflow.
Hello,
I run ./gradlew --stacktrace clean tensorflowCompile to build the so and jar TensorFlow files and get the following error:
FAILURE: Build failed with an exception.
What went wrong:
Could not determine java version from '9-internal'.
Try:
Run with --info or --debug option to get more log output.
Exception is:
java.lang.IllegalArgumentException: Could not determine java version from '9-internal'.
at org.gradle.api.JavaVersion.toVersion(JavaVersion.java:63)
at org.gradle.api.JavaVersion.current(JavaVersion.java:72)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:32)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
at org.gradle.launcher.Main.doAction(Main.java:33)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
at sun.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:531)
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
at sun.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:531)
at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:129)
at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
I also try to install gradle but still get the same error.
Any ideas how to complete the installation?
I posted this question on the community forum as well, but I'm not sure if it's a bug, or me using Tensorflow wrong:
I cannot get TensorFlow to build a meta graph, when using it with Deep Water. It works fine, when I align the input to use one of the included sizes (e.g. mlp_8x1x1_10.meta).
I built everything from master yesterday on CentOS 7 (selinux is disabled).
The following output is from one of my many experiments with the MNIST training set demo.
INFO: Hidden layers: [200, 200]
INFO: Activation function: Rectifier
INFO: Input dropout ratio: 0.0
INFO: Hidden layer dropout ratio: [0.0, 0.0]
INFO: Creating a fresh model of the following network type: MLP
ERRR: java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: resource mlp_784x1x1_10.meta not found.
ERRR: at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:246)
ERRR: at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:193)
ERRR: at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:225)
ERRR: at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:127)
ERRR: at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:114)
ERRR: at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:169)
ERRR: at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:107)
ERRR: at water.H2O$H2OCountedCompleter.compute(H2O.java:1220)
ERRR: at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
ERRR: at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
ERRR: at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
ERRR: at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
ERRR: at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
[2017-07-10 11:37:44] failure_details: Unable to initialize the native Deep Learning backend: null
[2017-07-10 11:37:44] failure_stack_traces: java.lang.RuntimeException: Unable to initialize the native Deep Learning backend: null
at hex.deepwater.DeepWaterModelInfo.setupNativeBackend(DeepWaterModelInfo.java:267)
at hex.deepwater.DeepWaterModelInfo.(DeepWaterModelInfo.java:214)
at hex.deepwater.DeepWaterModel.(DeepWaterModel.java:227)
at hex.deepwater.DeepWater$DeepWaterDriver.buildModel(DeepWater.java:131)
at hex.deepwater.DeepWater$DeepWaterDriver.computeImpl(DeepWater.java:118)
at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:173)
at hex.deepwater.DeepWater$DeepWaterDriver.compute2(DeepWater.java:111)
at water.H2O$H2OCountedCompleter.compute(H2O.java:1256)
at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent