xbpeng/deeploco

Intro

This project is designed to learn good navigation skills for simulated characters

Setup

This section covers some of the steps to setup and compile the code. The software depends on many libraries that need to be carefully prepared and placed for the building and linking to work properly.

Linux (Ubuntu 16.04)

Install system dependencies

Run the deb_deps.sh script to install the system dependencies required to build the project.

OpenGL >= 3.3

Ensure that your machine is capable of running OpenGL version 3.3 or greater.

You can verify the version of OpenGL your machine currently has installed via the following command:

glxinfo | grep "OpenGL version"

If glxinfo is not installed on your system, install the mesa-utils package.

OpenGL should come as part of the drivers for your graphics hardware (whether part of the motherboard or dedicated graphics card). If you are missing a compatible version of OpenGL, consider updating your graphics drivers; if you have a GPU, ensure that the system is actually using it.

Download premake4

Download premake4 from here. Extract the premake4 binary into a directory of your choosing (preferably not your local Downloads directory).

Add premake to your PATH variable in .bashrc:

# Add premake to path
export PATH=[PREMAKE_DIR]:$PATH

...where [PREMAKE_DIR] should be the directory containing the premake4 binary.

Build Instructions

1. Download the most recent compressed external file from the newest release.

2. Extract it and move into the DeepLoco directory. The top directory of the DeepLoco repository should now contain a directory called external, in addition to all the other directories that were there before.

3. Build the source code for caffe that came in the external directory.

   cd external/caffe
   make clean
   make
   cd ../../
   

4. Copy the newly-compiled caffe lib directory from external/caffe/build/lib to the top directory of DeepLoco.

   cp -r external/caffe/build/lib .
   

5. Copy other prebuilt libraries from the external folder

   cp external/caffe/build/lib/libcaffe.* lib/
   cp external/Bullet/bin/*.so lib/
   cp external/jsoncpp/build/debug/src/lib_json/*.so* lib/
   

6. Generate makefiles using premake4.

   premake4 clean
   premake4 gmake
   

7. Build all the targets!

   cd gmake
   make config=debug64
   

   Note: you can speed up the build by appending the -j8 flag to this last make command, where 8 here is the number of concurrent build threads that make will launch. Choose a number that makes sense based on the hardware resources you have available to you.

Note: There are some issues with the installation on Ubuntu 14.04. Some of the libraries have changed their location and name (see BVLC/caffe#2347 for a solution).

Windows

This setup has been tested on Windows 7 and 10 with visual studio 2013.

1. Download the library.zip file that contains almost all of the relevant pre compiled external libraries and source code.
2. Unpack this library in the same directory the project is located in. For example, DeepLoco/../.
3. You might need to install opengl/glu/GL headers. We have been using freeglut for this project. glew might already be included in library.zip.
4. You will need to copy some dll files from dynamic_lib.zip to the directory the project is compiled to. For example, optimizer/x64/Debug/. These files are needed by the framework during runtime.
5. Might need to create a folder in DeepLoco called "output", This is where temprary and current policies will be dumped.

Running The System

After the system has been build there are two executable files that server different purposes. The DeepLoco program is for visually simulating the a controller and DeepLoco_Optimize is for optimizing the parameters of some controller.

Examples:
To simulate a controller/character
./DeepLoco -arg_file= args/test_args.txt To simulate a controller/character with a specific policy
./DeepLoco_Optimizer -arg_file= args/opt_int_poli_hopper_eval.txt -policy_model= output/intermediate/trainer_int_model_0000160000.h5
To Train a controller
./DeepLoco_Optimizer -arg_file= args/opt_args_train.txt
./DeepLoco_Optimizer -arg_file= args/opt_args_train_hopper.txt
To Optimize a controllers parameters
./DeepLoco_Optimizer -arg_file= args/opt_args_jump.txt

Key Bindings

Most of these are toggles

• c fixed camera mode
• y draw COM path and contact locations
• q draw "filmstrip" like rendering
• f draw torques
• h draw Actor value functions?
• shift + '>' step one frame
• p toggle draw value function
• ',' and '.' change render speed, decrease and increase.
• "spacebar" to pause simulation
• r restart the scenario
• l reload the simulation (reparses the arg file)
• g draw state features

Contents of external dependencies folders

These lists are provided for reference only. Normally, if you follow the instructions above, you shouldn't need to know about any of this.

Linux

• caffe source code (must still be built)
  • Specific version (https://github.com/niuzhiheng/caffe.git @ 7b3e6f2341fe7374243ee0126f5cad1fa1e44e14)
  • In the instruction to make and build Caffe we uncomment the CPU only line

    # CPU-only switch (uncomment to build without GPU support).
    CPU_ONLY := 1
    

• BulletPhysics (This specific threadsafe version)
• Json_cpp
• Eigen
• CMA-ES
• LodePNG

Windows

• Caffe: https://github.com/initialneil/caffe-vs2013
• TODO: Finish documenting this

deeploco's Issues

External Files

Hi, I am really interested in this project, but when I build it, I didn't found external files, should I download from the link you mentioned?

Interface documentation

Hey,
I'm trying to understand your code, especially how and where you are interfacing with bullet and caffe.
I'd be glad, if you could answer me some questions:
Where do you evaluate the current net in the simulation?
Where do you get the value used for reward out of the simulation?
Where and how do you hand the reward back to the solver?
Thanks for the great work and your help.
Best wishes

shader compilation failed

I managed to build all the targets thanks to the previous issues.
Then, I tried to run the system with the command below.
./DeepLoco -arg_file= args/test_args.txt

However, I had the error

Compiling shader: render/shaders/FullScreenQuad_VS.glsl
shader compilation failed: render/shaders/FullScreenQuad_VS.glsl
0:1(1): error: syntax error, unexpected $end

Compiling shader: render/shaders/DownSample_PS.glsl
shader compilation failed: render/shaders/DownSample_PS.glsl
0:1(1): error: syntax error, unexpected $end

shader linking failed
Segmentation fault (core dumped)

`

NaN Output When running on Ubuntu 16.04

We just complied it as the instruction on Ubuntu 16.04, and run it with argument -arg_file= args/train_llc_args.txt. A NaN output will be got only after several iterations, as shown following.

`Actor Iter 4
Update Net 0:
I0808 12:46:07.273452 25515 solver.cpp:228] Iteration 10, loss = 0.0132303
I0808 12:46:07.273478 25515 solver.cpp:244] Train net output #0: loss = 0.0132303 (* 1 = 0.0132303 loss)
I0808 12:46:07.273486 25515 sgd_solver.cpp:106] Iteration 10, lr = 0.01
I0808 12:46:07.299680 25515 solver.cpp:228] Iteration 4, loss = inf
I0808 12:46:07.299707 25515 solver.cpp:244] Train net output #0: loss = inf (* 1 = inf loss)
I0808 12:46:07.299715 25515 sgd_solver.cpp:106] Iteration 4, lr = 0.001

Actor Iter 5
Update Net 0:
I0808 12:46:07.632088 25517 solver.cpp:228] Iteration 12, loss = 0.0219066
I0808 12:46:07.632109 25517 solver.cpp:244] Train net output #0: loss = 0.0219066 (* 1 = 0.0219066 loss)
I0808 12:46:07.632114 25517 sgd_solver.cpp:106] Iteration 12, lr = 0.01
I0808 12:46:07.651747 25517 solver.cpp:228] Iteration 5, loss = -nan
I0808 12:46:07.651769 25517 solver.cpp:244] Train net output #0: loss = -nan (* 1 = -nan loss)
I0808 12:46:07.651777 25517 sgd_solver.cpp:106] Iteration 5, lr = 0.001`

We tried it on different Linux machines. The output is similar. But on Windows 10, no NaN will be got.

About the CNN model

Hi,
I noticed that in your paper, you process the terrain map by there convolutional layers in HLC network, and your filters are 16 5×5, 32 4×4, and 32 3×3 respectively.

However, I found that in your source code (keras folder: deepLoco_model.py), the number of convolutional layers is 9, and the filters are 16 5×5, 16 5×5, 64 5×5, 64 5×5, 64 5×5, 256 3×3, 256 3×3, 256 3×3, 256 3×3 respectively.

So I would like to know is this file wrong? Maybe the right CNN model is not in this file?

Thanks in advance! @xbpeng @Neo-X

Any hint for creating a new HLC .h5 model file

Dear developers, recently I've been trying to train some new models based on your DeepLoco proj. However there's no existing hlc model for reference, i.e. the way to name datasets of an HLC model .h5 file. Can I get some hint from any part of the existing project? Many thanks.

build error

Hello, when I try to "make config=debug64"

I got this error

Linking DeepLoco
/usr/bin/ld: cannot find -lBulletDynamics_gmake_x64_debug
/usr/bin/ld: cannot find -lBulletCollision_gmake_x64_debug
/usr/bin/ld: cannot find -lLinearMath_gmake_x64_debug
collect2: error: ld returned 1 exit status
DeepLoco.make:130: recipe for target '../x64/Debug/DeepLoco' failed
make[1]: *** [../x64/Debug/DeepLoco] Error 1
Makefile:16: recipe for target 'DeepLoco' failed
make: *** [DeepLoco] Error 2
make: *** Waiting for unfinished jobs....
Linking DeepLoco_Optimizer
/usr/bin/ld: cannot find -lBulletDynamics_gmake_x64_debug
/usr/bin/ld: cannot find -lBulletCollision_gmake_x64_debug
/usr/bin/ld: cannot find -lLinearMath_gmake_x64_debug
collect2: error: ld returned 1 exit status
DeepLoco_Optimizer.make:143: recipe for target '../x64/Debug/DeepLoco_Optimizer' failed
make[1]: *** [../x64/Debug/DeepLoco_Optimizer] Error 1
Makefile:44: recipe for target 'DeepLoco_Optimizer' failed
make: *** [DeepLoco_Optimizer] Error 2

undefined reference to lodepng::decode

when do the command to compile "make config=debug64"
there occur the error as below, how to fix the it?

/usr/include/c++/5/backward/binders.h:108:11: note: declared here
class binder1st
^
../x64/Debug/libdeepLocoRender.a(TextureDesc.o): In function cTextureDesc::cTextureDesc(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)': /disk4/graphdev/RlRobSim/Test/DeepLoco-0.81/gmake/../render/TextureDesc.cpp:74: undefined reference to lodepng::decode(std::vector<unsigned char, std::allocator >&, unsigned int&, unsigned int&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, LodePNGColorType, unsigned int)'
collect2: error: ld returned 1 exit status
DeepLoco.make:129: recipe for target '../x64/Debug/DeepLoco' failed
make[1]: *** [../x64/Debug/DeepLoco] Error 1
Makefile:16: recipe for target 'DeepLoco' failed
make: *** [DeepLoco] Error 2
make: *** Waiting for unfinished jobs....
Linking DeepLoco_Optimizer

Cannot open final file on ubuntu16.04

I followed the steps to generate the Final document in the/DeepLoco-master/x64/Debug on ubuntu16.04, but I can't open them.so I try it at the terminal，but it tell me that ./DeepLoco: error while loading shared libraries: libBulletDynamics_gmake_x64_release.so: cannot open shared object file: No such file or directory.
I want to know if the location of the executable I generated is incorrect or my process is wrong.Thank you very much for your answer.

Where is the external folder?

Thanks for sharing this nice simulation. I am trying to install it, but can't find the external folder you are mentioning in the installation instructions. Where is it?

Issues running on Windows, blank screen no DeepLoco_Optimizer .exe

Hello, and thank you for creating this wonderful piece of software. I am very excited, an looking forward to playing with it. I followed your instructions and built DeepLoco for Windows 10. During the build process I ran into and fixed the following issues:

• The path to the Bullet sourcecode had changed. The real path in the library folder is ../Bullet/include instead of ../Bullet3/src.
• Similarly the Bullet libs had the wrong path for the linker.
• The boost libs also had the wrong path for the linker.

With these changes and using VS 2013 I am able to build DeepLoco. However I now have the following issues:

1. In the documentation on your github page you mention that there should be two executables. DeepLoco, and DeepLoco_Optimizer. The VS project only built one executable and the VS solution DeepLoco.sln only appears to create one executable. Am I missing something?

2. When I run the program DeepLoco using the command line below, a window opens up and displays FPS in the corner. I can toggle the commands described on the github page, but the window is pure white, nothing else is ever visible. Should there be something visible?:
   x64\Debug\DeepLoco.exe -arg_file=args/test_args.txt

Thanks again for the time and effort you have put into this.

Try it on window, but can not found Library.zip

What's the possible solution for the model loading failure?

I tired some weird ways to setup and compile the code, and finally executed the DeepLoco successfully. There is no error reported, but it seemed that the model could not be loaded , and the only things I could see were some squares like this:

My OpenGL version is 4.3, which is enough for this project, according to the ReadMe file.
Thx.

Issue occur when built on Windows by Visual Studio 2013

Hi, @xbpeng @Neo-X , I met with a serious issue when I compiled the project on Windows:

** The code execution cannot proceed because openblas.dll was not found. Reinstalling the program may fix this problem **

 I downloaded the binary file from https://sourceforge.net/projects/openblas/files/, and add libopenblas.dll to the same folder of .exe, aftere that, the popup sayed missing some other .dll files. I added the missing files using the same way as dealing with libopenblas.dll. But the project stiil cannnot work. It gave the message:

Error 0xc00007b “Application was unable to start correctly”

So could you give me some advice to solve the problem, thank you very much.

Error compile caffe with caffe_mods

First, thank you for opensourcing your project.

I have a problem compiling caffe with caffe_mods. When I clone the specific version (https://github.com/niuzhiheng/caffe.git @ 7b3e6f2341fe7374243ee0126f5cad1fa1e44e14), there is no folders layers under include/caffe to add:

• caffe_mods/fixed_euclidean_loss_layer.hpp
• caffe_mods/weighted_euclidean_loss_layer.hpp

So I suppose that you copied the README.md of https://github.com/xbpeng/DeepTerrainRL without modifying the caffe version.
Then I clone the official caffe https://github.com/BVLC/caffe, add files in caffe_mods and compile but I get the error:

CXX src/caffe/layers/infogain_loss_layer.cpp
src/caffe/layers/infogain_loss_layer.cpp: In instantiation of ‘void caffe::InfogainLossLayer::LayerSetUp(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = float]’:
src/caffe/layers/infogain_loss_layer.cpp:221:1: required from here
src/caffe/layers/infogain_loss_layer.cpp:18:3: error: ‘const class caffe::InfogainLossParameter’ has no member named ‘axis’
softmax_param->set_axis(this->layer_param_.infogain_loss_param().axis());
^
src/caffe/layers/infogain_loss_layer.cpp: In instantiation of ‘void caffe::InfogainLossLayer::Reshape(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = float]’:
src/caffe/layers/infogain_loss_layer.cpp:221:1: required from here
src/caffe/layers/infogain_loss_layer.cpp:55:18: error: ‘const class caffe::InfogainLossParameter’ has no member named ‘axis’
infogain_axis_ =
^
src/caffe/layers/infogain_loss_layer.cpp: In instantiation of ‘void caffe::InfogainLossLayer::LayerSetUp(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = double]’:
src/caffe/layers/infogain_loss_layer.cpp:221:1: required from here
src/caffe/layers/infogain_loss_layer.cpp:18:3: error: ‘const class caffe::InfogainLossParameter’ has no member named ‘axis’
softmax_param->set_axis(this->layer_param_.infogain_loss_param().axis());
^
src/caffe/layers/infogain_loss_layer.cpp: In instantiation of ‘void caffe::InfogainLossLayer::Reshape(const std::vector<caffe::Blob>&, const std::vector<caffe::Blob>&) [with Dtype = double]’:
src/caffe/layers/infogain_loss_layer.cpp:221:1: required from here
src/caffe/layers/infogain_loss_layer.cpp:55:18: error: ‘const class caffe::InfogainLossParameter’ has no member named ‘axis’
infogain_axis_ =
^
Makefile:581: recipe for target '.build_release/src/caffe/layers/infogain_loss_layer.o' failed
make: *** [.build_release/src/caffe/layers/infogain_loss_layer.o] Error 1

Can you precise the caffe version needed to be compiled with files in caffe_mods ? Thank you

Will you release soccer models ?

It would be very nice if we can have the soccer models to test. Thanks

How to train for running?

The codes seems to be quite sophisticated and I'm not familiar with the libraries used, so here's a quickie question:

I successfully trained given scenarios, but I cannot find a scenario to train 'running'.
How can the LLC be trained for running?

Build Instructions

1. Download the most recent compressed external file from the newest release.
2. Extract it and move into the DeepLoco directory. The top directory of the DeepLoco repository should now contain a directory called external, in addition to all the other directories that were there before.

From where to download compressed external file from newest release? What kind of compressed external files are these?

shader linking failed

Have you ever encountered this problem? is a cause also due to opengl? My OpenGL version is 4.5 and i use Quadro M4000 (NVIDIA) and OS is Ubuntu 16.04. I don't use OpenGL 3.3 needed oon intel HD Graphics. The small window whose title is TerrainRL was displayed but there is not displayed at all. The color of window is kept gray. When I pushed a key "q" on the small window, "Filmstrip mode enabled" is shown on the terminal. i think your logic itself is running in the back of the gray window. On my same machine, I tried your previous logic (Deep Terrain Learning). It was run and the good animation was displayed smoothly without any problem. I want to fix the window's condition being gray color. I runed from x64/Debug. I need your help.

~$ DeepLoco -arg_file=args/test_args.txt
Compiling shader: render/shaders/FullScreenQuad_VS.glsl
Compiling shader: render/shaders/DownSample_PS.glsl
shader linking failed
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501

Can't find args file for DeepLoco_Optimizer

Sorry but I wonder where these files are as you mentioned in the readme file.

Can't figure out where is Library.zip

I am trying to run this project on Windows so I am supposed to download libray.zip but I am not familiar where can I find it. Could you help me on this

No executable files

Thank you for creating the great tool. I use Ubuntu 16.04. Apart from some warning appeared in the "make config=debug64" command, I got no error information. But after I finish the instructions, I don't see the two executable files in any place.

Could you give me some suggestions? Thanks.

Best Regards.

undefined reference in ../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so

Thank you for your help in the previous issue.
However I still get another error when run the command make config=debug64

==== Building deepLocoUtil (debug64) ====
==== Building deepLocoAnim (debug64) ====
==== Building deepLocoLearning (debug64) ====
==== Building deepLocoSim (debug64) ====
==== Building deepLocoRender (debug64) ====
==== Building deepLocoScenarios (debug64) ====
==== Building DeepLoco (debug64) ====
Linking DeepLoco
../x64/Debug/libdeepLocoLearning.a(NeuralNet.o): 関数 `cNeuralNet::LoadScale(std::string const&)' 内:
/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:141: `Json::Value::operator[](std::string const&)' に対する定義されていない参照です
/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:144: `Json::Value::operator[](std::string const&)' に対する定義されていない参照です
/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:159: `Json::Value::operator[](std::string const&)' に対する定義されていない参照です
/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:162: `Json::Value::operator[](std::string const&)' に対する定義されていない参照です
/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:177: `Json::Value::operator[](std::string const&)' に対する定義されていない参照です
../x64/Debug/libdeepLocoLearning.a(NeuralNet.o):/home/kangaroo/LocomotionRL/DeepLoco/gmake/../learning/NeuralNet.cpp:180: `Json::Value::operator[](std::string const&)' に対する定義されていない参照がさらに続いています

[...]

../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::end() const@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(char const*)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(char const*) const@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::empty() const@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&&)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, std::allocator<char> const&)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(unsigned long, char, std::allocator<char> const&)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_data() const@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::insert(unsigned long, char const*)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::append(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string()@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator+=(char)@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::compare(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const@GLIBCXX_3.4.21' に対する定義されていない参照です
../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21' に対する定義されていない参照です
collect2: error: ld returned 1 exit status
make[1]: *** [../x64/Debug/DeepLoco] エラー 1
make: *** [DeepLoco] エラー 2

The error was written in Japanese, but for example

../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21' に対する定義されていない参照です

means

../external/jsoncpp/build/debug/src/lib_json/libjsoncpp.so: undefined reference to `std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long)@GLIBCXX_3.4.21' 

My OS environment is Ubuntu 14.04.

No opt_*.txt files in the folder named args

When I execute the code in terminal as follow:
./DeepLoco_Optimizer -arg_file= args/opt_int_poli_hopper_eval.txt -policy_model= output/intermediate/trainer_int_model_0000160000.h5 To Train a controller ./DeepLoco_Optimizer -arg_file= args/opt_args_train.txt ./DeepLoco_Optimizer -arg_file= args/opt_args_train_hopper.txt To Optimize a controllers parameters ./DeepLoco_Optimizer -arg_file= args/opt_args_jump.txt
The terminal show 'Failed to open......'

cannot find -lBulletDynamics_gmake_x64_debug

I am following the instruction in README.md to build instructions of your project.
I have successfully built the source code for caffe and finished copy prebuilt libraries from the external folder.
Then, I run the command make config=debug64.
However, the compilation stopped with this error:

==== Building deepLocoUtil (debug64) ====
==== Building deepLocoAnim (debug64) ====
==== Building deepLocoLearning (debug64) ====
==== Building deepLocoSim (debug64) ====
==== Building deepLocoRender (debug64) ====
==== Building deepLocoScenarios (debug64) ====
==== Building DeepLoco_Optimizer (debug64) ====
Linking DeepLoco_Optimizer
/usr/bin/ld: -lBulletDynamics_gmake_x64_debug が見つかりません
/usr/bin/ld: -lBulletCollision_gmake_x64_debug が見つかりません
/usr/bin/ld: -lLinearMath_gmake_x64_debug が見つかりません
collect2: error: ld returned 1 exit status
make[1]: *** [../x64/Debug/DeepLoco_Optimizer] エラー 1
make: *** [DeepLoco_Optimizer] エラー 2

(The error was written in Japanese, but it means

/usr/bin/ld: cannot find -lBulletDynamics_gmake_x64_debug
/usr/bin/ld: cannot find -lBulletCollision_gmake_x64_debug 
/usr/bin/ld: cannot find -lLinearMath_gmake_x64_debug 

)
My OS environment is Ubuntu 14.04.
Could anyone please help? Thanks alot for your time.

Definition of reference motion angles from mocap

Hello,

May I ask a question on the reference motion from mocap?

It looks like to me that the reference motion lies in the folder DeepLoco/data/motions/mocap/biped3d_walk/ as separate files of 28x30 matrices. If I am correct, then what does each row and each column stand for? From the paper the walker has 15 degree of freedom (3+3+1 for each leg and 1 for torso), it doesn't look like the dimension matches the degree of freedom (if each row or column represents one frame).

Thank you!

DeepLoco crashes

Hi, I managed to successfully build the project on my Ubuntu 16.04 and copied all files from x64/Debug to the root folder.

When I try to run DeepLoco using the command ./DeepLoco -arg_file= args/test_args.txt

The program crashes with Segmentation fault (core dumped) :

(for full output: log.txt )
...

I1110 23:59:07.444648 4474 net.cpp:274] Network initialization done.
Compiling shader: render/shaders/Mesh_VS.glsl
Compiling shader: render/shaders/Depth_PS.glsl
Compiling shader: render/shaders/Mesh_VS.glsl
Compiling shader: render/shaders/Lighting_Lambert_PS.glsl
Char Draw Shapes Disabled
Loaded scenario: Imitate Step Evaluation
*** Aborted at 1510358347 (unix time) try "date -d @1510358347" if you are using GNU date ***
PC: @ 0x64a872 btCollisionObject::getActivationState()
*** SIGSEGV (@0x7fbb000004ed) received by PID 4474 (TID 0x7fbb4cb92b00) from PID 1261; stack trace: ***
@ 0x7fbb4a09a4b0 (unknown)
@ 0x64a872 btCollisionObject::getActivationState()
@ 0x68725b btSimulationIslandManager::addManifoldsToIslands()
@ 0x687ac8 btSimulationIslandManager::buildAndProcessIslands()
@ 0x645b28 btDiscreteDynamicsWorld::solveConstraints()
@ 0x64507f btDiscreteDynamicsWorld::internalSingleStepSimulation()
@ 0x644e93 btDiscreteDynamicsWorld::stepSimulation()
@ 0x565b40 cWorld::Update()
@ 0x476ddd cScenarioSimChar::UpdateWorld()
@ 0x475311 cScenarioSimChar::Update()
@ 0x442a14 cScenarioExp::Update()
@ 0x48978c cScenarioExpImitateTarget::Update()
@ 0x4512f2 cDrawScenarioSimChar::UpdateScene()
@ 0x44bae5 cDrawScenarioTerrainRL::Update()
@ 0x450c28 cDrawScenarioSimChar::Update()
@ 0x425c8e UpdateScenario()
@ 0x426627 Update()
@ 0x42677a Animate()
@ 0x7fbb4af73894 glutMainLoopEvent
@ 0x7fbb4af74145 glutMainLoop
@ 0x42e72e main
@ 0x7fbb4a085830 __libc_start_main
@ 0x4104b9 _start
@ 0x0 (unknown)
Segmentation fault (core dumped)

Note: I installed pybullet using pip install pybullet since the build procedure wasn't able to find pre-built Bullet libraries (#5).

Windows Build Error

When trying to build the project on Windows, I end up with the following error (after following the installation steps and putting required librairies at the right places):

Error: LNK2038	mismatch detected for '_MSC_VER': value '1800' doesn't match value '1900' in blob.obj	DeepLoco	File: BulletDynamics_Debug.lib(btRigidBody.obj)	Line: 1	

which seems to indicate that Bullet was compiled for a different Visual Studio version.
I am using Visual Studio 2017, and tried using VS2015 but encountered another error related to Platform Toolset.

shader linking failed, Buffer Assignment Failed: 00000501

Platform

• OS: windows 10 64bit
• IDE: VS2013

Step

• cmd run DeepLoco.exe -arg_file=args/test_args.txt

Got error info

F:\tuibot\DeepLoco\x64\Debug>DeepLoco.exe  -arg_file=args/test_args.txt
Compiling shader: render/shaders/FullScreenQuad_VS.glsl
Compiling shader: render/shaders/DownSample_PS.glsl
shader linking failed
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501
Buffer Assignment Failed: 00000501

@xbpeng @Neo-X

DeepLoco Optimizer aborts

After a short time of running (tested arg-files train_hlc_args.txt and train_llc_args.txt) DeepLoco_Optimizer crashes:

Both files "data/policies/biped3d/nets/biped3d_hlc_critic_solver.prototxt" and "data/policies/biped3d/nets/biped3d_hlc_critic_net.prototxt" exist.

Here with train_hlc_args.txt:

I0405 09:43:25.671207  4578 net.cpp:84] Creating Layer output
I0405 09:43:25.671213  4578 net.cpp:406] output <- relu1
I0405 09:43:25.671224  4578 net.cpp:380] output -> output
I0405 09:43:25.671244  4578 net.cpp:122] Setting up output
I0405 09:43:25.671253  4578 net.cpp:129] Top shape: 32 1 (32)
I0405 09:43:25.671258  4578 net.cpp:137] Memory required for data: 13358976
I0405 09:43:25.671267  4578 layer_factory.hpp:77] Creating layer loss
F0405 09:43:25.671345  4578 layer_factory.hpp:81] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: WeightedEuclideanLoss (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, Concat, ContrastiveLoss, Convolution, Crop, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, Input, LRN, LSTM, LSTMUnit, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, Parameter, Pooling, Power, Python, RNN, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, Softmax, SoftmaxWithLoss, Split, TanH, Threshold, Tile, WindowData)
*** Check failure stack trace: ***

Thread 1 "DeepLoco_Optimi" received signal SIGABRT, Aborted.
0x00007ffff545e428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff545e428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff546002a in __GI_abort () at abort.c:89
#2  0x00007ffff675ae49 in ?? () from /usr/lib/x86_64-linux-gnu/libglog.so.0
#3  0x00007ffff675c5cd in google::LogMessage::Fail() () from /usr/lib/x86_64-linux-gnu/libglog.so.0
#4  0x00007ffff675e433 in google::LogMessage::SendToLog() () from /usr/lib/x86_64-linux-gnu/libglog.so.0
#5  0x00007ffff675c15b in google::LogMessage::Flush() () from /usr/lib/x86_64-linux-gnu/libglog.so.0
#6  0x00007ffff675ee1e in google::LogMessageFatal::~LogMessageFatal() ()
   from /usr/lib/x86_64-linux-gnu/libglog.so.0
#7  0x00007ffff6a9b614 in caffe::Net<float>::Init(caffe::NetParameter const&) ()
   from /home/dfki.uni-bremen.de/hwiedemann/deeploco-dev/DeepLoco/lib/libcaffe.so.1.0.0
#8  0x00007ffff6a9ca2e in caffe::Net<float>::Net(caffe::NetParameter const&) ()
   from /home/dfki.uni-bremen.de/hwiedemann/deeploco-dev/DeepLoco/lib/libcaffe.so.1.0.0
#9  0x00007ffff6c0eada in caffe::Solver<float>::InitTrainNet() ()
   from /home/dfki.uni-bremen.de/hwiedemann/deeploco-dev/DeepLoco/lib/libcaffe.so.1.0.0
#10 0x00007ffff6c100a5 in caffe::Solver<float>::Init(caffe::SolverParameter const&) ()
   from /home/dfki.uni-bremen.de/hwiedemann/deeploco-dev/DeepLoco/lib/libcaffe.so.1.0.0
#11 0x00007ffff6c103bf in caffe::Solver<float>::Solver(caffe::SolverParameter const&) ()
   from /home/dfki.uni-bremen.de/hwiedemann/deeploco-dev/DeepLoco/lib/libcaffe.so.1.0.0
#12 0x000000000049d14a in caffe::SGDSolver<float>::SGDSolver (this=0x9d3eae8, param=...)
    at ../external/caffe/include/caffe/sgd_solvers.hpp:19
#13 0x000000000049cc68 in cCaffeSolver<caffe::SGDSolver<float> >::cCaffeSolver (this=0x9d3eae0, param=...)
    at ../learning/NNSolver.h:27
#14 0x000000000049c651 in cNNSolver::BuildSolver (
    solver_file="data/policies/biped3d/nets/biped3d_hlc_critic_solver.prototxt", 
    out_solver=std::shared_ptr (empty) 0x0) at ../learning/NNSolver.cpp:17
#15 0x0000000000482349 in cNeuralNet::LoadSolver (this=0x8c70b80, 
    solver_file="data/policies/biped3d/nets/biped3d_hlc_critic_solver.prototxt") at ../learning/NeuralNet.cpp:116
#16 0x0000000000494fb4 in cNeuralNetTrainer::BuildNetPool (this=0x9a1c170, 
    net_file="data/policies/biped3d/nets/biped3d_hlc_critic_net.prototxt", 
    solver_file="data/policies/biped3d/nets/biped3d_hlc_critic_solver.prototxt", pool_size=1)
    at ../learning/NeuralNetTrainer.cpp:772
#17 0x000000000047a1f4 in cCaclaTrainer::BuildNetPool (this=0x9a1c170, 
    net_file="data/policies/biped3d/nets/biped3d_hlc_critic_net.prototxt", 
    solver_file="data/policies/biped3d/nets/biped3d_hlc_critic_solver.prototxt", pool_size=1)
    at ../learning/CaclaTrainer.cpp:140
#18 0x0000000000473860 in cACTrainer::BuildNets (this=0x9a1c170) at ../learning/ACTrainer.cpp:356
#19 0x0000000000492afc in cNeuralNetTrainer::Init (this=0x9a1c170, params=...)
    at ../learning/NeuralNetTrainer.cpp:97
#20 0x00000000004727fe in cACTrainer::Init (this=0x9a1c170, params=...) at ../learning/ACTrainer.cpp:29
#21 0x00000000004799a4 in cCaclaTrainer::Init (this=0x9a1c170, params=...) at ../learning/CaclaTrainer.cpp:24
#22 0x0000000000455857 in cScenarioTrain::InitTrainer (this=0xad2c90) at ../scenarios/ScenarioTrain.cpp:368
#23 0x000000000046f04e in cScenarioTrainCacla::InitTrainer (this=0xad2c90)
    at ../scenarios/ScenarioTrainCacla.cpp:61
#24 0x000000000042ef9f in cScenarioTrainHike::InitTrainer (this=0xad2c90)
    at ../scenarios/ScenarioTrainHike.cpp:138
#25 0x00000000004549e2 in cScenarioTrain::Init (this=0xad2c90) at ../scenarios/ScenarioTrain.cpp:124
#26 0x000000000042eaea in cScenarioTrainHike::Init (this=0xad2c90) at ../scenarios/ScenarioTrainHike.cpp:73
#27 0x000000000042af7f in SetupScenario () at ../optimizer/Main.cpp:131
#28 0x000000000042b2bc in main (argc=3, argv=0x7fffffffdf38) at ../optimizer/Main.cpp:247

Thanks in advance!