svalorzen / ai-toolbox Goto Github PK
View Code? Open in Web Editor NEWA C++ framework for MDPs and POMDPs with Python bindings
License: GNU General Public License v3.0
A C++ framework for MDPs and POMDPs with Python bindings
License: GNU General Public License v3.0
At the time of writing, it's sometimes difficult to figure out how the generated Python objects work. This is partially due to their hard-to-read boostpython-generated docs:
In [8]: q = MDP.QLearning()
---------------------------------------------------------------------------
ArgumentError Traceback (most recent call last)
<ipython-input-8-de58c583951f> in <module>()
----> 1 q = MDP.QLearning()
ArgumentError: Python argument types in
QLearning.__init__(QLearning)
did not match C++ signature:
__init__(_object*, AIToolbox::MDP::SparseModel)
__init__(_object*, AIToolbox::MDP::SparseModel, double)
__init__(_object*, AIToolbox::MDP::Model)
__init__(_object*, AIToolbox::MDP::Model, double)
__init__(_object*, AIToolbox::MDP::SparseRLModel<AIToolbox::MDP::SparseExperience, void>)
__init__(_object*, AIToolbox::MDP::SparseRLModel<AIToolbox::MDP::SparseExperience, void>, double)
__init__(_object*, AIToolbox::MDP::RLModel<AIToolbox::MDP::Experience, void>)
__init__(_object*, AIToolbox::MDP::RLModel<AIToolbox::MDP::Experience, void>, double)
__init__(_object*, unsigned long, unsigned long)
__init__(_object*, unsigned long, unsigned long, double)
__init__(_object*, unsigned long, unsigned long, double, double)
Adding an example - like the gridworld example from the docs - will make them simpler to understand. This will be significantly less time-consuming than writing docstrings. It would be nice to show how the ValueIterationModel
relates to a Model
instance, how to properly instantiate them and run the solver.
For example, I'm having trouble constructing the Python equivalent of:
AIToolbox::MDP::ValueIteration solver;
auto solution = solver(world);
Due to a Python version mismatch, cmake 2.8 can not find the python binary:
auke@auke-GS60-2QE:~/AI-Toolbox/build$ cmake -DCMAKE_BUILD_TYPE=Release ..
-- Boost version: 1.54.0
CMake Error at /usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:108 (message):
Could NOT find PythonLibs: Found unsuitable version "2.7.6", but required
is exact version "2.7" (found /usr/lib/x86_64-linux-gnu/libpython2.7.so)
Call Stack (most recent call first):
/usr/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:313 (_FPHSA_FAILURE_MESSAGE)
/usr/share/cmake-2.8/Modules/FindPythonLibs.cmake:208 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
src/CMakeLists.txt:10 (find_package)
-- Configuring incomplete, errors occurred!
See also "/home/auke/AI-Toolbox/build/CMakeFiles/CMakeOutput.log".
This issue can be solved by upgrading to cmake >3.0, as specified in this stackoverflow answer.
What cmake version are you using? Perhaps you can specify that cmake >3 is needed, or alternatively include a 'known issues' tab in the readme?
Hey @Svalorzen ,
what's your take regarding the addition of more examples in the examples
folder? Right now there in only an MDP code, but also a POMDP example would have been nice.
Transition functions using S,S,A notation are not standard.
A transition function T is SxA -> Pr(S) thus involving a S,A,S notation.
Moreover, to sample in the model from a pair <s,a>, you will be able to use a discrete_distribution on T[s][a] which will directly be an array (no loop required to construct the distribution).
Currently with the time I have I generally implement the simpler algorithms that I come across and that give basic solving capabilities. It would be nice to have a complete list of methods that are both simple to understand and powerful in order to have a TODO list to make the library more relevant.
Does the library provide any facilities for generalisation and value function approximation ? (e.g. simple Q-Learning with linear function approximation)
In order to use AI-Toolbox files in my programs, the only way I've been able to get them to work so far is by adding in my program files within one of the AI-Toolbox folders and adjusting the applicable CMakeLists.txt file before rebuilding the project. I've been looking to use some of the files with an OMNeT++ project, but attempting to use the AI-Toolbox files with that makes it very complicated if I try to just move an OMNeT++ project into the AI-Toolbox folders, because it seems that I would have to change several file paths in potentially hundreds of files.
Would you would happen to have an idea of any simpler way to integrate AI-Toolbox with OMNeT++? Thank you!
Hello,
I have a use case that might be of interest to you.
My set of allowed actions is not stable, meaning that for each state, the set of actions that can be performed changes. In addition, the total number of actions can't be represented with an unsigned int, or at least not easily (it's a combinatorial problem).
So what I've done is to take the code in MCTS.hpp/cpp and change the std::vector<ActionNode>
into a map. I also had to change some places like the _graph->children.resize(A)
.
That being said, what do you think about delegating the creation of the ActionNode
s (and potentially the StateNode
s as well) to a function, so just extending the MCTS and overriding this function would allow for a different ActionNode. On top of that, a bit on templating on the class for the container ActionNodes
would work as well, by default it would be <std::vector<ActionNode>, unsigned int>
where the second item is the type of the index when calling children.at()
.
In my case it would be <std::map<boost::dynamic_bitset, ActionNode>, boost::dynamic_bitset>
.
This + a bit more of inheritance would allow for removing a lot of duplication between MCTS and POMCP.
I can't have a go before March but if you fancy starting earlier, I can work on MCTS, model and POMCP because that's the classes I know best but I won't be able to test the whole lot or dig elsewhere for side effects.
Cheers,
Emmanuel
I'm currently trying to find out which library would be best to use in order to solve linear programming equations. Problems that require to find solutions over continuous spaces (e.g. POMDPs) need this type of tools.
Currently my main choice is the C library lp_solve, though a pure C++ library would probably be better. I'm still not sure whether uBLAS provides this type of facilities, or other libraries that we could use.
Serialization for sparse models at the moment does the exact same thing as for dense models: it outputs all entries into a stream sequentially without checking whether they are zero or not.
Ideally the serialization output would look more like a sparse matrix, i.e. a series of index+value tuples.
Create a new interface MDP::ModelInterface, and implement RLModel in terms of Model, not public inheritance.
I have finished building the library, and got the AIToolbox.dylib file generated. Now what should I do to import the module to python scripts? I set -DMAKE_PYTHON=1 in the cmake step, and now when i open python CLI, it still says "NameError: name 'AIToolbox' is not defined"
Each implemented algorithm should provide setVerbosity``getVerbosity
methods in order to allow the user to have some insight as to what is going on during long computations.
Is it possible to extract the expected value of the reward accrued up to a specific time step? I'm modeling the effect of different drugs on MS patients and would like to be able to extract the accumulated reward (quality adjusted life years) after n time steps (years). Thank you.
I tried running the SysAdmin example but got this error:
what(): Could not solve the LP for this MDP
This is my main program:
#include <iostream>
#include <iomanip>
#include <fstream>
#include <array>
#include <cmath>
#include <chrono>
#include <thread>
#include <algorithm>
#include <AIToolbox/MDP/Algorithms/ValueIteration.hpp>
#include <AIToolbox/MDP/Policies/Policy.hpp>
#include <AIToolbox/MDP/IO.hpp>
#include <AIToolbox/MDP/SparseModel.hpp>
#include <AIToolbox/Factored/MDP/Environments/SysAdmin.hpp>
#include <AIToolbox/Utils/Core.hpp>
#include <AIToolbox/Factored/MDP/Algorithms/LinearProgramming.hpp>
#include <AIToolbox/Factored/MDP/Policies/QGreedyPolicy.hpp>
constexpr unsigned agents = 4;
constexpr double pFailBase = 0;
constexpr double pFailBonus = 0;
constexpr double pDeadBase = 0;
constexpr double pDeadBonus = 0;
constexpr double pLoad = 1.0;
constexpr double pDoneG = 1.0;
constexpr double pDoneF = 1.0;
int main() {
AIToolbox::Factored::MDP::CooperativeModel model =
AIToolbox::Factored::MDP::makeSysAdminUniRing(
agents,
pFailBase,
pFailBonus,
pDeadBase,
pDeadBonus,
pLoad,
pDoneG,
pDoneF);
AIToolbox::Factored::FactoredVector h;
for (uint i = 0; i < 3; i++) {
AIToolbox::Vector values = AIToolbox::Vector::Zero(model.getS().size(), 1);
values[i] = 1.0;
AIToolbox::Factored::BasisFunction b{{i}, values};
h.bases.emplace_back(std::move(b));
}
AIToolbox::Factored::MDP::LinearProgramming solver;
std::tuple<AIToolbox::Vector, AIToolbox::Factored::MDP::QFunction> solution = solver(model, h);
AIToolbox::Factored::MDP::QGreedyPolicy policy(
model.getS(),
model.getA(),
std::get<AIToolbox::Factored::MDP::QFunction>(solution));
// Start state is where all agents are Good
AIToolbox::Factored::State s(model.getS().size());
std::fill(std::begin(s), std::end(s), AIToolbox::Factored::MDP::SysAdminEnums::MachineStatus::Good);
AIToolbox::Factored::Action a(model.getA().size());
std::fill(std::begin(a), std::end(a), 0);
AIToolbox::Factored::State s1;
double r, totalReward = 0.0;
size_t t = 100;
while (true) {
// Print it!
std::cout << AIToolbox::Factored::MDP::printSysAdminRing(s) << std::endl;
// We give a time limit
if (t == 0) break;
// We sample an action for this state according to the optimal policy
a = policy.sampleAction(s);
// And we use the model to simulate what is going to happen next (in
// case of a "real world" scenario where the library is used this step
// would not exist as the world would automatically step to the next
// state. Here we simulate.
std::tie(s1, r) = model.sampleSR(s, a);
// Add into the total reward (we don't use this here, it's just as an
// example)
totalReward += r;
std::cout << "IMMEDIATE REWARD: " << r << ", TOTAL REWARD: " << totalReward << std::endl;
// Update the current state with the new one.
s = s1;
--t;
// Sleep 1 second so the user can see what is happening.
std::this_thread::sleep_for(std::chrono::seconds(1));
}
return 0;
}
I suspect that my basis functions might be defined wrongly (lines 40 to
45) but I am not sure. If you have some example usage code (such as the
one above) for FMDP that you can share, that could be really
helpful.
Hello,
I referred to your tiger_door project to create my own POMDP model that involves human behavior modeling. I noticed that in this example, the rewards are dependent only on the current state and action. In my case, however, the rewards are determined by the last state and the current state.
I noticed that the function getRewardFunction() returns a matrix that has S rows and A columns. What I'm looking for though is a reward function that is the same for all actions, depending only on the change in states- meaning it would have S rows and S columns.
Is there a different function that I need to use to implement such a scenario?
Hey @Svalorzen ,
what is your plan w.r.t. multi-agent MDPs? do you plan on adding them at some point?
Hello, I created the folder "build" and have installed Lp-solve by "sudo apt install lp-solve", but when I tried to run cmake .., I was told "could not find LpSolve", would you please tell me how to fix this error? Thank you very much!
My system is Ubuntu 20.04.
By the way, boost and eigent3 can be found.
I'm not sure if it is a issue or if it is even supposed to be a feature but I haven't been able to create a POMDP model with sparse transitions, rewards or observations.
I'm working on a pursuit domain environment and since the states scale so much with the size of the problem I wanted to try and use some sparse matrix's since most of the entries are empty anyway but I haven't been able to do so.
Would appreciate any kind help on the problem, thank you for your time
PS: I'm using the Python version
Edit : I realize now that POMDP.SparseModel exists but still don't understand why only the observation matrix is sparse in this model since in MDP.SparseModel the transitions and reward matrix seems to be sparse as well
For some reason when I started the library I had the idea that an epsilon(-greedy) policy was a policy that did the normal action epsilon percent of the time, and a random action (1-epsilon) percent of the time.
Unfortunately, this is exactly the opposite as everyone else does things. As many other small incongruities that have been hammered down during the life of this project, I think the time has come to fix this one too. This will probably result in some mess in client's code, but I think it is a big enough wart to warrant fixing.
I'll start up a branch for this in the coming days. Let me know if you have any comments, since I'm doing this might as well do it in a way that makes everyone happy :)
This is more for the future since it may require lots of tweaking to get right, and to check which data structures that the code currently uses are thread-safe or not, but many algorithms could definitely use multi-threading to speed up performance. OpenMP would be my favorite choice here.
Hello again,
Sorry to be bothering you with this simple error but I installed a new ubuntu environment and I can't seem to compile the library. I've tried with a old version of the library and it worked but the current master branch doesn't seem to work.
The problem occurs after the "cmake .." when I try to run make I have an error on that goes:
[ 2%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Utils/Probability.cpp.o
In file included from /home/goldencrow/Documents/secondstest/AI-Toolbox-master/include/AIToolbox/Utils/Probability.hpp:8,
from /home/goldencrow/Documents/secondstest/AI-Toolbox-master/src/Utils/Probability.cpp:1:
/home/goldencrow/Documents/secondstest/AI-Toolbox-master/include/AIToolbox/Utils/Core.hpp:6:10: fatal error: compare: No such file or directory
6 | #include <compare>
| ^~~~~~~~~
compilation terminated.
make[2]: *** [src/CMakeFiles/AIToolboxMDP.dir/build.make:128: src/CMakeFiles/AIToolboxMDP.dir/Utils/Probability.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1140: src/CMakeFiles/AIToolboxMDP.dir/all] Error 2
make: *** [Makefile:95: all] Error 2
I'm not sure if I'm missing any dependency or something but I'm sure its a problem of my setup, I just can't get my finger on what could be missing.
EDIT: My gcc version is 9.3.0. GNU make is 4.2.1 and cmake 3.16.3
EDIT2: I've noticed that the missing library "" is from c++20 utility, could it be that I'm missing something on my system?
Thank you for your time,
Cassandro Martinho
Building on Windows 10.
Having trouble getting AI-Toolbox to recognize LP Solve.
We followed the other issue #32 suggestions but it still doesn't work.
-- Found Boost: C:/MinGW/include (found suitable version "1.71.0", minimum required is "1.59")
-- Found Eigen3: C:/Program Files/eigen3 (Required is at least version "3.2.92")
CMake Error at C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:164 (message):
Could NOT find LpSolve (missing: LPSOLVE_LIBRARIES)
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.17/Modules/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
cmake/Modules/FindLpSolve.cmake:91 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:151 (find_package)
Here is what we are running to build:
rm -rf build
mkdir build
cd build
cmake .. \
-DLPSOLVE_LIBRARIES="C:\Program Files\lpsolve"\
-DLPSOLVE_INCLUDE_PATH="C:\Program Files\lpsolve"\
-DLPSOLVE_LINKS="C:\Program Files\lpsolve"\
-A x64\
Hello after running make I get the following error:
In file included from /home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:1:
/home/jpk/Desktop/AI-Toolbox-master/include/AIToolbox/Factored/MDP/Algorithms/Utils/FactoredLP.hpp:76:18: error: ‘optional’ in namespace ‘std’ does not name a template type
76 | std::optional<Vector> operator()(const FactoredVector & C, const FactoredVector & b, bool addConstantBasis = false);
| ^~~~~~~~
/home/jpk/Desktop/AI-Toolbox-master/include/AIToolbox/Factored/MDP/Algorithms/Utils/FactoredLP.hpp:7:1: note: ‘std::optional’ is defined in header ‘<optional>’; did you forget to ‘#include <optional>’?
6 | #include <AIToolbox/Factored/MDP/Types.hpp>
+++ |+#include <optional>
7 |
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:32:27: error: no declaration matches ‘std::optional<Eigen::Matrix<double, -1, 1> > AIToolbox::Factored::MDP::FactoredLP::operator()(const AIToolbox::Factored::FactoredVector&, const AIToolbox::Factored::FactoredVector&, bool)’
32 | std::optional<Vector> FactoredLP::operator()(const FactoredVector & C, const FactoredVector & b, bool addConstantBasis) {
| ^~~~~~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:32:27: note: no functions named ‘std::optional<Eigen::Matrix<double, -1, 1> > AIToolbox::Factored::MDP::FactoredLP::operator()(const AIToolbox::Factored::FactoredVector&, const AIToolbox::Factored::FactoredVector&, bool)’
In file included from /home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:1:
/home/jpk/Desktop/AI-Toolbox-master/include/AIToolbox/Factored/MDP/Algorithms/Utils/FactoredLP.hpp:35:11: note: ‘class AIToolbox::Factored::MDP::FactoredLP’ defined here
35 | class FactoredLP {
| ^~~~~~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:198:10: warning: ‘void AIToolbox::Factored::MDP::{anonymous}::Global::makeResult(AIToolbox::Factored::GenericVariableElimination<long unsigned int>::FinalFactors&&)’ defined but not used [-Wunused-function]
198 | void Global::makeResult(VE::FinalFactors && finalFactors) {
| ^~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:183:10: warning: ‘void AIToolbox::Factored::MDP::{anonymous}::Global::endCrossSum()’ defined but not used [-Wunused-function]
183 | void Global::endCrossSum() {
| ^~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:178:10: warning: ‘void AIToolbox::Factored::MDP::{anonymous}::Global::crossSum(const Factor&)’ defined but not used [-Wunused-function]
178 | void Global::crossSum(const Factor & f) {
| ^~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:168:10: warning: ‘void AIToolbox::Factored::MDP::{anonymous}::Global::beginCrossSum()’ defined but not used [-Wunused-function]
168 | void Global::beginCrossSum() {
| ^~~~~~
/home/jpk/Desktop/AI-Toolbox-master/src/Factored/MDP/Algorithms/Utils/FactoredLP.cpp:159:10: warning: ‘void AIToolbox::Factored::MDP::{anonymous}::Global::initNewFactor()’ defined but not used [-Wunused-function]
159 | void Global::initNewFactor() {
| ^~~~~~
make[2]: *** [src/CMakeFiles/AIToolboxFMDP.dir/build.make:459: src/CMakeFiles/AIToolboxFMDP.dir/Factored/MDP/Algorithms/Utils/FactoredLP.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1109: src/CMakeFiles/AIToolboxFMDP.dir/all] Error 2
make: *** [Makefile:114: all] Error 2
^~~~~~~~
How can I overcome this?
I'm running Ubuntu 21.10 and I appear to have the required g++-10(g++ (Ubuntu 11.2.0-7ubuntu2) 11.2.0
), eigen, boost and lpsolve (installed as lpsolve dev)
Hey, thanks for the work on this library, its come in very handy.
One question, and I suppose feature request: do you have any plans on implementing log probabilities in the MDP models to improve numerical stability? I've run into small rounding errors and I wonder if using logs might help here.
I can look into it myself and make a pull request, but my C++ skills aren't quite on your level so there might be massive inefficiencies
Hello,
I followed what was done in #35. I wanted to use python37, so I changed
For Python3.7, change COMPONENTS python3 REQUIRED to COMPONENTS python37 REQUIRED (line 167 of CMakeList.txt) and set(BOOST_PYTHON_LIBRARY_NAME "Boost_PYTHON37_LIBRARY") (line 168 of CMakeList.txt)
I also made /usr/local/include/lpsolve and copied all the lp_*.h into it. I also used the bash file cedavidyang provided.
This allowed cmake to succeed. Here is the output:
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- IPO / LTO enabled
Build type: Release
Logging is DISABLED
# Building MDP
# Building Factored MDP
# Building POMDP
# Building Python bindings
# Building Tests
# Building Examples
-- Boost version: 1.70.0
-- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "3.2.92")
-- Performing Test LPSOLVE_LINKS_ALONE
-- Performing Test LPSOLVE_LINKS_ALONE - Success
-- Found LpSolve: /usr/local/lib/liblpsolve55.dylib
-- Found PythonInterp: /anaconda/bin/python (found version "3.7.3")
-- Found PythonLibs: /usr/local/Frameworks/Python.framework/Versions/3.7/lib/libpython3.7.dylib (found suitable version "3.7.5", minimum required is "3")
-- Boost version: 1.70.0
-- Found the following Boost libraries:
-- python37
-- Boost version: 1.70.0
-- Found the following Boost libraries:
-- unit_test_framework
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/Adam/Documents/UofT/Thesis/code/AI-Toolbox/build`
And then used make and got the error:
Scanning dependencies of target AIToolboxMDP
[ 0%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Impl/Seeder.cpp.o
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Impl/CassandraParser.cpp.o
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Utils/Combinatorics.cpp.o
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Utils/Probability.cpp.o
[ 2%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Utils/Polytope.cpp.o
[ 2%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Utils/LP/LpSolveWrapper.cpp.o
[ 2%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Tools/Statistics.cpp.o
[ 3%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Algorithms/RollingAverage.cpp.o
[ 3%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/EpsilonPolicy.cpp.o
[ 3%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/QGreedyPolicy.cpp.o
[ 4%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/QSoftmaxPolicy.cpp.o
[ 4%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/ThompsonSamplingPolicy.cpp.o
[ 5%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/LRPPolicy.cpp.o
[ 5%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Bandit/Policies/ESRLPolicy.cpp.o
[ 5%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Experience.cpp.o
[ 6%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Utils.cpp.o
[ 6%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Model.cpp.o
[ 6%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/SparseExperience.cpp.o
[ 7%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/SparseModel.cpp.o
[ 7%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/IO.cpp.o
[ 7%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/QLearning.cpp.o
[ 8%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/HystereticQLearning.cpp.o
[ 8%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/SARSA.cpp.o
[ 8%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/ExpectedSARSA.cpp.o
[ 9%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/SARSAL.cpp.o
[ 9%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/ValueIteration.cpp.o
[ 9%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/PolicyIteration.cpp.o
[ 10%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Algorithms/Utils/OffPolicyTemplate.cpp.o
[ 10%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/PolicyWrapper.cpp.o
[ 10%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/Policy.cpp.o
[ 11%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/RandomPolicy.cpp.o
[ 11%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/EpsilonPolicy.cpp.o
[ 11%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/QPolicyInterface.cpp.o
[ 12%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/QGreedyPolicy.cpp.o
[ 12%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/QSoftmaxPolicy.cpp.o
[ 12%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/WoLFPolicy.cpp.o
[ 13%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Policies/PGAAPPPolicy.cpp.o
[ 13%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Environments/Utils/GridWorld.cpp.o
[ 13%] Linking CXX static library ../libAIToolboxMDP.a
[ 13%] Built target AIToolboxMDP
Scanning dependencies of target AIToolboxPOMDP
[ 14%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Utils.cpp.o
[ 14%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/IO.cpp.o
[ 14%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/AMDP.cpp.o
[ 15%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/GapMin.cpp.o
[ 15%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/IncrementalPruning.cpp.o
[ 15%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/LinearSupport.cpp.o
[ 16%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/PBVI.cpp.o
[ 16%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/PERSEUS.cpp.o
[ 16%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/BlindStrategies.cpp.o
[ 17%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/FastInformedBound.cpp.o
[ 17%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/QMDP.cpp.o
[ 17%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Algorithms/Witness.cpp.o
[ 18%] Building CXX object src/CMakeFiles/AIToolboxPOMDP.dir/POMDP/Policies/Policy.cpp.o
[ 18%] Linking CXX static library ../libAIToolboxPOMDP.a
[ 18%] Built target AIToolboxPOMDP
Scanning dependencies of target AIToolboxFMDP
[ 19%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/FactoredContainer.cpp.o
[ 19%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/Core.cpp.o
[ 19%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/FactoredMatrix.cpp.o
[ 20%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/FactoredVectorOps.cpp.o
[ 20%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/FactoredMatrix2DOps.cpp.o
[ 20%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Utils/BayesianNetwork.cpp.o
[ 21%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Bandit/Algorithms/Utils/VariableElimination.cpp.o
[ 21%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Bandit/Algorithms/Utils/MultiObjectiveVariableElimination.cpp.o
[ 21%] Building CXX object src/CMakeFiles/AIToolboxFMDP.dir/Factored/Bandit/Algorithms/Utils/UCVE.cpp.o
/Users/Adam/Documents/UofT/Thesis/code/AI-Toolbox/src/Factored/Bandit/Algorithms/Utils/UCVE.cpp:162:55: error:
'max' in capture list does not name a variable
std::remove_if(begin + 1, bound, [max, x_u = x_u, logtA12 = logtA12...
^
/Users/Adam/Documents/UofT/Thesis/code/AI-Toolbox/src/Factored/Bandit/Algorithms/Utils/UCVE.cpp:162:155: error:
reference to local binding 'max' declared in enclosing function
'AIToolbox::Factored::Bandit::(anonymous namespace)::Global::endFactorCrossSum'
...= logtA12](const UCVE::Entry & e) { return computeValue(e, x_u, logtA12) <= max; }),
^
/Users/Adam/Documents/UofT/Thesis/code/AI-Toolbox/src/Factored/Bandit/Algorithms/Utils/UCVE.cpp:156:30: note:
'max' declared here
auto [maxIt, max] = max_element_unary(begin, bound, [x_l = x_l, logtA1...
^
2 errors generated.
make[2]: *** [src/CMakeFiles/AIToolboxFMDP.dir/Factored/Bandit/Algorithms/Utils/UCVE.cpp.o] Error 1
make[1]: *** [src/CMakeFiles/AIToolboxFMDP.dir/all] Error 2
make: *** [all] Error 2
I'm not the best at using CMake, so now I'm a little confused about how to proceed. I also tried to not build FMDP using
cmake -DPYTHON_LIBRARY=$PYTHON_LIBRARY \
-DPYTHON_INCLUDE_DIR=$PYTHON_INCLUDE_DIR -DMAKE_LIB=1 \
-DMAKE_MDP=1 -DMAKE_POMDP=1 -DMAKE_TESTS=1 -DMAKE_EXAMPLES=1 ..
But I received the same error. Any thoughts?
After compiling, I get the following error when trying to run the tiger_antelope.py in the examples/MDP directory:
18:30:32 - Constructing MDP...
Traceback (most recent call last):
File "tiger_antelope.py", line 341, in
solve_mdp(horizon=args.horizon, tolerance=args.tolerance)
File "tiger_antelope.py", line 289, in solve_mdp
model.setTransitionFunction(T)
ValueError: Input transition matrix does not contain valid probabilities.
When I follow the build instructions, I can run the first three commands but then get an error after entering 'make'. I'm using an Ubuntu virtual machine and have all of the requirements listed in the dependencies section (including g++-10).
The error message that comes up is the following:
Scanning dependencies of target AIToolboxMDP
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Impl/Seeder.cpp.o
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Impl/CassandraParser.cpp.o
In file included from /home/ben/AI/AI-Toolbox-master/include/AIToolbox/Impl/CassandraParser.hpp:4,
from /home/ben/AI/AI-Toolbox-master/src/Impl/CassandraParser.cpp:1:
/home/ben/AI/AI-Toolbox-master/include/AIToolbox/Types.hpp:7:10: fatal error: boost/multi_array.hpp: No such file or directory
7 | #include <boost/multi_array.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [src/CMakeFiles/AIToolboxMDP.dir/build.make:76: src/CMakeFiles/AIToolboxMDP.dir/Impl/CassandraParser.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1140: src/CMakeFiles/AIToolboxMDP.dir/all] Error 2
make: *** [Makefile:95: all] Error 2
cmake was able to find Boost and I can see that the multi_array.hpp file is in the folder /home/ben/AI/boost_1_77_0/boost, so I'm not sure why make can't find the file.
I also have to add several options to the cmake command I run right before make in order to get rid of any errors with that; here's what I enter in case that would do anything or if I need to add something else in there:
cmake .. -DBOOST_ROOT=/home/ben/AI/boost_1_77_0 -DLPSOLVE_INCLUDE_PATH=/home/ben/AI/lpsolve -DVCPKG_TARGET_TRIPLET=x64-linux -DCMAKE_TOOLCHAIN_FILE=/home/ben/AI/vcpkg/scripts/buildsystems/vcpkg.cmake
Would anyone have any ideas as to how I can get past the make error?
I just compiled the latest version of the toolbox (cmake 3.5.1, g++ 7.1.0). Despite of the huge amount of warnings during the compilation, it apparently succeeded, as all the unit tests completed successfully.
I am now trying the "tiger_antelope" example, but the behavior looks a bit strange to me.
Am I expected to see this behavior?
Also, is it normal that the Python version of this example is SO much slower than the C++ version?
The 8x8 grid example is solved almost instantly with C++, but it takes forever with Python.
I am unable to use Python module on macOS (I did manage to use Python module on an Ubuntu machine following the instructions on README.md)
Among the dependencies, I used homebrew to install cmake, boost, boost-build, boost-python, and eigen3; lp_solve is downloaded from sourceforge.
Since I use Python installed by homebrew, I used -DPYTHON_LIBRARY
and -DPYTHON_INCLUDE_DIR
to direct cmake to homebrew versions of libpython2.7.dylib and Headers.
To expose Python to CMakeList, I also changed find_package(Boost ${BOOST_VERSION_REQUIRED} COMPONENTS python REQUIRED)
to find_package(Boost ${BOOST_VERSION_REQUIRED} COMPONENTS python27 REQUIRED)
and set(BOOST_PYTHON_LIBRARY_NAME "Boost_PYTHON_LIBRARY")
to set(BOOST_PYTHON_LIBRARY_NAME "Boost_PYTHON27_LIBRARY")
By using g++ shipped with macOS (clang-1000.10.44.4), I can cmake && make. C++ examples can run without a problem. However, Python examples are not working. Specifically, I cannot "import AIToolbox" even through I have AIToolbox.dylib compiled successfully.
Ways I have tried so far to solve this problem:
-DCMAKE_C_COMPILER=/usr/local/bin/gcc-7 -DCMAKE_CXX_COMPILER=/usr/local/bin/g++-7
to force cmake use g++-7export PATH="/usr/local/opt/make/libexec/gnubin:$PATH
to force cmake use gmake export LD_LIBRARY_PATH=/usr/local/Cellar/gcc\@7/7.4.0/lib/gcc/7/:/usr/local/lib/:/usr/lib/:$LD_LIBRARY_PATH
to direct python to the gcc-7 libstdc++Overall, my script for build is as follows:
#!/bin/bash
export LD_LIBRARY_PATH=/usr/local/Cellar/gcc\@7/7.4.0/lib/gcc/7/:/usr/local/lib/:/usr/lib/:$LD_LIBRARY_PATH
mkdir build
cd build
PYTHON_LIBRARY=/usr/local/Cellar/python\@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib
PYTHON_INCLUDE_DIR=/usr/local/Cellar/python\@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/Headers/
cmake -DCMAKE_C_COMPILER=/usr/local/bin/gcc-7 -DCMAKE_CXX_COMPILER=/usr/local/bin/g++-7 -DPYTHON_LIBRARY=$PYTHON_LIBRARY -DPYTHON_INCLUDE_DIR=$PYTHON_INCLUDE_DIR -MAKE_LIB=1 -MAKE_PYTHON=1 ..
By doing so, I am able to use g++-7 as required. However, I came across linking errors both during cmake and make (stopped at 47%). The details are as follows:
Upon using the script (i.e. cmake), the output is
-- The CXX compiler identification is GNU 7.4.0
-- Checking whether CXX compiler has -isysroot
-- Checking whether CXX compiler has -isysroot - yes
-- Checking whether CXX compiler supports OSX deployment target flag
-- Checking whether CXX compiler supports OSX deployment target flag - yes
-- Check for working CXX compiler: /usr/local/bin/g++-7
-- Check for working CXX compiler: /usr/local/bin/g++-7 -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- IPO / LTO not supported: <Change Dir: /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin
Run Build Command:"/usr/local/bin/gmake"
/usr/local/Cellar/cmake/3.13.3/bin/cmake -S/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/src -B/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/Cellar/cmake/3.13.3/bin/cmake -E cmake_progress_start /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin/CMakeFiles /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin/CMakeFiles/progress.marks
/usr/local/bin/gmake -f CMakeFiles/Makefile2 all
gmake[1]: Entering directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
/usr/local/bin/gmake -f CMakeFiles/foo.dir/build.make CMakeFiles/foo.dir/depend
gmake[2]: Entering directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
cd /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin && /usr/local/Cellar/cmake/3.13.3/bin/cmake -E cmake_depends "Unix Makefiles" /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/src /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/src /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin/CMakeFiles/foo.dir/DependInfo.cmake
Scanning dependencies of target foo
gmake[2]: Leaving directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
/usr/local/bin/gmake -f CMakeFiles/foo.dir/build.make CMakeFiles/foo.dir/build
gmake[2]: Entering directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
[ 25%] Building CXX object CMakeFiles/foo.dir/foo.cpp.o
/usr/local/bin/g++-7 -flto -fno-fat-lto-objects -o CMakeFiles/foo.dir/foo.cpp.o -c /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/src/foo.cpp
cc1plus: error: -fno-fat-lto-objects are supported only with linker plugin
gmake[2]: *** [CMakeFiles/foo.dir/build.make:66: CMakeFiles/foo.dir/foo.cpp.o] Error 1
gmake[2]: Leaving directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
gmake[1]: *** [CMakeFiles/Makefile2:113: CMakeFiles/foo.dir/all] Error 2
gmake[1]: Leaving directory '/Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build/CMakeFiles/_CMakeLTOTest-CXX/bin'
gmake: *** [Makefile:87: all] Error 2
>
Build type: Release
Logging is DISABLED
# Building MDP
# Building Factored MDP
# Building POMDP
# Building Python bindings
# Building Tests
# Building Examples
-- Boost version: 1.68.0
-- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "3.2.92")
-- Performing Test LPSOLVE_LINKS_ALONE
-- Performing Test LPSOLVE_LINKS_ALONE - Success
-- Found LpSolve: /usr/local/lib/liblpsolve55.dylib
-- Found PythonInterp: /usr/local/bin/python (found version "2.7.15")
-- Found PythonLibs: /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib (found suitable exact version "2.7.15")
-- Boost version: 1.68.0
-- Found the following Boost libraries:
-- python27
-- Boost version: 1.68.0
-- Found the following Boost libraries:
-- unit_test_framework
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/davidyang/Desktop/RBIcomparison/AI-Toolbox/build
Running make
in build folder results in error at Linking CXX shared library ../AIToolbox.dylib:
[ 47%] Linking CXX shared library ../AIToolbox.dylib
Undefined symbols for architecture x86_64:
"boost::python::objects::function_object(boost::python::objects::py_function const&, std::pair<boost::python::detail::keyword const*, boost::python::detail::keyword const*> const&)", referenced from:
boost::python::api::object boost::python::detail::make_function_aux<void (*)(_object*), boost::python::default_call_policies, boost::mpl::vector2<void, _object*>, mpl_::int_<0> >(void (*)(_object*), boost::python::default_call_policies const&, boost::mpl::vector2<void, _object*> const&, std::pair<boost::python::detail::keyword const*, boost::python::detail::keyword const*> const&, mpl_::int_<0>) [clone .isra.972] in Types.cpp.o
void boost::python::indexing_suite<std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > >, boost::python::detail::final_vector_derived_policies<std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > >, false>, false, false, Eigen::Matrix<double, -1, 1, 0, -1, 1>, unsigned long, Eigen::Matrix<double, -1, 1, 0, -1, 1> >::visit<boost::python::class_<std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > >, boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified> >(boost::python::class_<std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > >, boost::python::detail::not_specified, boost::python::detail::not_specified, boost::python::detail::not_specified>&) const in Types.cpp.o
exportTypes() in Types.cpp.o
exportBanditRollingAverage() in RollingAverage.cpp.o
exportBanditPolicyInterface() in PolicyInterface.cpp.o
...
"boost::python::objects::register_dynamic_id_aux(boost::python::type_info, std::pair<void*, boost::python::type_info> (*)(void*))", referenced from:
exportTypes() in Types.cpp.o
boost::python::api::object boost::python::objects::detail::demand_iterator_class<__gnu_cxx::__normal_iterator<Eigen::Matrix<double, -1, 1, 0, -1, 1>*, std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > > >, boost::python::return_internal_reference<1ul, boost::python::default_call_policies> >(char const*, __gnu_cxx::__normal_iterator<Eigen::Matrix<double, -1, 1, 0, -1, 1>*, std::vector<Eigen::Matrix<double, -1, 1, 0, -1, 1>, std::allocator<Eigen::Matrix<double, -1, 1, 0, -1, 1> > > >*, boost::python::return_internal_reference<1ul, boost::python::default_call_policies> const&) in Types.cpp.o
boost::python::api::object boost::python::objects::detail::demand_iterator_class<__gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >, boost::python::return_value_policy<boost::python::return_by_value, boost::python::default_call_policies> >(char const*, __gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int, std::allocator<unsigned int> > >*, boost::python::return_value_policy<boost::python::return_by_value, boost::python::default_call_policies> const&) in Types.cpp.o
boost::python::api::object boost::python::objects::detail::demand_iterator_class<__gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >, boost::python::return_value_policy<boost::python::return_by_value, boost::python::default_call_policies> >(char const*, __gnu_cxx::__normal_iterator<unsigned long*, std::vector<unsigned long, std::allocator<unsigned long> > >*, boost::python::return_value_policy<boost::python::return_by_value, boost::python::default_call_policies> const&) in Types.cpp.o
exportBanditRollingAverage() in RollingAverage.cpp.o
exportBanditPolicyInterface() in PolicyInterface.cpp.o
exportBanditEpsilonPolicy() in EpsilonPolicy.cpp.o
...
ld: symbol(s) not found for architecture x86_64
collect2: error: ld returned 1 exit status
make[2]: *** [src/CMakeFiles/AIToolbox.dir/build.make:975: AIToolbox.dylib] Error 1
make[1]: *** [CMakeFiles/Makefile2:995: src/CMakeFiles/AIToolbox.dir/all] Error 2
make: *** [Makefile:95: all] Error 2
Currently the formats in which the library reads/writes data are very limited, and the format does not comply with existing libraries. Implementing an I/O module to read and write data in the format of Cassandra's POMDP format would be nice.
Is there any example code or documentation available on using the Cassandra parser for a POMDP model? When you say it parses a reasonable subset of the Cassandra POMDP format, what exactly do you mean?
Hello,
I recently downloaded this library to use on Windows 10. I have run into trouble building the library though. Firstly, the lpsolve link takes to you to the main website, but doesn't clarify which version of lpsolve to download (https://sourceforge.net/projects/lpsolve/files/lpsolve/5.5.2.5/). I also think this instruction could be made clearer:
(a shared library must be available to compile the Python wrapper)."
Which shared library must I download for this? Are there additional instructions to execute this step?
I am currently getting this error when attempting to build the toolbox:
-- Could not link against lpsolve55!
CMake Error at C:/Program Files/CMake/share/cmake-3.11/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find LpSolve (missing: LPSOLVE_LIBRARIES LPSOLVE_LINKS)
Call Stack (most recent call first):
C:/Program Files/CMake/share/cmake-3.11/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
cmake/Modules/FindLpSolve.cmake:88 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:149 (find_package)
Can you tell me which directories LPSOLVE_LIBRARIES and LPSOLVE_LINKS should be pointing to?
Thank you.
Hi Svalorzen,
When I try to save the policy using pickle, I received error:
RuntimeError: Pickling of "MDP.QGreedyPolicy" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html)
How do I save the policy?
Thank you!
Jing
Whenever I try to compile any of the three tutorials (using the line g++ -Iinclude -I/usr/local/include/eigen3 -std=c++20 examples/MDP/tiger_antelope.cpp
as an example, running from the AI-Toolbox-master directory, or running g++ -I/home/ben/AI/AI-Toolbox-master/include -I/usr/local/include/eigen3 -std=c++20 tiger_antelope.cpp
from the directory containing the tutorial) I get a bunch of errors. I'm using C++ 20 and g++-10 and I have verified these using the --version options for each. I currently have installed Eigen using apt, but I originally tried downloading from the Eigen site and get the same errors either way. The first few errors I'm getting look like the following:
tiger_antelope.cpp:(.text+0x854): undefined reference to `AIToolbox::MDP::ValueIteration::ValueIteration(unsigned int, double, AIToolbox::MDP::ValueFunction)'
/usr/bin/ld: tiger_antelope.cpp:(.text+0x8d7): undefined reference to `AIToolbox::MDP::ValueIteration::getTolerance() const'
/usr/bin/ld: tiger_antelope.cpp:(.text+0x953): undefined reference to `AIToolbox::MDP::Policy::Policy(unsigned long, unsigned long, AIToolbox::MDP::ValueFunction const&)'
/usr/bin/ld: tiger_antelope.cpp:(.text+0x958): undefined reference to `AIToolbox::Impl::Seeder::getSeed()'
/usr/bin/ld: tiger_antelope.cpp:(.text+0x9ca): undefined reference to `AIToolbox::MDP::SparseModel::isTerminal(unsigned long) const'
And then after that, I get several errors that are similar to this:
/usr/bin/ld: /tmp/ccLrW1Ts.o: in function `Eigen::Matrix<double, -1, -1, 1, -1, -1> AIToolbox::MDP::computeQFunction<AIToolbox::MDP::SparseModel>(AIToolbox::MDP::SparseModel const&, Eigen::Matrix<double, -1, 1, 0, -1, 1> const&, Eigen::Matrix<double, -1, -1, 1, -1, -1>)':
tiger_antelope.cpp:(.text._ZN9AIToolbox3MDP16computeQFunctionINS0_11SparseModelEEEN5Eigen6MatrixIdLin1ELin1ELi1ELin1ELin1EEERKT_RKNS4_IdLin1ELi1ELi0ELin1ELi1EEES5_[_ZN9AIToolbox3MDP16computeQFunctionINS0_11SparseModelEEEN5Eigen6MatrixIdLin1ELin1ELi1ELin1ELin1EEERKT_RKNS4_IdLin1ELi1ELi0ELin1ELi1EEES5_]+0x3c): undefined reference to `AIToolbox::MDP::SparseModel::getA() const'
The final lines of errors are these:
/usr/bin/ld: /tmp/ccECV0k8.o:(.data.rel.ro._ZTIN9AIToolbox3MDP6PolicyE[_ZTIN9AIToolbox3MDP6PolicyE]+0x10): undefined reference to `typeinfo for AIToolbox::MDP::PolicyWrapper'
collect2: error: ld returned 1 exit status
I haven't been able to figure out how to fix any of this; most of it seems to be the tutorial programs being unable to access other files within other AI-Toolbox folders. I'd appreciate any help you could give on this!
Would it be technically possible to have a module in AI-Toolbox capable of loading MDP/POMDP models in RDDL and/or PPDDL format (which seems to be the "official" formats for the international planning competitions IPC) ?
There is a lot of planning domains / models in these formats and it would be cool to be able to easily test the AI-Toolbox algorithms on them.
Hi to all,
I am new of AI-Toolbox. I properly set and test my POMDP problem. However, I am searching for a method to save the policy after that it has been solved. My problem is quite huge (more than 100 states). Is there a method to save the policy, loading it when my POMDP node starts?
While it has been all fun and games storing the R(s,a,s) for all models, I think that the time has come to remove this "feature" and just store the R(s,a), which is pretty much equivalent.
This simultaneously reduces the computational costs of pretty much all methods, since they often have to compute R(s,a) anyway, and memory consumption (by a factor of |S|, which is not little).
We should still offer an API in order to create models from a 3D reward function, but internally it would be much much better to stop storing the whole thing.
This will change a bunch of the API for models since they won't offer a s,a,s' access to the reward function anymore, but I think in the long run it's going to be better, considering the performance improvements.
I'm currently finishing a branch, and this will be the next thing I do after that.
While each method & related code is well commented, I'd like more examples on how to use the library, and possibly explanations of the underlying algorithms.
I'm very open to contributions for this. I think that if somebody else can write the documentation it will be much more helpful than if I write it myself, given that already know how the code works and is thus much harder for me to guess what are the pain points for new users.
If you feel like helping out, I'd be very glad to help you along.
Hi,
I've recently found your repository. It looks very promising.
I can't help at the moment but I probably will very soon if you accept it.
If I may suggest you two things:
In either case, put the badges in your readme for people to be able to see them.
Carry on!
Cheers,
Emmanuel
is the pomcp solver(I see pomdp value iteration work, while not of pomcp) of AI_Toolbox implemented or not? Or is there any example I can test the pomcp codes?
I am trying to use this toolbox to solve the tag problem mentioned in the PBVI paper, and i have the following two questions:
__call__( (PBVI)self, (Model)model, (POMDP_VFun)v) -> object :
Hi,
I've been trying to run the Tiger Antelope code in the example
folder, but after countless time (I'd say that is runs for something like 20mins), the code segfaults. This is the terminal output:
(~/code/AI-Toolbox/examples) (master)
[alecive@malakim]$ make
g++ -O3 -std=c++11 ./tutorialcode.cpp -o tiger_antelope -I../include -I/usr/include/eigen3 -L../build -lAIToolboxMDP
(~/code/AI-Toolbox/examples) (master)
[alecive@malakim]$ ./tiger_antelope
Mon Nov 14 10:57:09 2016
- Copying model...!
Mon Nov 14 11:14:25 2016
- Init solver...!
Mon Nov 14 11:14:25 2016
- Starting solver!
tiger_antelope: /usr/include/eigen3/Eigen/src/Core/CwiseBinaryOp.h:131: Eigen::CwiseBinaryOp<BinaryOp, Lhs, Rhs>::CwiseBinaryOp(const Lhs&, const Rhs&, const BinaryOp&) [with BinaryOp = Eigen::internal::scalar_product_op<double, double>; Lhs = const Eigen::SparseMatrix<double, 1>; Rhs = const Eigen::SparseMatrix<double, 1>]: Assertion `aLhs.rows() == aRhs.rows() && aLhs.cols() == aRhs.cols()' failed.
Aborted (core dumped)
As the new logging system seems to be working, there's probably a need to "backport" it to all methods, so that if needed an user can print how an algorithm is doing. I'm not exactly sure what would be best to print, so if anyone has a good idea let me know.
Hi,
I follow this work from the beginning and it becomes more and more interesting. Well done.
However, I got two questions:
I will sadly retain myself to contribute as there is too much Boost for me to be plainly efficient and sure to not break anything.
Thank you and keep up!
Emmanuel
Hello @Svalorzen, I've got the following issues while compiling your software:
[ 40%] Built target AIToolboxMDP
Scanning dependencies of target MDP_RLModelTests
Scanning dependencies of target MDP_MCTSTests
Scanning dependencies of target MDP_WoLFPolicyTests
Scanning dependencies of target AIToolboxPOMDP
Scanning dependencies of target MDP_PrioritizedSweepingTests
Scanning dependencies of target MDP_ModelTests
Scanning dependencies of target FactoredMDP_SparseCooperativeQLearningTests
Scanning dependencies of target FactoredMDP_VariableEliminationTests
Scanning dependencies of target FactoredMDP_FactoredContainerTests
Scanning dependencies of target MDP_ExperienceTests
Scanning dependencies of target MDP_SARSATests
Scanning dependencies of target MDP_TypesTests
Scanning dependencies of target MDP_QLearningTests
Scanning dependencies of target MDP_SparseExperienceTests
Scanning dependencies of target FactoredMDP_UtilsTests
Scanning dependencies of target FactoredMDP_MultiObjectiveVariableEliminationTests
Scanning dependencies of target MDP_SparseModelTests
Scanning dependencies of target MDP_ValueIterationTests
Scanning dependencies of target MDP_SparseRLModelTests
Scanning dependencies of target FactoredMDP_FactorGraphTests
CMakeFiles/Global_UtilsTests.dir/UtilsTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
UtilsTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/Global_UtilsTests.dir/build.make:95: recipe for target '../test/bin/Global_UtilsTests' failed
make[2]: *** [../test/bin/Global_UtilsTests] Error 1
CMakeFiles/Makefile2:1134: recipe for target 'test/CMakeFiles/Global_UtilsTests.dir/all' failed
make[1]: *** [test/CMakeFiles/Global_UtilsTests.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 71%] Built target AIToolboxPOMDP
CMakeFilesCMakeFiles/MDP_MCTSTests.dir/MDP/MCTSTests.cpp.o: In function `boost::CMakeFilesunit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::CMakeFilesunit_test:/:basic_cstringMDP_RLModelTests.dir</charMDP /const>RLModelTests.cpp.o):' :In
MCTSTests.cppfunction: (`.boosttext._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE:[:unit_test_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE:]:+make_test_case0x23()boost:: :undefinedunit_test :reference: callback0to< boost`:boost::unit_test::unit_test::ut_detail::ut_detail::unused:>normalize_test_case_name (constboost&:,: unit_testboost::::basic_cstringunit_test<:char: basic_cstringconst<>char) 'const
>)':
RLModelTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::collect2: error: ld returned 1 exit status
basic_cstring<char const>)'
/FactoredMDP_FactorGraphTests.dircollect2: error: ld returned 1 exit status
/FactoredMDP/FactorGraphTests.cpp.o: In/MDP_QLearningTests.dir/MDP/QLearningTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
QLearningTests.cpp:(CMakeFiles.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE/[MDP_SparseRLModelTests.dir/_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEEMDP]/+SparseRLModelTests.cpp.o0x23:) :In undefined functionreference `toboost :`:boostunit_test::::unit_test:make_test_case:(ut_detailboost::::normalize_test_case_nameunit_test(:boost::callback0:<unit_testboost::::basic_cstringunit_test<:char: ut_detailconst:>:)unused'>
const&, boost::unit_test::basic_cstring<char const>)':
SparseRLModelTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEEcollect2: error: ld returned 1 exit status
[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
functioncollect2: error: ld returned 1 exit status
`boost::unit_test::make_test_case(boost::unit_test::CMakeFiles/MDP_WoLFPolicyTests.dir/MDP/WoLFPolicyTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::test/CMakeFiles/MDP_MCTSTests.dir/build.make:96: recipe for target '../test/bin/MDP_MCTSTests' failed
basic_cstring<charmake[2]: *** [../test/bin/MDP_MCTSTests] Error 1
const>)':
WoLFPolicyTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
callback0<CMakeFiles/Makefile2:726: recipe for target 'test/CMakeFiles/MDP_MCTSTests.dir/all' failed
collect2: error: ld returned 1 exit status
make[1]: *** [test/CMakeFiles/MDP_MCTSTests.dir/all] Error 2
boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
test/CMakeFiles/MDP_QLearningTests.dir/build.make:96: recipe for target '../test/bin/MDP_QLearningTests' failed
make[2]: *** [../test/bin/MDP_QLearningTests] Error 1
FactorGraphTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_nameCMakeFilesCMakeFiles/Makefile2:985: recipe for target 'test/CMakeFiles/MDP_QLearningTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_QLearningTests.dir/all] Error 2
/MDP_TypesTests.dir/(boost::unit_test::basic_cstring<char const>)'
CMakeFiles/MDP_ValueIterationTests.dirCMakeFiles/MDP_PrioritizedSweepingTests.dir/MDP/PrioritizedSweepingTests.cpp.o: In function collect2: error: ld returned 1 exit status
`boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
PrioritizedSweepingTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
MDP/MDP//ValueIterationTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_testcollect2: error: ld returned 1 exit status
::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
TypesTests.cpp.oValueIterationTests.cpp:(:.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[ _ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::Inut_detail::normalize_test_case_name(boost::unit_test::basic_cstringtest/CMakeFiles/MDP_RLModelTests.dir/build.make:96: recipe for target '../test/bin/MDP_RLModelTests' failed
<charmake[2]: *** [../test/bin/MDP_RLModelTests] Error 1
const>)'
function `boost::unit_test::make_test_case(boost:collect2: error: ld returned 1 exit status
:unit_test::callback0<boost::unit_test::ut_detail::unusedtest/CMakeFiles/MDP_SparseRLModelTests.dir/build.make:96: recipe for target '../test/bin/MDP_SparseRLModelTests' failed
> make[2]: *** [../test/bin/MDP_SparseRLModelTests] Error 1
const&, boost:CMakeFiles/Makefile2:800: recipe for target 'test/CMakeFiles/MDP_RLModelTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_RLModelTests.dir/all] Error 2
:unit_test::basic_cstring<char const>)':
TypesTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/MDP_WoLFPolicyTests.dir/build.make:96: recipe for target '../test/bin/MDP_WoLFPolicyTests' failed
make[2]: *** [../test/bin/MDP_WoLFPolicyTests] Error 1
test/CMakeFiles/FactoredMDP_FactorGraphTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_FactorGraphTests' failed
CMakeFilesmake[2]: *** [../test/bin/FactoredMDP_FactorGraphTests] Error 1
/MDP_SARSATests.dir/MDP/SARSATests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
SARSATests.cpp:test/CMakeFiles/MDP_PrioritizedSweepingTests.dir/build.make:96: recipe for target '../test/bin/MDP_PrioritizedSweepingTests' failed
(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEEmake[2]: *** [../test/bin/MDP_PrioritizedSweepingTests] Error 1
[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::CMakeFiles/Makefile2:948: recipe for target 'test/CMakeFiles/MDP_SparseRLModelTests.dir/all' failed
normalize_test_case_name(make[1]: *** [test/CMakeFiles/MDP_SparseRLModelTests.dir/all] Error 2
boost::unit_test::basic_cstring<charCMakeFiles/Makefile2:615: recipe for target 'test/CMakeFiles/MDP_WoLFPolicyTests.dir/all' failed
constmake[1]: *** [test/CMakeFiles/MDP_WoLFPolicyTests.dir/all] Error 2
>)'
CMakeFiles/Makefile2:1060: recipe for target 'test/CMakeFiles/FactoredMDP_FactorGraphTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_FactorGraphTests.dir/all] Error 2
test/CMakeFiles/MDP_ValueIterationTests.dir/build.make:96: recipe for target '../test/bin/MDP_ValueIterationTests' failed
make[2]: *** [../test/bin/MDP_ValueIterationTests] Error 1
collect2: error: ld returned 1 exit status
CMakeFiles/Makefile2:541: recipe for target 'test/CMakeFiles/MDP_PrioritizedSweepingTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_PrioritizedSweepingTests.dir/all] Error 2
test/CMakeFiles/MDP_TypesTests.dir/build.make:96: recipe for target '../test/bin/MDP_TypesTests' failed
make[2]: *** [../test/bin/MDP_TypesTests] Error 1
CMakeFiles/Makefile2:1097: recipe for target 'test/CMakeFiles/MDP_ValueIterationTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_ValueIterationTests.dir/all] Error 2
CMakeFiles/FactoredMDP_FactoredContainerTests.dir/FactoredMDP/FactoredContainerTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
FactoredContainerTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
CMakeFiles/Makefile2:652: recipe for target 'test/CMakeFiles/MDP_TypesTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_TypesTests.dir/all] Error 2
test/CMakeFiles/MDP_SARSATests.dir/build.make:96: recipe for target '../test/bin/MDP_SARSATests' failed
make[2]: *** [../test/bin/MDP_SARSATests] Error 1
CMakeFiles/Makefile2:689: recipe for target 'test/CMakeFiles/MDP_SARSATests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_SARSATests.dir/all] Error 2
test/CMakeFiles/FactoredMDP_FactoredContainerTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_FactoredContainerTests' failed
make[2]: *** [../test/bin/FactoredMDP_FactoredContainerTests] Error 1
CMakeFiles/Makefile2:1171: recipe for target 'test/CMakeFiles/FactoredMDP_FactoredContainerTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_FactoredContainerTests.dir/all] Error 2
CMakeFiles/FactoredMDP_SparseCooperativeQLearningTests.dir/FactoredMDP/SparseCooperativeQLearningTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
SparseCooperativeQLearningTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/FactoredMDP_SparseCooperativeQLearningTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_SparseCooperativeQLearningTests' failed
make[2]: *** [../test/bin/FactoredMDP_SparseCooperativeQLearningTests] Error 1
CMakeFiles/Makefile2:1320: recipe for target 'test/CMakeFiles/FactoredMDP_SparseCooperativeQLearningTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_SparseCooperativeQLearningTests.dir/all] Error 2
CMakeFiles/FactoredMDP_VariableEliminationTests.dir/FactoredMDP/VariableEliminationTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
VariableEliminationTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/FactoredMDP_VariableEliminationTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_VariableEliminationTests' failed
make[2]: *** [../test/bin/FactoredMDP_VariableEliminationTests] Error 1
CMakeFiles/Makefile2:1208: recipe for target 'test/CMakeFiles/FactoredMDP_VariableEliminationTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_VariableEliminationTests.dir/all] Error 2
CMakeFiles/FactoredMDP_UtilsTests.dir/FactoredMDP/UtilsTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
UtilsTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/FactoredMDP_UtilsTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_UtilsTests' failed
make[2]: *** [../test/bin/FactoredMDP_UtilsTests] Error 1
CMakeFiles/Makefile2:837: recipe for target 'test/CMakeFiles/FactoredMDP_UtilsTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_UtilsTests.dir/all] Error 2
CMakeFiles/FactoredMDP_MultiObjectiveVariableEliminationTests.dir/FactoredMDP/MultiObjectiveVariableEliminationTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
MultiObjectiveVariableEliminationTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/FactoredMDP_MultiObjectiveVariableEliminationTests.dir/build.make:96: recipe for target '../test/bin/FactoredMDP_MultiObjectiveVariableEliminationTests' failed
make[2]: *** [../test/bin/FactoredMDP_MultiObjectiveVariableEliminationTests] Error 1
CMakeFiles/Makefile2:1245: recipe for target 'test/CMakeFiles/FactoredMDP_MultiObjectiveVariableEliminationTests.dir/all' failed
make[1]: *** [test/CMakeFiles/FactoredMDP_MultiObjectiveVariableEliminationTests.dir/all] Error 2
CMakeFiles/MDP_SparseExperienceTests.dir/MDP/SparseExperienceTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstringCMakeFiles</charMDP_ModelTests.dir /constMDP>/)ModelTests.cpp.o'::
InSparseExperienceTests.cpp :function( `boost.:text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE:[unit_test:_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE:]make_test_case+(0x23boost)::: unit_testundefined: :referencecallback0 <toboost :`:boostunit_test::::unit_testut_detail::::ut_detailunused:>: normalize_test_case_nameconst(&boost,: :boostunit_test::::unit_testbasic_cstring:<:charbasic_cstring <constchar> )const'>
)':
ModelTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
collect2: error: ld returned 1 exit status
test/CMakeFiles/MDP_SparseExperienceTests.dir/build.make:96: recipe for target '../test/bin/MDP_SparseExperienceTests' failed
test/CMakeFiles/MDP_ModelTests.dir/build.make:96: recipe for target '../test/bin/MDP_ModelTests' failed
make[2]: *** [../test/bin/MDP_SparseExperienceTests] Error 1
make[2]: *** [../test/bin/MDP_ModelTests] Error 1
CMakeFiles/Makefile2:874: recipe for target 'test/CMakeFiles/MDP_SparseExperienceTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_SparseExperienceTests.dir/all] Error 2
CMakeFiles/Makefile2:578: recipe for target 'test/CMakeFiles/MDP_ModelTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_ModelTests.dir/all] Error 2
CMakeFiles/MDP_SparseModelTests.dir/MDP/SparseModelTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused> const&, boost::unit_test::basic_cstring<char const>)':
SparseModelTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<CMakeFileschar/ constMDP_ExperienceTests.dir>/)MDP'/
ExperienceTests.cpp.o: In function `boost::unit_test::make_test_case(boost::unit_test::callback0<boost::unit_test::ut_detail::unused>collect2: error: ld returned 1 exit status
const&, boost::unit_test::basic_cstring<char const>)':
ExperienceTests.cpp:(.text._ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE[_ZN5boost9unit_test14make_test_caseERKNS0_9callback0INS0_9ut_detail6unusedEEENS0_13basic_cstringIKcEE]+0x23): undefined reference to `boost::unit_test::ut_detail::normalize_test_case_name(boost::unit_test::basic_cstring<char const>)'
collect2: error: ld returned 1 exit status
test/CMakeFiles/MDP_SparseModelTests.dir/build.make:96: recipe for target '../test/bin/MDP_SparseModelTests' failed
make[2]: *** [../test/bin/MDP_SparseModelTests] Error 1
CMakeFiles/Makefile2:911: recipe for target 'test/CMakeFiles/MDP_SparseModelTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_SparseModelTests.dir/all] Error 2
test/CMakeFiles/MDP_ExperienceTests.dir/build.make:96: recipe for target '../test/bin/MDP_ExperienceTests' failed
make[2]: *** [../test/bin/MDP_ExperienceTests] Error 1
CMakeFiles/Makefile2:763: recipe for target 'test/CMakeFiles/MDP_ExperienceTests.dir/all' failed
make[1]: *** [test/CMakeFiles/MDP_ExperienceTests.dir/all] Error 2
[ 72%] Linking CXX shared library ../MDP.so
[ 72%] Built target MDP
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2
-- The C compiler identification is GNU 4.9.3
-- The CXX compiler identification is GNU 4.9.3
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.58.0
-- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "3.2")
-- Found PythonLibs: /home/jing/anaconda2/lib/libpython2.7.so (found suitable version "2.7.12", minimum required is "2.7")
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- python
-- Performing Test LPSOLVE_LINKS_ALONE
-- Performing Test LPSOLVE_LINKS_ALONE - Success
-- Found LpSolve: /usr/local/lib/liblpsolve55.so
-- Boost version: 1.58.0
-- Found the following Boost libraries:
-- unit_test_framework
-- Found PythonInterp: /home/jing/anaconda2/bin/python2.7 (found suitable exact version "2.7.12")
-- Configuring done
-- Generating done
Importing POMDP in python raises the following error:
import POMDP
>>> ImportError: ./POMDP.so: undefined symbol: _ZNK9AIToolbox5POMDP18IncrementalPruning10getEpsilonEv
The error does not occur on import MDP
, if I try again the import POMDP
, python segfaults. The import was working on dcbe953.
(Compiled from 722826b, with python 1.7.12
, boost 1.62.0
and eigen 3.2.95
.)
There is a high need of writing better tests for existing functionality, possibly using problems and solutions available from the literature. Currently tests mostly verify basic functionality and are probably not very indicative of the library correctness.
I am wondering if there is already a way to read in files in Tony Cassandra's file format: http://www.pomdp.org/code/pomdp-file-spec.html
Hello @Svalorzen , I've got the following issues while compiling your software:
(~/code/AI-Toolbox/build) (master)
[alecive@malakim]$ make
Scanning dependencies of target AIToolboxMDP
[ 1%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/Impl/Seeder.cpp.o
[ 3%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Experience.cpp.o
[ 4%] Building CXX object src/CMakeFiles/AIToolboxMDP.dir/MDP/Utils.cpp.o
In file included from /home/alecive/code/AI-Toolbox/include/AIToolbox/MDP/Utils.hpp:5:0,
from /home/alecive/code/AI-Toolbox/src/MDP/Utils.cpp:1:
/home/alecive/code/AI-Toolbox/include/AIToolbox/MDP/Types.hpp:120:111: error: expected primary-expression before ‘,’ token
static_cast<double (Z::*)(size_t,size_t,size_t) const> (&Z::getTransitionProbability),
^
/home/alecive/code/AI-Toolbox/include/AIToolbox/MDP/Types.hpp:218:105: error: expected primary-expression before ‘,’ token
static_cast<long unsigned (Z::*)(size_t,size_t,size_t) const> (&Z::getVisits),
^
make[2]: *** [src/CMakeFiles/AIToolboxMDP.dir/MDP/Utils.cpp.o] Error 1
make[1]: *** [src/CMakeFiles/AIToolboxMDP.dir/all] Error 2
make: *** [all] Error 2
To complete the issue, here is the output of my cmake step:
(~/code/AI-Toolbox/build) (master)
[alecive@malakim]$ cmake -DCMAKE_BUILD_TYPE=Release -DMAKE_MDP=1 -DMAKE_PYTHON=1 .. # Will build MDP and Python libraries
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.54.0
-- Found Eigen3: /usr/include/eigen3 (Required is at least version "2.91.0")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.6", minimum required is "2.7")
-- Boost version: 1.54.0
-- Found the following Boost libraries:
-- python
-- Boost version: 1.54.0
-- Found the following Boost libraries:
-- unit_test_framework
-- Found PythonInterp: /usr/bin/python2.7 (found suitable exact version "2.7.6")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alecive/code/AI-Toolbox/build
Any hint?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.