ds4dm / ecole Goto Github PK
View Code? Open in Web Editor NEWExtensible Combinatorial Optimization Learning Environments
Home Page: https://www.ecole.ai
License: BSD 3-Clause "New" or "Revised" License
Extensible Combinatorial Optimization Learning Environments
Home Page: https://www.ecole.ai
License: BSD 3-Clause "New" or "Revised" License
At the moment, pressing CTL-C while Ecole is running results in either one the two following effects:
I think behaviour 2) is OK: CTRL-C signals should stop everything. Behaviour 1) is ok if one runs SCIP in interactive mode via the console, and wants to resume the optimization process afterwards. But with Ecole, the behaviour does not make much sense. Also it is simply annoying, say I want to kill Ecole in a jupyter Notebook, or in console, I have to keep pressing CTRL-C until Python catches the signal in order for the current execution to terminate.
All settings.
Run an MDP loop (while True: reset, then while not done: step). Press CTRL-C during the execution.
A Python KeyboardInterruption.
A fix is to tell SCIP to not catch CTRL-C signals:
"misc/catchctrlc": False
In the documentation, we talk about environments, while referring to the class EnvironmentComposer. This can be confusing, as it could suggest there is actually an Environment class somewhere, but this class is nowhere to be found.
Change the Python class name EnvironmentComposer to Environment.
Have a proper Environment class, which inherits from EnvironmentComposer ?
NA
Currently, given for instance a reward function, the process of calling reset
and obtain_reward
is not transparent to the user. Is When is reset
called? Before of after a the dynamics are themselves reset?
When writing a new function, it also leads to confusions. Should reset
return a reward or will obtain_reward
be called right after?
Furthermore, this current setting offers no possibility to access the Model
outside of very specific places.
Finally, the code for calling the callback is very similar between observation functions, reward functions, and soon info functions.
To alleviate this, we could generalize the idea of callback event, to reset_pre
, reset_post
, step_pre
, step_post
, seed_pre
, seed_post
.
Ideally, such a callback would be created by simply defining a method a the same name.
struct AnyEvent: Event {
void reset_pre(scip::Model const&) override { /* Brand new model not yet on initial state */ }
};
An reward function would then become an event, with a __call__
/operator()
method for the reward.
struct LpIterations: RewardFunction, Event {
void reset_pre(scip::Model const&) { last_lp_iter = 0; }
Reward operator()(scip::Model model const&) {
auto reward = ... - last_lp_iter;
return reward;
}
private:
double last_lp_iter = 0;
};
override
on scip::Model&
and scip::Model const&
__call__
/operator()
becomes redundant with reset_post
and step_post
, so should it simply be a simple getter, that guarantee to return the same on two successive calls.Currently, Model.seed()
only sets randomization/randomseedshift
. The permutation of rows / cols depends on a distinct seed, "randomization/permutationseed". We have to deal with that in Model
, so that users only have to change one seed, which then affects both. I suggest setting both of them to the same value, or a shifted value, e.g.,
randomization/permutationseed = randomization/randomseedshift + 42
State function could be composed easily to create new ones.
TupleObservationFunction
and DictObservationFunction
for observation functionsImplement the same Python API #49 in C++
Reward function all derive from the same ABC, so this could be done with virtual calls (rather than templates) and bound to Python (minimize code duplication).
For observation functions, template (and hence code duplication) seem necessary.
Due to the fact that SCIP and Ecole are compiled in Release mode in CI, all assertions are silently ignored in CI tests.
This is problematic, especially since SCIP relies heavily on it.
CMAKE_BUILD_TYPE
? conda CXX_FLAGS
?The GIL is currently in place.
Environments should not ask for an observation in the final state (when done == True). This observation is most likely impossible to compute, since SCIP will likely not be in a SOLVING state. And the final observation of an episode is not relevant for learning anyway. That way, Observation functions do not deal with that situation, and silently return empty observations (current behaviour).
The Environment does not call the registered ObservationFunction when done==True
, and the ObservationFunctions do not check any more for SCIP_STAGE_SOLVING.
na
na
The tutorial should showcase how Ecole can be used with the Configure
environment and a bandit algorithm to find good parameters for SCIP.
The file should go under /examples
When the user pass a wrong action to Environment::step
, the environment needs to be reset.
However, when using the environment interactively, it would be more natural that wrong value are rejected, but that the user could still try to pass other values.
This requires either rolling back on exception, or checking action value at the start of step before attempting to step_state
.
Some environment, such as selecting the branching variable or the next branching node, have a dynamic action set. This means that, within different states of the episode, the set of possible action changes.
This is not the same as the Action Space defined by OpenAi Gym because the latter is a static property of the environment and does not change during an episode (or between episodes).
The set of available actions is critical information that needs to be given to the user.
Here are some ideas I could think of, none is perfect. Any other ideas are welcome.
The action set is returned as part of the transition in step
and reset
obs, action_set, reward, done, info = env.step(action)
Downsides:
..., reward, done, ...
, or ..., done, reward, ...
, this will add much more confusion.An attribute of the environment. Something in the like of
action = user_policy(obs, env.current_action_set())
obs, reward, done, info = env.step(action)
Downsides:
Part of the Observation. After all, this is a POMDP, if the observation is not good enough to tell that agent what to do, it is not a good observation, period.
Some thoughts
NodeBipartite
could have a flag for every variable indicating which variable is fractional, which would have the benefit of avoiding having permutation issues with the action set.NodeBipartite
for other environments, now it needs to contain all action spaces?Should be corrected everywhere, e.g. the ECOLE_DEVELOPPER cmake argument.
Some environment MDPs may be over after reset(), without any call to step(). In such a case, no reward is available. This is annoying, as typically one would like to obtain cumulated rewards for every episode of the MDP, for example when benchmarking different policies. This situation happens often for example in branching, when the instance is solved in preprocessing. We still want to measure the running time, or the number of nodes, or the primal-dual integral etc.
observation, action_set, reward, done = env.reset("path/to/problem")
Obtaining the cumulated reward at any moment by calling env.getCumulatedReward()
. However, that cumulated reward would not match a cumulated reward manually computed by the user. Observing such a difference can be very confusing for users. Also, it introduces an additional function, which must be safe-guarded (can be called only during / after an episode).
fmtlib
for simpler string creationfmtlib
for Ecole classesecole::scip::Exception
ecole::environment::Exception
Provide a way to compare against a default SCIP baseline, in terms of Ecole metrics (reward functions).
A new environment (DefaultSCIP ? Default ?) with a single step MDP, and empty action set. That way SCIP is ran with default config, and one can extract Ecole reward functions to compare to other Environments. That DefaultSCIP environment could simply be a specialization of the Configuring environment, with an empty action set.
Simply using the Configuring environment to provide a default baseline. This can be a bit confusing for users though.
The branching environments should be able to toggle between using regular, fractional branching candidates (SCIPgetLPBranchCands
) as hardcoded right now in Ecole, and using all non-fixed variables (SCIPgetPseudoBranchCands
), which is what learn2branch uses. We will need it to reproduce learn2branch in Ecole.
To reimplement learn2branch, we will need to reproduce pseudocost branching. This requires a new observation function that returns pseudocosts (similar to the current strong branching observation function).
I would like a new ObservationFunction
that returns the pseudocosts, like the current StrongBranchingScores
.
Code to reproduce:
import ecole
env = ecole.environment.Configuring()
env.reset(filename='libecole/tests/data/enlight8.mps')
env.step({'presolving/maxrounds': 0.1})
I suggest we check types more carefully, e.g., with a narrow cast:
template<class Target, class Source>
Target narrow_cast(Source v) {
auto r = static_cast<Target>(v);
if (static_cast<Source>(r) != v)
throw Exception("narrow_cast<>() failed");
return r;
Implement the state from Khalil et al.
https://github.com/ds4dm/PySCIPOpt/blob/9b88eddcfe99208e737236d39ccd6c9dd1a9bec4/src/pyscipopt/scip.pyx#L4203
Currently, to run the tests, the user is expected to have Ecole importable.
This leaves room for running tests with outdated versions of Ecole.
#32 makes would make this easier to debug, but a better behavior would be to avoid that altogether.
Tox is a tool that can solve this issue but is hardly compatible with our building tools.
Have a way to programmatically query/declare what SCIP stages are valid for a given observation/reward function, dynamics.
This has the following advantages:
optional
on terminal states (if made constexpr
), e.g. would help solve #59An observation function for instance could have:
struct NodeBipartite: ObservationFunction<SomeObs> {
static std::array<scip::Stage, 2> valid_stages = {scip::Stage::Solving}
...
};
static
/constexpr
variable or could be deduced from constructor argumentsdoc.ecole.ai
repository, and set proper DNS;To simplify the Python bindings, we should bind components (ObservationFunction
, RewardFunction
, SimpleEnvironment
...) independently, and combine them in a Python Environment class.
In practice, the DefaultEnvironment
would not be bound but reproduced in Python.
This would remove the need to have common base class for all the state functions, as well as simplify the default Environment class.
Python Environment inheritance and composition would be much more easy.
Environment
to be standalone classes, rather than inheriting from DefaultEnvironment
DefaultEnvironment
clone
methods and dereference
None
types, and manage them in PythonSimilarily to RewardFunction
, and ObservationFunction
, we want to have a composable/customizable type to return in the information dictionary.
An important question is what it the type of the information dictionary in C++, as it cannot be as dynamic as Python.
Something with std::map<Key, Value>
where:
Key
is std::string
? std::variant<std::string, long int>
?Value
is std::variant<std::string, long int, double, xt::xarray>
?Additionally, we also want to construct InformationFunction
from RewardFunction
.
Alternatively, the Value
type could be templated and the variant merged at compile time when merging different information function. Unfortunately, it may mean wrapping information function before binding them to Python.
Some feature in the NodeBipartite
observation only need to be computed on reset
and not on all transitions.
Cache the features:
Currently not possible via Cast_SFNIAE
, because the std::string
goes out of scope (and is deallocated) before the call to SCIP.
Possible solutions:
std::string
(no &
, no const
): possible copy of stringThe tutorial should showcase how Ecole can be used with the Branching environment, a node bipartite observation, and imitation learning to learn a branching policy.
The file should go under /examples
Write a test that verify that spaces inherited from their base class or from an existing space have the desired behavior when given back to the C++ end (through an environment).
test_inheritance.py
Note:
model
fixture to get a valid MILP problem)setup.py
sciki-build
to drive compilation with CMake
pyproject.toml
and PEP518 to resolve build time dependencies
libecole
dependencies) or CMake FetchContentxtensor-python
(no plan so far xtensor-stack/xtensor-python#219)Manifest.in
tests_require=["pytest", "pyscipopt"]
for pytestDockerfile.src
--no-build-isolation
)PYTHONPATH
)
SCIP_DIR
using piptox
for automated testingAdd type hints for IDE completion, static check, and documentation purposes.
Protocol
but this is Python 3.8+To more easily debug Ecole installs, in particular when the user is not running the version they are expecting, it would be useful to add Ecole version and Git revision (hash) in both libraries and tests.
In order to be able to combine several observation functions. The observations are then returned in a list, in the same order as the passed ObservationFunction's.
Pseudocode of the use case:
of = ObservationAggregator([NodeBipartite(), StrongBranchingScores()])
env = Environment(observation_function=of)
obs, done = env.reset()
bipartitegraph, sb_scores = obs
Replace Exception
for std::exception
in order to allow std::throw_with_nested
. That would improve error messages for the users.
Figure if it is possible to convert xtensor
to pyxtensor
without copy.
This would make it possible to remove the template from Observation
and ObservtionFunction
classes.
Everything is in the plural, get and set multiple SCIP parameters in one call for convenience.
C++, use the ParamType
variant to hold different parameters values.
It may also be an opportunity to bind the Python code using the variant as well to reduce code duplication.
Modularize feature extraction from scip::Model
Currently, feature extracted in NodeBipartite
cannot be easily reused by other classes, or customized in the current class.
On the other end, VarProxy
... create confusion with SCIP variables.
scip::Model
to the observation
namespace.vector
/ array
of names (enum)std::span
Think about the possibility of composing environments, e.g., Node selection + branching
Can be implemented as an ObservationFunction in Ecole.
Will need SCIP 7.0
Problem:
Python cannot give away ownership of its references. Using holder types such as std::unique_ptr
is impossible without making copies (of base::XSpace
, or scip::Model
).
Move semantics from Python are work in progress in Pybind11, and unexpected to the Python user anyways.
unique_ptr
or shared_ptr
shared_ptr
for spacesModel
with the holder typeModel::reset
to also use holder of modelshared_ptr
of spaces for Envptr<Model>
in favor of Model<Holder>
Env.reset
make_unqiue
for temaplated make_ptr
to statically dipatch between shared and unique pointersdone
flag on resetreset
None
on terminal statesenv.state.model.get_param("random/somehting")
)Currently the lifetime of Controller
and the model it uses Model
are independent but this assumption is wrong because
Model
should not be accessed without validation form Controller
;Controller
needs a valid Model
to be destructed, as it needs to terminate the solving (this cannot be prematurely terminated, as we want SCIP to properly free the solving resources).The lifetimes are currently manually maintained, but this design is not fullproof.
Proposed solution:
State
inside of Controller
, as std::shared_ptr
;std::shared_ptr<State>
to the auxillary thread, hence avoiding address errors;State
using Controller
lock;DefaultEnvironment
to access State
through Branching
Controller
(tied with #19)Continuous integration should test the following configurations:
python >= 3.6
gcc
vs clang
scip >= 6.0
To simplify builds, environments should be completely defined by Docker images.
TODO:
python-version
x compiler
;ld64
and libcxxabi
(for sanitizers)sanitizers-cmake
, cpp_starter_project
clang-tidy
and cppcheck
Make the following conventions consistent across the codebase:
auto x =
declaration (may be a copy in C++<17);m_var
void name(val&)
and auto name()
auto& name(val)
get
and set
prefixessnake_case
(std
convention) vs PascalCase
(Python convention) for typesMany of this things can be don by clang-tidy
Environments are made composable by their observation, reward, termination spaces.
TerminationSpace
TerminationSpace
subclassTerminationSpace
RewardSpace
APIRewardSpace
subclassRewardSpace
reset
for all spacesreset
that does nothingreset
and edit testsA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.