juliapomdp/despot.jl

[USE ARDESPOT.jl INSTEAD - This package is not maintained] Package implementing DESPOT POMDP solver for POMDPs.jl

despot.jl's Issues

add requirements

The requirements for this solver should be specified with a @POMDP_require block, see instructions

beliefUpdateParticle in module

Should beliefUpdateParticle.jl be included in the module so that it's easier to use? Right now the file has to be included explicitly to be accessed.

Happy to submit a PR if you agree with this, @ebalaban

Extra actions in bounds

Hi @ebalaban, right now there are some extra actions being passed around between the bounds - specifically the VNode constructor returns an action, and there is an action argument in the lower bound function. Do you think this should be part of the interface (I ask since the README doesn't include this)?

I think that the VNode constructor should only return a VNode, and the lower_bound function should not take an action as an argument.

One way to rectify this would be to get rid of the bounds abstract types - then the upper and lower bounds could be the same object. Then that object could cache actions calculated for the upper bound to be used in the lower bound. I can implement this if it sounds good to you (this is the approach that I have taken for my solver packages, and I have found that it works very well).

Random Numbers

If we want DESPOT to be able to be used out of the box with a variety of problems, we need to have a more sophisticated random number generation system.

For all POMDPs to work with DESPOT, the rng we feed to the interface functions should support all of the random number generation funcitons, e.g rand(rng, UInt32), randn(rng), rand!(rng, A), etc.

There are four ways that I can think of to do this:

1. write all of the rand() functions that we can think of for DESPOTRandomNumber
2. try to identify some minimal internal interface in julia/base/random.jl that we can implement for DESPOTRandomNumber to get all of the versions of the rand function automagically
3. make RandomStreems a list of MersenneTwisters instead of floats and feed copies of those to the POMDP model
4. make problem writers who want compatibility with DESPOT implement versions of their model functions that use a DESPOTRandomNumber.

@ebalaban did you have something in mind for this? What do you think we should do?

Here's my thoughts:
4 is just pushing the challenge to the problem-writer's side, which will make DESPOT essentially unusable for a lot of problems.
It would take a great deal of thinking to efficiently map a single float to a large set of uncorrelated random numbers, so I think 1 and 2 are not feasible.
Thus, I think 3 is the only viable option.

Hopefully this won't cause a prohibitive performance hit. I think a copy! of a MersenneTwister takes about a microsecond (not sure how that compares to other operations), but it looks like it might take hundreds of megabytes to store 10000 MersenneTwisters, so that is suboptimal. We should also provide the option for the user to use a DESPOTRandomNumber if they only need that and want higher performance.

@mykelk @rejuvyesh @etotheipluspi any advice? It might be hard to make this both efficient and flexible.

Execution of some RockSample scenarios does not terminate properly

For example, executing RockSample(5,6) with the default problem settings results in the rover cycling between two adjacent cells without ever reaching a terminal state.

No Argument-less Constructor

Hello! I'm trying to get to grips with the DESPOT solver. I notice that running solver = DESPOTSolver() yields a MethodError: no method matching DESPOT.DESPOTSolver().

I've seen some examples where the arguments are defined and the constructor is called, but wouldn't it be nice to have an argument-less default constructor? Would it be unreasonable to expect one, given DESPOT's theoretical framework? Is there no set of "reasonable" arguments that could be used by default?

Travis/Coveralls

Wanted to close JuliaPOMDP/POMDPs#61 , and the only reason it was still open is because Max wanted us to add travis and coveralls to our solvers. I figured here is a better place to have that issue

Tree visualization

Hey @ebalaban , I think it would be useful to be able to visualize the tree. Do you have any way to do that setup now or did you have any plans to? If you don't, I am thinking we could just use the same javascript as MCTS and POMCP use. Should I start working on that?

Maintain perfect repeatability with C implementation

As I understand it, all the ccalls to rand_r are for the purpose of being able to exactly reproduce the same results as the C++ implementation. Is that correct?

Do we want to maintain this compatibility? I think maintaining it will make it much more difficult to add functionality.

I think there are 3 options

1. Add features while maintaining compatibility
2. forget compatibility
3. Fork the code and maintain one branch for compatibility and benchmarking and maintain another branch that works with a wider variety of pomdps and options.

What do you think @ebalaban ? I think my vote is for 2 or 3.

eta solver parameter in docs

Hi @ebalaban, can you take a moment to clarify the documentation for the solver parameters?

Is the DESPOTSolver parameter pruning_constant the regularization constant that is represented by \lambda in the papers? (I am pretty sure of this one)
What is the DESPOTSolver parameter eta? Is it the \xi or \epsilon_0 in Algorithm 6 of this paper: https://www.jair.org/media/5328/live-5328-9753-jair.ps?

Thanks!

testing for action == -1

There is some code in DESPOT.jl that checks for action == -1 to decide what to do next. I am assuming that this is indicating some sort of exceptional circumstance. This is problematic for a couple reasons (I'm sure these have been considered but just haven't been fixed yet).

1. There could be problems for which -1 is a valid action, and
2. if the actions are not integers, there could be type stability issues.

We should fix this eventually, perhaps by having the functions return a Nullable{A}.

pruning

Hi @ebalaban, do you remember anything about the status of pruning? Right now the prune functions are commented out with #TODO: fix pruning at the top. Do you remember anything about what needed to be fixed? It's ok if not; I just didn't want to jump in and start debugging it if you had a more specific idea of what needs to be fixed.

Thanks!

POMDPs.State, POMDPs.Action, etc. in README

Hey Ed, I was just reading the readme and I noticed you forgot to remove some mentions POMDPs.State etc. after the changes to the API - just thought I'd give you a heads up so you remember to remove it eventually