bawcos/pomdps.jl

API for solving POMDPs in Julia

POMDPs

This package provides a basic interface for working with partially observable Markov decision processes (POMDPs).

NEWS: We recently made a significant change to the interface, introducing parametric types (see issue #56). If you wish to continue using the old interface, the v0.1 release may be used, but we recommend that all projects update to the new version.

The goal is to provide a common programming vocabulary for researchers and students to use primarily for three tasks:

1. Expressing problems using the POMDP format.
2. Writing solver software.
3. Running simulations efficiently.

For problems and solvers that only use a generative model (rather than explicit transition and observation distributions), see also GenerativeModels.jl.

Installation

Pkg.clone("https://github.com/JuliaPOMDP/POMDPs.jl.git")

Supported Solvers

The following MDP solvers support this interface:

• Value Iteration
• Monte Carlo Tree Search

The following POMDP solvers support this interface:

• QMDP
• SARSOP
• POMCP
• POMDPSolve

To install a solver run the following command:

using POMDPs
# the following command adds the SARSOP solver, you can add any supported solver this way
POMDPs.add("SARSOP") 

Tutorials

The following tutorials aim to get you up to speed with POMDPs.jl:

• MDP Tutorial for beginners gives an overview of using Value Iteration and Monte-Carlo Tree Search with the classic grid world problem
• POMDP Tutorial gives an overview of using SARSOP and QMDP to solve the tiger problem

Documentation

Detailed documentation can be found here.