rl-exploration
Reinforcement Learning papers on exploration methods.
Count-based
Count-based exploration is a category of exploration methods that encourage the agent to explore novel states by memorizing states' visitation counts.
Title | AKA | Author | Date | Links |
---|---|---|---|---|
Exploration in Model-based Reinforcement Learning by Empirically Estimating Learning Progress | - | Lopes et al. | 2012.12 | [Paper] |
An Analytic Solution to Discrete Bayesian Reinforcement Learning | BEETLE | Poupart et al. | 2006.06 | [Paper] |
Unifying Count-Based Exploration and Intrinsic Motivation | DQN-CTS | Bellmare et al. | 2016.06 | [Paper] |
#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning | - | Tang et al. | 2016.11 | [Paper] |
Count-Based Exploration with Neural Density Models | DQN-PixelCNN | Ostrovski et al. | 2017.03 | [Paper] |
Novel behavior-based
Novel behavior-based is a category of exploration methods that encourage the agent to perform novel behaviors. Behaviors are specified by application-specific behavior-characteristic. For eg. distance traveled or speed of agent.
Title | AKA | Author | Date | Links |
---|---|---|---|---|
Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents | NS-ES | Conti et al. | 2017.12 | [Paper] |
Novelty-Guided Reinforcement Learning via Encoded Behaviors | - | Ramamurthy et al. | 2020.07 | [Paper] |
Guided Reinforcement Learning via Sequence Learning | - | Ramamurthy et al. | 2020.10 | [Paper] |
Curiosity
Curiosity-driven exploration is a category of exploration methods that "generate an intrinsic reward signal based on how hard it is for the agent to predict the consequences of its own actions, i.e. predict the next state given the current state and the executed action." (Pathak et al., 2017)
Title | AKA | Author | Date | Links |
---|---|---|---|---|
Curious Model-building Control Systems | - | Schmidhuber | 1991.11 | [Paper] |
Reinforcement-Driven Information Acquisition in Non-deterministic Environments | - | Storck et al. | 1995 | [Paper] |
Formal Theory of Creativity, Fun, and Intrinsic Motivation (1990โ2010) | - | Schmidhuber | 2010.07 | [Paper] |
Planning to Be Surprised: Optimal Bayesian Exploration in Dynamic Environments | - | Sun et al. | 2011.03 | [Paper] |
VIME: Variational Information Maximizing Exploration | VIME | Houthooft et al. | 2016.05 | [Paper] |
Curiosity-driven Exploration by Self-supervised Prediction | ICM | Pathak et al. | 2017.05 | [Paper] |
Large-Scale Study of Curiosity-Driven Learning | ICM | Burda et al. | 2018.08 | [Paper] |
Misc.
Title | AKA | Author | Date | Links |
---|---|---|---|---|
Intrinsically Motivated Reinforcement Learning | - | Singh et al. | 2004 | [Paper] |
A Theoretical Analysis of Model-Based Interval Estimation | MBIE | Strehl and Littman | 2005.08 | [Paper] |
An Analysis of Model-based Interval Estimation for Markov Decision Processes | MBIE-EB | Strehl and Littman | 2008.09 | [Paper] |
Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models | - | Stadie et al. | 2015.07 | [Paper] |
Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning | - | Mohamed and Rezende | 2015.09 | [Paper] |