equinor/deepwell

Summer student project 2020

deepwell's Issues

Create custom policy to change layers

Acceptance criteria:

• Start training with more layers

Implement env and agent using Hindsight Experience Replay

HER, a method for sparse rewards:
https://arxiv.org/pdf/1707.01495.pdf

Acceptance Criteria:

• Adapt env to GoalEnv for it to be compatible with HER
• Reach a point with Stable Baselines HER agent

Train an agent over several levels with increasing difficulty

Acceptance Criteria:

• Implement different classes that inherit properties from DeepWellEnvSpher
• Define the properties of the levels
• Make docker 'leveltrain' functionality
• Make the agent hit 5 targets and avoid 2 hazards using leveltraining

Save and retrain model

Acceptance criteria:

• Be able to load a trained model and continue training it without starting from scratch

• Tensorboard integration with new training starting where last one stopped

Obtain agent that hits targets while having 2 targets and 1 hazard in observation space

Implement environment in spherical coordinates

Acceptance criteria:

• Create an environment that uses spherical coordinates at the drill bit and let the agent change the angular acceleration

-Need to generate a smooth curve that hits the targets and avoids the hazards

-Curvature and steering actions must be minimised

Change plotting library from matplotlib to alternative to increase performance

Acceptance Criteria:

• "Drop-in" replacement for current solution. Should work without any extra work.
• New plotter has better performance than current (Not laggy)

Creating script for running docker more easily

Acceptance Criteria:

• Being able to both build and run container with simpler command

• Being able to specify flags for desired behaviour:

  • Just build
  • Just run
  • Build and run
  • Build, run and open plot in browser

Create different custom policies to figure out which network architecture works best

Acceptance criteria:

• Try out different sizes of layers and nodes and compare them to our default MlpPolicy