mauicv/evo-quad

REINFORCE-ES applied to quadruped environment

evo-quad's Issues

fix cli interface

• consolodate all cli functionality

include pretrained solutions

• Include pretrained solutions

Complete standing env with instability effects

Want the quadruped to be able to balance on multiple slopes and regain balance after being knocked.

Clean and make code presentable

• Write Readme

Remove absolution posistion from network inputs

Current the network recieves absolute x, y position data from the environment which we should replace with rate of change of x, y.

Ensure correct abstraction of training class

currently we have and Env class and then a StandingEnv and WalkingEnv class that derive from it. I think this is correct, but should be reviewed. The behavour we want is that given some behavour we want the quadruped to learn it should implemented as a EnvBehavour class. It might be possible to ensure this is the only code that needs to be touched and the training functionality is common across all training tasks.

Correct scoping for env variables

If we're training mutliple behavours how do we organise mutlple env settings and config?

Investigate input scalings effect on model behavour

Currently models output single stances but nothing like gates. I think part of this might be that the network weight initalizations mean that all the inputs are all mapped to the same output. A similar issue was present in the TF models for ddpg however which means it's something to do with how the environment differs from openai gym enviornments. Note that It's quite likely that openai gym environments scale state.

Meta behavours

Once we've trained solutions to multple tasks how do we then combine those solutions for more general problems.

1. One solution is to have a leader network that outputs a vector of weights for each sub solution. We then use this vector to weight the agent action outputs.
2. I think we'd also need a adjustment solution to train at the same time that can be used to fill in any gaps that the pretrained solutions don't capture. In particular we're not changing the pretrained solutions but rather how much infulence they have over the actual agent output.

exp cause math range error sometimes

https://www.yawintutor.com/overflowerror-math-range-error-in-python/