bgreen-litl / wildcatting-ai Goto Github PK

wildcatting-ai is an oil drilling agent


Agent initialization:

wcai init <agent> [--force] [--components {surveying report drilling sales}]

Creates the following filesystem structure:

agent/
    surveying/
        training/
	bootstrap.net
	gameplay.net
    report/
        training/
    	bootstrap.net
	gameplay.net
    drilling/
	training/
    	bootstrap.net
	gameplay.net
    sales/
	training/
    	bootstrap.net
	gameplay.net
    probability/
	training/
	gameplay.net
    drill_cost/
        training/
        gameplay.net

Data for supervised learning may be copied into the training directories for
each component for use by the bootstrap command. The .net files contain weights
for the neural networks that back each component.

There are four reinforcement learning components: surveying, report, drilling,
and sales. These components may be bootstrapped with supervised learning, but
can be subsequently trained using real game play rewards.

In addition, there are two components which are trainable by supervised
learning only: probability and drill_cost. These are backed by neural networks
trained autoassociatively, in order to create appropriate field models given
limited sample points. The output of these component can then be used as inputs
to the surveying component.


Training:

wcai train <agent> <component>

Perform supervised learning on an individual component.  There are two contexts
in which this may be useful. The first is to train static components that are 
not part of the reinforcement learning chain. In particular, the probability
and drill_cost reconstruction components are trained solely by reinforcement
learning.  The second context is for bootstrapping RL components to get them
pointed in the right general direction while operated under a limited but
primary subset of the input space. For example, the surveying component can be
trained on mapping from probabilty space to reservoir size space alone. During
RL, more inputs such as drill cost may be added and outputs are now updated
by actual utilities.


Bootstrapping:

wcai bootstrap <agent> <component>

Populates the gameplay NN with the weights from the training NN.  The gameplay
NN may have more inputs than the gameplay NN.


Simulation:

wcai simulate <agent> <component> [--visualize] [--file=<file>]

Apply generated inputs to the gameplay network. Outputs the inputs and output
choice.


Reinforcement learning:

wcai learn [--games <num>] [--components {surveying report drilling sales}]

Here the agent plays itself repeatedly. Some subset of the components may be
specified for update, while non updating components will remain frozen.
Specifying less than the full set of components may be advantageous when some
components have been reasonably bootstrapped while others have not.