Giter Site home page Giter Site logo

mzy2240 / l2rpn-using-a3c Goto Github PK

View Code? Open in Web Editor NEW

This project forked from amar-iastate/l2rpn-using-a3c

0.0 1.0 0.0 31.97 MB

Reinforcement Learning using the Actor-Critic framework for the L2RPN challenge (https://l2rpn.chalearn.org/ & https://competitions.codalab.org/competitions/22845#learn_the_details-overview). The agent trained using this code was one of the winners of the challenge. The code runs on the pypownet environment (https://github.com/MarvinLer/pypownet). It is released under a license of LGPLv3

License: GNU Lesser General Public License v3.0

MATLAB 20.23% Python 79.77%

l2rpn-using-a3c's Introduction

L2RPN-using-A3C

Reinforcement Learning using the Actor-Critic framework for the L2RPN challenge (https://l2rpn.chalearn.org/ & https://competitions.codalab.org/competitions/22845#learn_the_details-overview). The agent trained using this code was one of the winners of the challenge. The code uses the pypownet environment (https://github.com/MarvinLer/pypownet). The code is released under a license of LGPLv3.

Requirements

  • Python >= 3.6
  • Keras
  • pypownet
  • Virtual Environment (conda/venv) Recommended

Pypownet Installation and Documentation : https://github.com/MarvinLer/pypownet

Explaination of Files

  • PDF Files
    • Amar_L2RPN_IJCNN_git.pdf - Presentation of the method at IJCNN-2019 in the L2RPN workshop. Summarizes the idea beind the approach and the training methodology.
  • Numpy Files
    • valid_actions_array_uniq.npz - matrix of valid unique actions
    • valid_actions_masking_subid_perm.npz - matrix that maps the substation-Ids to the unique valid actions to be used for masking the output of the actor
  • Python Files
    • valid_switching_controls.py - python file that creates the numpy files explained above
    • pypow_14_a3c_final.py - python file used to train the actor & critic neural networks using A3C
  • Chronic Datasets in public_data folder
    • datasets - Original chronics data given by the L2RPN contest
    • datasets_sub_4 - Subsampled chronics from the original data by 4
    • datasets_sub_7 - Subsampled chronics from original data by 7
    • you can create other subsamples by modifying the value of the 'sub_sample' value in the matlab file create_sub_files.m

Usage

Training your own A3C model

python pypow_14_a3c_final.py

This will create two new files

  • pypow_14_a3c_actor.h5 - The weights of the actor neural network
  • pypow_14_a3c_critic.h5 - The weights of the critic neural network

Key Hyper-Parameter Tuning for Training

To speed up the learning, the enviornment difficulty level is slowly increased and the following hyper-parameters in the code can be used to make the environment difficult or easy

  • game_level_global - chooses subsampled data so that the agents can see data from farther in the dataset
  • game_over_mode_global - controls the behavior of the lines in the environment
  • chronic_loop_mode_global - controls how the envirnment 'Reset' function will behave

License information

Copyright 2019 Amarsagar Reddy Ramapuram Matavalam

This Source Code is subject to the terms of the GNU Lesser General Public License v3.0. If a copy of the LGPL-v3 was not distributed with this file, You can obtain one at https://www.gnu.org/licenses/lgpl-3.0.html.

l2rpn-using-a3c's People

Contributors

amar-iastate avatar

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.