Comments (10)
This is just standard batch normalization. Normalization of data is a standard neural network procedure to help reduce variance in training as well as improve speed of training. It will still perform if trained without but slightly slower and much more variance.
from rl_a3c_pytorch.
I usually set the tau which is the lambda in the generalized advantage estimation to 0.92 and see favorable results with that setting. It allows my agent variance in exploration and easier time to find sparse rewards.
And no don't replace variables that mess up gradient assignment. I would leave that part as is but on other hand you should also do your own exploration and have fun configuring at your fancy. 👍
from rl_a3c_pytorch.
Oh. I am going to learn about BN. I have another doubt that in the fireEnv. Every time reset environment, not real done, firstly execute action 1, that is fire, that execute action 2, that is right.
And in the code always check if is is done
My doubt is that why after fire action always execute right action? And why do we need to check if it is done? Is it possible only execute 2 actions the game is done...
from rl_a3c_pytorch.
I just matched baselines wrapper for comparison. Gonna go back to my old more simpler env wrapper. The whole fire reset stuff is not really needed. All my openai top scores never used fire reset. A good agent should learn to hit the fire button itself personally I think
from rl_a3c_pytorch.
Yeah, I also think so. Thank you!
from rl_a3c_pytorch.
Could I ask if I don't use shared-optimizer, does it will cause a big influence on performance?
from rl_a3c_pytorch.
It slows down training a little (Though last time I trained with unshared was nearly a year ago so not certain on actually degree of influence). It also increases the variances of performance while training. I would suggest to use the shared optimizer as it is superior unless you have motive not too.
from rl_a3c_pytorch.
Hi,
In train.py, I saw you using generalized advantage estimation, while by my calculation, in the default model the tau is 1, so gae is equivalent to advantage function. So can I replace the Variable(gae) with advantage? Is my thought correct?
Thanks!
from rl_a3c_pytorch.
I am not sure if you mean although the gae and advantage has same value if tau is 1. But they are different variable since they belongs two different loss, one is policy loss and another is value loss. So I should construct different Variable.
Could I ask more what if they use the same variable, that is, replace gae directly with advantage, then in the back propagation, same variable take place in two loss function although they are added up to one scalar and backward.
Thanks!
from rl_a3c_pytorch.
Sorry i missed understood you. Yes if tau=1 you can think of as the “advantage estimate” but you can’t do the “advantage function” as we don’t calculate Q values in A3C
from rl_a3c_pytorch.
Related Issues (20)
- Why ensure_shared_grads HOT 1
- NotImplementedError HOT 1
- Pretrained models HOT 6
- Quick question on batch processing HOT 4
- eps for Adam HOT 2
- plot rewards as a function of number of timesteps HOT 1
- Stuck when training in MsPacman-v0 HOT 7
- Reward Smoothing HOT 2
- Need for trained models HOT 2
- Cannot import test HOT 1
- Question about Test function HOT 1
- Is there any necessary to lock when update params? HOT 1
- How can I let the training automatically stop after a given number of episodes or after a given period of time? HOT 1
- Clarification needed regarding num_workers HOT 2
- question about trained models HOT 1
- UserWarning: This overload of add_ is deprecated
- Need a model, thank you HOT 1
- The links to the Gym environment evaluations is 404.
- run a3c on 8 cpus, it still slow. HOT 1
- Hyperparameters for training
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rl_a3c_pytorch.