haroldsultan / mcts Goto Github PK
View Code? Open in Web Editor NEWPython Implementations of Monte Carlo Tree Search
Python Implementations of Monte Carlo Tree Search
According to the paper A Survey of Monte Carlo Tree Search Methods, the UCB function should look like this.
And here is the pseudo-code about the implementation.
The coefficient 2
should be outsize of the log
. And this line of code is a little different from the paper https://github.com/haroldsultan/MCTS/blob/master/mcts.py#L115 .
this is a interesting example.
but i think mcts is find a acceptable result in computational budget limit。
i try to exhaustive results,it get better result in less time.
maybe replace [2,-2,3,-3] to trick nums will be more convincing,i try it,but failed。
Hey,
I want to have a 2-player game where they take turns. In the beginning there are 114 possible actions and they decrease by 1 every time a player makes a move. The game is played for 10 turns (that's the terminal state). I have my own function for the reward.
Here is a sample game tree:
START- available actions to both players -> [1,2,3,4,5,6....112,113,114]
Player 1 - takes action 5 -> [5,0,0,0,0,0,0,0,0,0] -remove action 5 from available actions
Player 2 - takes action 32->[5,0,0,0,0,32,0,0,0,0] - remove action 32 from the available actions
Player 1- takes action 97 ->[5,97,0,0,0,32,0,0,0,0] - remove action 97 from the available actions
Player 2 takes action 56 -> [5,97,0,0,0,32,56,0,0,0] - remove action 56 from the available actions
....
Final (example) game state after each player makes 5 actions -> [5,97,3,5,1,32,56,87,101,8]
First 5 entries present the actions taken by Player1, second 5 entries present the actions taken by Player 2
Finally, I apply a reward function to this vector [5,97,3,5,1,32,56,87,101,8]
My python skill is really bad. I hope you can help with this.
This is the most easy and comprehensive implementation of MCTS I have found so far.
Can you show how to build an MCTS for time series prediction with out-of sample forecasting as well.
I cannot find this anywhere. I want to predict the stock market, which is the biggest game of all. :-D
The possible moves would be MOVES=[1, 0, -1] - I think!
Hi,
Thank you for your awesome code (mcts.py).
However, I think the algorithm is not working according to the policy network. For example, in the link you gave, under section 2.1, it is stated that it aims to find the policy that yields the highest reward.
When I ran the algorithm, the nodes/child selected were not the highest rewards. The root child selected also did not have the highest reward. Does this mean that the policy network is not optimized, and therefore, did not choose the best action/search?
E.g. as follows:
level 0
Num Children: 4
(0, Node; children: 4; visits: 354; reward: 295.106667)
(1, Node; children: 4; visits: 928; reward: 826.746667) <--- This was selected by the algorithim
(2, Node; children: 4; visits: 582; reward: 504.422222)
(3, Node; children: 4; visits: 933; reward: 831.537778) <--- Shouldn't this be selected?
Best Child: Value: 20; Moves: [20]
The input parameters I ran was 10000 loops and 8 levels.
Thank you once again for your help. Hope to hear from you soon.
James
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.