Hi folks, its my implementation for MapReduce paper. The code bases on Lab 1 6.824, but I modified a little bit. You can refer my summary MapReduce paper .
Besides the basic requirements of 6.824, I added a couple of things. Here it is:
- Task Dispatch ( Map & Reduce task )
- Parallelism
- Heartbeat
- Failed and Straggler Detection
- Re-execution ( failed and straggler workers)
- Skipping Bad Records
- Status Information
- Counters
For master: Linux, macOS & Window:
go run mrmaster.go pg-.*txt
For multiple workers: Linux, macOS & Window::
go run mrworker.go
go run mrworker.go
The source code for the site is licensed under the MIT license, which you can find in the MIT-LICENSE.txt file.