This project forked from lujinda/mapreduce
go实现多机版MapReduce
参考http://oserror.com/distributed/mapreduce-implementation-in-golang/?utm_source=tuicool&utm_medium=referral 这篇文章 , 把它从单机版本, 改成了多机分布式版本.(单master, 多worker)
主要流程是:
master接收数据分片, 下发文件路径给map worker
map worker根据路径向master获取分片数据, 以分片文件名保存到本地
每次map worker计算完数据后, 将本地的结果中间文件路径注册到master中
reduce worker向master查询map结果中间文件所有的map worker路径. 然后向map worker获取中间结果文件.
reduce worker处理完数据后, 将本地的结果文件路径再汇报给master
master在合并文件时, 向所有的reduce worker索要结果文件, 下载到本地, 执行合并
功能比较low, 就图个好玩哈.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.
