lidesheng1949 Goto Github PK

big_data_resources

:boom:大数据/数据挖掘/推荐系统/机器学习相关资源

cdhproject

hadoop各组件使用，持续更新

dataflow

A framework that lets you easily create spark ETL jobs using simple configuration files

dataflow-analysis

数据流分析技术工程汇总

datalink

简单易用的ETL工具

dr-elephant

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

examples-spark

flink-learning

flink learning blog. http://www.flink-learning.com 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

flink-training-course

Flink 中文视频课程（持续更新...）

git-for-win

Git for Windows. 国内直接从官网下载比较困难，需要翻墙。这里提供一个国内的下载站，方便网友下载

hadoop-deployer

hadoop (hadoop,hive,hue,hbase) deployer

hellogithub

:octocat: Find pearls on open-source seashore 分享 GitHub 上有趣、入门级的开源项目

interview_bigdata

关于大数据的面试题，包括hadoop、hbase、hive、spark、storm、zookeeper、kafka、flume、logstash、redis、ELK、ETL、算法等等，持续更新中

ip2region

Ip2region is a offline IP location library with accuracy rate of 99.9% and 0.0x millseconds searching performance. DB file is ONLY a few megabytes with all IP address stored. binding for Java,PHP,C,Python,Nodejs,Golang,C#,lua. Binary,B-tree,Memory searching algorithm

ipdb-java

IPIP.net officially supported IP database ipdb format parsing library

kafkaoffsetmonitor

A little app to monitor the progress of kafka consumers and their lag wrt the queue.

metorikku

A simplified, lightweight ETL Framework based on Apache Spark

python-small-examples

告别枯燥，致力于打造 Python 实用小例子

setl

A simple Spark-powered ETL framework that just works 🍺

source-code-hunter

😱 从源码层面，剖析挖掘互联网行业主流技术的底层实现原理，为广大开发者 “提升技术深度” 提供便利。目前开放 Spring 全家桶，Mybatis、Netty、Dubbo 框架，及 Redis、Tomcat 中间件等

spark-etl

Apache Spark based ETL Engine

spark-etl-1

Set of ETL utils for Spark

spark-etl-2

A collection of examples from my blog post

spark-etl-demo

Demo of an ETL Spark Job

spark2-etl-examples

A project with examples of using few commonly used data manipulation/processing/transformation APIs in Apache Spark 2.0.0

sparkstreaming-kafka

:boom: :rocket: 封装spark读取kafka，sparkstreaming动态调节batch time；封装sparkstreaming 1.6 - kafka 010 用以支持 SSL。

sparkstreamingkafkaperformance

spark将hdfs数据高性能灌入kafka，然后spark streaming/structured streaming高速消费，关注性能，欢迎提供性能/代码优化建议

tool-tips

idea 中相见恨晚的技巧 :poop: :poop::poop::poop::poop::poop: 文档访问地址 http://atips.cn/idea/

utils4s

scala、spark使用过程中，各种测试用例以及相关资料整理

yarnmonitor