Topic: bigdata Goto Github

👇 Here are 2043 public repositories matching this topic...

absaoss / spline
bigdata,Data Lineage Tracking And Visualization Solution
Organization: absaoss
Home Page: https://absaoss.github.io/spline/
spark lineage tracking visualization bigdata hadoop scala

apache / amoro
bigdata,Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.
Organization: apache
Home Page: https://amoro.apache.org/
bigdata datalake lakehouse

apache / avro
bigdata,Apache Avro is a data serialization system.
Organization: apache
Home Page: https://avro.apache.org/
csharp ruby cplusplus python php java c avro bigdata dotnet perl rust

apache / celeborn
bigdata,Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
Organization: apache
Home Page: https://celeborn.apache.org/
shuffle spark bigdata

apache / hudi
bigdata,Upserts, Deletes And Incremental Processing on Big Data.
Organization: apache
Home Page: https://hudi.apache.org/
hudi apachehudi datalake bigdata apachespark incremental-processing stream-processing data-integration apacheflink

apache / incubator-livy
bigdata,Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
Organization: apache
Home Page: https://livy.apache.org/
livy bigdata spark apachelivy

apache / shardingsphere
bigdata,Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
Organization: apache
database distributed-database distributed-sql-database distributed-transactions sql shard database-cluster mysql postgresql database-plus

bigartm / bigartm
bigdata,Fast topic modeling platform
Organization: bigartm
Home Page: http://bigartm.org/
topic-modeling c-plus-plus python bigartm regularizer python-api text-mining machine-learning bigdata

byzer-org / byzer-lang
bigdata,Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.
Organization: byzer-org
Home Page: https://www.byzer.org
machine-learning mlsql bigdata sql-like-dsl

collabh / bigdata-growth
bigdata,大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
User: collabh
flink kafka hive mapreduce spark olap kudu hadoop hbase debezium

dataexpert-io / data-engineer-handbook
bigdata,This is a repo with links to everything you'd ever want to learn about data engineering
Organization: dataexpert-io
apachespark awesome bigdata data dataengineering sql

datafuselabs / databend
bigdata,𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
Organization: datafuselabs
Home Page: https://docs.databend.com
rust database serverless bigdata snowflake ai lakehouse olap

dotnet / spark
bigdata,.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
Organization: dotnet
Home Page: https://dot.net/spark
spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core

douban / dpark
bigdata,Python clone of Spark, a MapReduce alike framework in Python
Organization: douban
bigdata mapreduce dpark stream-processing spark python

dtstack / chunjun
bigdata,A data integration framework
Organization: dtstack
Home Page: https://dtstack.github.io/chunjun/
bigdata data-integration flink framework java

dtstack / flinkstreamsql
bigdata,基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
Organization: dtstack
flink sql stream bigdata

fdv / running-elasticsearch-fun-profit
bigdata,A book about running Elasticsearch
User: fdv
Home Page: https://fdv.github.io/running-elasticsearch-fun-profit/
elasticsearch documentation sysadmin ebook bigdata

gangly / datafaker
bigdata,Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具
User: gangly
mysql bigdata postgresql kafka datafaker testing hive hbase python fakedata

gearpump / gearpump
bigdata,Lightweight real-time big data streaming engine over Akka
Organization: gearpump
Home Page: https://gearpump.github.io/gearpump/
akka bigdata scala stream-processing

griddb / griddb
bigdata,GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.
Organization: griddb
Home Page: https://griddb.org/
bigdata database fast griddb iot newsql nosql sql time-series timeseries

heibaiying / bigdata-notes
bigdata,大数据入门指南 :star:
User: heibaiying
hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka

hi-primus / optimus
bigdata,:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Organization: hi-primus
Home Page: https://hi-optimus.com
spark pyspark data-wrangling bigdata big-data-cleaning data-science data-cleansing data-cleaner data-transformation machine-learning

igaowei / bigdataview
bigdata,100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中
User: igaowei
Home Page: https://igaowei.github.io/BigDataView/
bigdataviewer html-template echarts bigdata viewmodel

intsmaze / flink-boot
bigdata,懒松鼠Flink-Boot 脚手架让Flink全面拥抱Spring生态体系，使得开发者可以以Java WEB开发模式开发出分布式运行的流处理程序，懒松鼠让跨界变得更加简单。懒松鼠旨在让开发者以更底上手成本（不需要理解分布式计算的理论知识和Flink框架的细节）便可以快速编写业务代码实现。为了进一步提升开发者使用懒松鼠脚手架开发大型项目的敏捷的度，该脚手架默认集成Spring框架进行Bean管理，同时将微服务以及WEB开发领域中经常用到的框架集成进来，进一步提升开发速度。比如集成Mybatis ORM框架，Hibernate Validator校验框架,Spring Retry重试框架等，具体见下面的脚手架特性。
User: intsmaze
flink bigdata java mcv sping spring-boot mybatis spring-retry flink-boot java-flink

jadianes / spark-movie-lens
bigdata,An on-line movie recommender using Spark, Python Flask, and the MovieLens dataset
User: jadianes
big-data bigdata flask movie-recommendation movielens-dataset python spark

jadianes / spark-py-notebooks
bigdata,Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
User: jadianes
Home Page: http://jadianes.github.io/spark-py-notebooks
big-data bigdata data-analysis data-science ipython ipython-notebook machine-learning mllib notebook pyspark python spark

josonle / coding-now
bigdata,学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等
User: josonle
ebook-collection notes coding java hadoop-hive spark linux bigdata

juicedata / juicefs
bigdata,JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Organization: juicedata
Home Page: https://juicefs.com
filesystem cloud-native golang redis distributed-systems storage object-storage posix hdfs s3

kubernetes-retired / kube-batch
bigdata,A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC
Organization: kubernetes-retired
bigdata hpc k8s-sig-scheduling kubernetes machine-learning

liyupi / sql-generator
bigdata,🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
User: liyupi
Home Page: http://sql.yupi.icu
ant-design bigdata hive javascript json monaco-editor mysql spark sql typescript vite vue vue3

microsoft / mobius
bigdata,C# and F# language binding and extensions to Apache Spark
Organization: microsoft
spark apache-spark rdd dataframe dstream dataset streaming csharp mobius kafka-streaming spark-streaming fsharp bigdata mapreduce eventhubs near-real-time

moran1607 / bigdataguide
bigdata,大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
User: moran1607
hadoop zookeeper hive flume kafka hbase spark flink scala bigdata

nationalsecurityagency / datawave
bigdata,DataWave is an ingest/query framework that leverages Apache Accumulo to provide fast, secure data access.
Organization: nationalsecurityagency
Home Page: https://code.nsa.gov/datawave
accumulo bigdata java

netflix / genie
bigdata,Distributed Big Data Orchestration Service
Organization: netflix
Home Page: https://netflix.github.io/genie
big-data bigdata orchestration configuration configuration-management java spring-boot distributed-systems netflixoss cloud

opendatadiscovery / odd-platform
bigdata,First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.
Organization: opendatadiscovery
Home Page: https://opendatadiscovery.org
alerting bigdata data-catalog data-discovery data-engineering data-exploration data-governance data-lineage data-observability data-pipelines data-platform data-profiling data-quality data-science datacatalog lineage metadata metadata-management observability oss

oxnr / awesome-bigdata
bigdata,A curated list of awesome big data frameworks, ressources and other awesomeness.
User: oxnr
Home Page: https://github.com/onurakpolat/awesome-bigdata
awesome bigdata data-warehouse data-analytics data-visualization data-stream database data-science data awesome-list

pingcap / tispark
bigdata,TiSpark is built for running Apache Spark on top of TiDB/TiKV
Organization: pingcap
bigdata spark tidb tikv

seagate / cortx
bigdata,CORTX Community Object Storage is 100% open source object storage uniquely optimized for mass capacity storage devices.
Organization: seagate
Home Page: https://github.com/Seagate/cortx
s3-storage software-defined-storage storage storage-api object-storage object-storage-service objectstorage objectstore opensource open-source

shzlw / poli
bigdata,An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.
User: shzlw
Home Page: https://shzlw.github.io/poli
bigdata business-intelligence dashboard data-visualization jdbc reactjs reporting spring-boot sql sql-editor

taosdata / tdengine
bigdata,High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
Organization: taosdata
Home Page: https://tdengine.com
iot bigdata time-series database industrial-iot connected-vehicles monitoring tsdb tdengine sql

tensorbase / tensorbase
bigdata,TensorBase is a new big data warehousing with modern efforts.
Organization: tensorbase
Home Page: https://tensorbase.io/
rust bigdata database analytics modern infrastructure data data-infrastructure high-performance engineering

vaexio / vaex
bigdata,Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
Organization: vaexio
Home Page: https://vaex.io
dataframe python bigdata tabular-data visualization memory-mapped-file hdf5 machine-learning machinelearning data-science

visualpython / visualpython
bigdata,GUI-based Python code generator for data science, extension to Jupyter Lab, Jupyter Notebook and Google Colab.
User: visualpython
Home Page: https://www.visualpython.ai
bigdata chrome-extension code-generator data-analysis jupyter-lab-extension jupyter-notebook-extension jupyterlab-extension pandas python visual-coding

volcano-sh / volcano
bigdata,A Cloud Native Batch System (Project under CNCF)
Organization: volcano-sh
Home Page: https://volcano.sh
batch-systems kubernetes golang hpc bigdata machine-learning gene

wadestack / bigdataie
bigdata,大数据博客、笔试题、教程、项目、面经的整理
User: wadestack
Home Page: https://wadestack.github.io/BigDataIE/
offer spark java work bigdata

wangzhiwubigdata / god-of-bigdata
bigdata,专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
User: wangzhiwubigdata
flink spark hadoop hdfs hive hbase kafka zookeeper bigdata flume

water8394 / bigdata-interview
bigdata,:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
User: water8394
bigdata flink hadoop hbase hdfs interview interview-questions kafka mapreduce spark yarn

webankfintech / wedatasphere
bigdata,WeDataSphere is a financial grade, one-stop big data platform suite.
Organization: webankfintech
analytics bigdata data-analysis datafabric datagovernance dataspherestudio exchangis flink hadoop hive ide linkis prophecis qualitis schedulis scriptis spark streamis visualis

yoongikim / autocrawler
bigdata,Google, Naver multiprocess image web crawler (Selenium)
User: yoongikim
crawler selenium deep-learning image-crawler chromedriver bigdata google multiprocess thread customizable

zeromicro / cds
bigdata,Data syncing in golang for ClickHouse.
Organization: zeromicro
bigdata clickhouse go golang kafka-consumer