Topic: elt Goto Github

Some thing interesting about elt

👇 Here are 281 public repositories matching this topic...

173tech / sayn

elt,Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).

Organization: 173tech

Home Page: https://173tech.github.io/sayn

analytics etl data-modeling data-engineering data-science python sql automation elt data-extraction

airbytehq / airbyte

elt,The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.

Organization: airbytehq

Home Page: https://airbyte.com

data pipeline data-analysis data-engineering java python etl change-data-capture data-collection data-integration

airbytehq / airflow-summit-airbyte-2022

elt,git push your data stack with Airbyte, Airflow, and dbt - 2022 Airflow Summit

Organization: airbytehq

Home Page: https://www.airbyte.com

airbyte airflow data-engineering elt

airbytehq / pyairbyte

elt,PyAirbyte brings the power of Airbyte to every Python developer.

Organization: airbytehq

Home Page: https://docs.airbyte.com/pyairbyte

data-engineering elt python

andrewtavis / wikirepo

elt,Python based Wikidata framework for easy dataframe extraction

User: andrewtavis

wikidata open-source social-sciences statistics data-science repository wikipedia analytics political-science demography

apache / airflow

elt,Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Organization: apache

Home Page: https://airflow.apache.org/

airflow apache apache-airflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration python scheduler workflow workflow-engine workflow-orchestration

apache / doris

elt,Apache Doris is an easy-to-use, high performance and unified analytics database.

Organization: apache

Home Page: https://doris.apache.org

olap database hadoop hive hudi iceberg real-time sql bigquery dbt

apache / flink-cdc

elt,Flink CDC is a streaming data integration tool

Organization: apache

Home Page: https://nightlies.apache.org/flink/flink-cdc-docs-stable

change-data-capture cdc batch data-integration data-pipeline distributed elt etl flink kafka

apache / seatunnel

elt,SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

Organization: apache

Home Page: https://seatunnel.apache.org/

data-integration high-performance offline real-time apache batch cdc change-data-capture data-ingestion elt

artie-labs / transfer

elt,Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

Organization: artie-labs

Home Page: https://artie.so

snowflake cdc change-data-capture golang bigquery kafka apache-kafka data-integration data-pipelines database

ascrus / getl

elt,A tool for developing and testing ETL and ELT processes for automating the capture, delivery and processing of information in data warehouses on the MicroFocus Vertica platform.

User: ascrus

vertica etl elt unit-testing dsl csv kafka excel sql json

astronomer / astro-sdk

elt,Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

Organization: astronomer

Home Page: https://astro-sdk-python.rtfd.io/

python sql pandas airflow sqlite bigquery postgres snowflake gcs s3

aws-samples / aws-etl-orchestrator

elt,A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.

Organization: aws-samples

aws glue etl elt stepfunctions orchestration lambda serverless big-data bigdata

brooklyn-data / meltano-on-github-actions

elt,Cookiecutter template for creating GitHub Actions orchestrated Meltano projects

Organization: brooklyn-data

meltano github-actions elt

cloudquery / cloudquery

elt,The open source high performance ELT framework powered by Apache Arrow

Organization: cloudquery

Home Page: https://cloudquery.io

aws gcp azure sql data-integration elt etl etl-framework bigquery data-collection

codeforkjeff / dbt-sqlite

elt,A SQLite adapter plugin for dbt (data build tool)

User: codeforkjeff

dbt sqlite elt etl

cuebook / cuelake

elt,Use SQL to build ELT pipelines on a data lakehouse.

Organization: cuebook

Home Page: https://cuelake.cuebook.ai

apache-iceberg delta lakehouse datalake data-lake elt etl data-engineering data-integration data-ingestion

datacoves / dbt-coves

elt,CLI tool for dbt users to simplify creation of staging models (yml and sql) files

Organization: datacoves

Home Page: https://pypi.org/project/dbt-coves/

analytics bigquery datacoves dbt elt etl jinja python redshift snowflake

dataform-co / dataform

elt,Dataform is a framework for managing SQL based data operations in BigQuery

Organization: dataform-co

Home Page: https://cloud.google.com/dataform/docs

data-pipelines elt data-engineering business-intelligence analytics etl hacktoberfest

datarecce / recce

elt,Data Reconnaissance - pull request review tool for dbt projects

Organization: datarecce

Home Page: https://datarecce.io

dbt elt codereview

datasphere-oss / datasphere-integration

elt,an data-centric integration platform

Organization: datasphere-oss

esb data-bus etl elt kettle realtime-messaging data-integration data-interchange data-sharing

datavault-uk / automate-dv

elt,A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

Organization: datavault-uk

Home Page: https://www.automate-dv.com

datavault20 dbt datawarehousing datavault etl snowflake dataengineering datawarehouse datalake elt

datayoga-io / datayoga

elt,streaming data pipeline platform

Organization: datayoga-io

Home Page: https://datayoga-io.github.io/datayoga/

etl pipeline transformation cdc sqlalchemy kafka singer elt data database

dbt-labs / dbt-core

elt,dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

Organization: dbt-labs

Home Page: https://getdbt.com

dbt-viewpoint slack pypa data-modeling business-intelligence analytics elt

dlt-hub / dlt

elt,data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

Organization: dlt-hub

Home Page: https://dlthub.com/docs

data python data-engineering data-lake data-loading data-warehouse elt extract load transform

faros-ai / airbyte-connectors

elt,Airbyte connectors (sources & destinations) + Airbyte CDK for JavaScript/TypeScript

Organization: faros-ai

npm airbyte cdk typescript javascript faros connectors airbyte-connectors etl elt

firebolt-db / dbt-firebolt

elt,The dbt adapter for Firebolt

Organization: firebolt-db

firebolt dbt data-modeling transformation elt data-warehouse

franloza / coches-net-dashboard

elt,Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market

User: franloza

dagster data dataengineering duckdb elt pipeline scraping visualization

gouline / dbt-metabase

elt,dbt + Metabase integration

User: gouline

Home Page: https://pypi.org/project/dbt-metabase/

dbt metabase python analytics business-intelligence data data-modelling elt pypa vizualisation

guidok91 / spark-movies-etl

elt,Spark data pipeline that processes movie ratings data.

User: guidok91

spark pyspark etl data-engineering poetry elt apache-airflow data-pipeline delta-lake

harrystech / arthur-redshift-etl

elt,ELT Code for your Data Warehouse

Organization: harrystech

arthur data-engineering etl open-source aws elt

jupyter-naas / drivers

elt,Low-code Python library enabling access to APIs, tools, data sources in seconds.

Organization: jupyter-naas

data data-science etl integration lowcode open-source opensource pipeline pipelines data-analysis

kestra-io / kestra

elt,Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

Organization: kestra-io

Home Page: https://kestra.io

workflow workflow-engine orchestration scheduler data-pipeline elt etl data data-engineering data-orchestration

kuwala-io / kuwala

elt,Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times

Organization: kuwala-io

Home Page: https://kuwala.io

data data-integration data-science open-data spatial-analysis elt kuwala open-source scraping dbt

mage-ai / mage-ai

elt,🧙 Build, run, and manage data pipelines for integrating and transforming data.

Organization: mage-ai

Home Page: https://www.mage.ai/

machine-learning artificial-intelligence data data-engineering data-science python elt etl pipelines data-pipelines

marcosmjd / ghcn-d

elt,Data Pipeline from the Global Historical Climatology Network DataSet

User: marcosmjd

airflow bigquery dataengineering elt etl dbt googlecloudstorage googledatastudio iac terraform

matts966 / alphasql

elt,AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery

User: matts966

bigquery zetasql typechecker parallelization graphviz dot-language datawarehouse datawarehouseautomation docker workflow-automation

meltano / meltano

elt,Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

Organization: meltano

Home Page: https://meltano.com/

dataops dataops-platform elt open-source opensource data pipelines extract-data meltano meltano-sdk

mundipagg / amora-data-build-tool

elt,Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.

Organization: mundipagg

Home Page: https://mundipagg.github.io/amora-data-build-tool

bigquery python transformation data-modeling business-intelligence elt analytics data-engineering datacleaning dataquality

osalvador / replicadb

elt,ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases

User: osalvador

Home Page: https://osalvador.github.io/ReplicaDB/

java fast nosql-databases elt database-replication nosql cross-platform postgresql sql database

quarylabs / quary

elt,Transform data together. Model, test and deploy as a team.

Organization: quarylabs

Home Page: https://www.quary.dev

analytics business-intelligence data-modeling elt

raystack / optimus

elt,Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Organization: raystack

Home Page: https://raystack.github.io/optimus

airflow etl workflows automation golang bigquery data-warehouse analytics data-modelling analytics-engineering