Topic: elt Goto Github

👇 Here are 299 public repositories matching this topic...

173tech / sayn
elt,Data processing and modelling framework for automating tasks (incl. Python & SQL transformations).
Organization: 173tech
Home Page: https://173tech.github.io/sayn
analytics automation data-engineering data-extraction data-modeling data-science elt etl python sql

airbytehq / airbyte
elt,The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Organization: airbytehq
Home Page: https://airbyte.com
data pipeline data-analysis data-engineering java python etl change-data-capture data-collection data-integration

airbytehq / airflow-summit-airbyte-2022
elt,git push your data stack with Airbyte, Airflow, and dbt - 2022 Airflow Summit
Organization: airbytehq
Home Page: https://www.airbyte.com
airbyte airflow data-engineering elt

airbytehq / pyairbyte
elt,PyAirbyte brings the power of Airbyte to every Python developer.
Organization: airbytehq
Home Page: https://docs.airbyte.com/pyairbyte
data-engineering elt python

andrewtavis / wikirepo
elt,Python based Wikidata framework for easy dataframe extraction
User: andrewtavis
wikidata open-source social-sciences statistics data-science repository wikipedia analytics political-science demography

apache / airflow
elt,Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Organization: apache
Home Page: https://airflow.apache.org/
airflow apache apache-airflow python scheduler workflow automation dag data-engineering data-integration

apache / doris
elt,Apache Doris is an easy-to-use, high performance and unified analytics database.
Organization: apache
Home Page: https://doris.apache.org
olap database hadoop hive hudi iceberg real-time sql bigquery dbt

apache / flink-cdc
elt,Flink CDC is a streaming data integration tool
Organization: apache
Home Page: https://nightlies.apache.org/flink/flink-cdc-docs-stable
change-data-capture cdc batch data-integration data-pipeline distributed elt etl flink kafka mysql paimon postgresql real-time schema-evolution

apache / seatunnel
elt,SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
Organization: apache
Home Page: https://seatunnel.apache.org/
data-integration high-performance offline real-time apache batch cdc change-data-capture data-ingestion elt

artie-labs / transfer
elt,Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
Organization: artie-labs
Home Page: https://artie.com
snowflake cdc change-data-capture golang bigquery kafka apache-kafka data-integration data-pipelines database

ascrus / getl
elt,A tool for developing and testing ETL and ELT processes for automating the capture, delivery and processing of information in data warehouses on the MicroFocus Vertica platform.
User: ascrus
csv dsl elt etl excel hdfs hive impala json kafka sql unit-testing vertica xml

astronomer / astro-sdk
elt,Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.
Organization: astronomer
Home Page: https://astro-sdk-python.rtfd.io/
python sql pandas airflow sqlite bigquery postgres snowflake gcs s3

aws-samples / aws-etl-orchestrator
elt,A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda.
Organization: aws-samples
aws glue etl elt stepfunctions orchestration lambda serverless big-data bigdata extract transform load athena amazon-web-services state-machine

brooklyn-data / meltano-on-github-actions
elt,Cookiecutter template for creating GitHub Actions orchestrated Meltano projects
Organization: brooklyn-data
meltano github-actions elt

cloudquery / cloudquery
elt,The open source high performance ELT framework powered by Apache Arrow
Organization: cloudquery
Home Page: https://cloudquery.io
aws gcp azure sql data-integration elt etl etl-framework bigquery data-collection

codeforkjeff / dbt-sqlite
elt,A SQLite adapter plugin for dbt (data build tool)
User: codeforkjeff
dbt elt etl sqlite

cuebook / cuelake
elt,Use SQL to build ELT pipelines on a data lakehouse.
Organization: cuebook
Home Page: https://cuelake.cuebook.ai
apache-iceberg apache-spark data-engineering data-ingestion data-integration data-lake data-pipeline data-transfer datalake delta elt etl incremental-updates lakehouse pipelines spark-sql sql upsert zeppelin-notebook

datacoves / dbt-coves
elt,CLI tool for dbt users to simplify creation of staging models (yml and sql) files
Organization: datacoves
Home Page: https://pypi.org/project/dbt-coves/
analytics bigquery datacoves dbt elt etl jinja python redshift snowflake sql

dataforgelabs / dataforge-core
elt,DataForge helps data teams write functional transformation pipelines by leveraging software engineering principles
Organization: dataforgelabs
Home Page: https://www.dataforgelabs.com
analytics data-modeling elt elt-pipeline etl etl-framework

dataform-co / dataform
elt,Dataform is a framework for managing SQL based data operations in BigQuery
Organization: dataform-co
Home Page: https://cloud.google.com/dataform/docs
data-pipelines elt data-engineering business-intelligence analytics etl hacktoberfest

datarecce / recce
elt,Data Reconnaissance - pull request review tool for dbt projects
Organization: datarecce
Home Page: https://datarecce.io
dbt elt codereview

datasphere-oss / datasphere-integration
elt,an data-centric integration platform
Organization: datasphere-oss
data-bus data-integration data-interchange data-sharing elt esb etl kettle realtime-messaging

datavault-uk / automate-dv
elt,A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
Organization: datavault-uk
Home Page: https://www.automate-dv.com
data-vault dataengineering datalake datavault datavault20 datawarehouse datawarehousing dbt elt etl metadata snowflake sql

datayoga-io / datayoga
elt,streaming data pipeline platform
Organization: datayoga-io
Home Page: https://datayoga-io.github.io/datayoga/
etl pipeline transformation cdc sqlalchemy kafka singer elt data database python redis cassandra sql

dbt-labs / dbt-core
elt,dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Organization: dbt-labs
Home Page: https://getdbt.com
dbt-viewpoint slack pypa data-modeling business-intelligence analytics elt

dlt-hub / dlt
elt,data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
Organization: dlt-hub
Home Page: https://dlthub.com/docs
data python data-engineering data-lake data-loading data-warehouse elt extract load transform

faros-ai / airbyte-connectors
elt,Airbyte connectors (sources & destinations) + Airbyte CDK for JavaScript/TypeScript
Organization: faros-ai
npm airbyte cdk typescript javascript faros connectors airbyte-connectors etl elt

firebolt-db / dbt-firebolt
elt,The dbt adapter for Firebolt
Organization: firebolt-db
firebolt dbt data-modeling transformation elt data-warehouse

franloza / coches-net-dashboard
elt,Sample project that use Dagster, dbt, DuckDB and Dash to visualize car and motorcycle Spanish market
User: franloza
dagster data dataengineering duckdb elt pipeline scraping visualization

gouline / dbt-metabase
elt,dbt + Metabase integration
User: gouline
Home Page: https://pypi.org/project/dbt-metabase/
dbt metabase python analytics business-intelligence data data-modelling elt pypa vizualisation

guidok91 / spark-movies-etl
elt,Spark data pipeline that processes movie ratings data.
User: guidok91
apache-airflow data-engineering data-pipeline delta-lake elt etl poetry pyspark spark

harrystech / arthur-redshift-etl
elt,ELT Code for your Data Warehouse
Organization: harrystech
arthur data-engineering etl open-source aws elt

jupyter-naas / drivers
elt,Low-code Python library enabling access to APIs, tools, data sources in seconds.
Organization: jupyter-naas
connectors data data-analysis data-collection data-engineering data-integration data-mesh data-science elt etl hacktoberfest hacktoberfest2022 integration lowcode open-source opensource pipeline pipelines python

kestra-io / kestra
elt,Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Organization: kestra-io
Home Page: https://kestra.io
data data-engineering data-integration data-orchestration data-orchestrator data-pipeline data-quality elt etl low-code orchestration pipeline reverse-etl scheduler workflow workflow-engine

kuwala-io / kuwala
elt,Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times
Organization: kuwala-io
Home Page: https://kuwala.io
data data-integration data-science open-data spatial-analysis elt kuwala open-source scraping dbt

mage-ai / mage-ai
elt,🧙 Build, run, and manage data pipelines for integrating and transforming data.
Organization: mage-ai
Home Page: https://www.mage.ai/
machine-learning artificial-intelligence data data-engineering data-science python elt etl pipelines data-pipelines

matts966 / alphasql
elt,AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery
User: matts966
bigquery zetasql typechecker parallelization graphviz dot-language datawarehouse datawarehouseautomation docker workflow-automation

meltano / meltano
elt,Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
Organization: meltano
Home Page: https://meltano.com/
dataops dataops-platform elt open-source opensource data pipelines extract-data meltano meltano-sdk

mundipagg / amora-data-build-tool
elt,Amora Data Build Tool enables analysts and engineers to transform data on the data warehouse (BigQuery) by writing Amora Models that describe the data schema using Python's "PEP484 - Type Hints" and select statements with SQLAlchemy. Amora is able to transform Python code into SQL data transformation jobs that run inside the warehouse.
Organization: mundipagg
Home Page: https://mundipagg.github.io/amora-data-build-tool
bigquery python transformation data-modeling business-intelligence elt analytics data-engineering datacleaning dataquality

osalvador / replicadb
elt,ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases
User: osalvador
Home Page: https://osalvador.github.io/ReplicaDB/
cross-platform database database-replication elt etl fast java kafka mysql nosql nosql-databases oracle postgresql sql transactional

quarylabs / quary
elt,Open-source BI for engineers
Organization: quarylabs
Home Page: https://www.quary.dev
analytics business-intelligence data-modeling elt big-data

raystack / optimus
elt,Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
Organization: raystack
Home Page: https://raystack.github.io/optimus
airflow etl workflows automation golang bigquery data-warehouse analytics data-modelling analytics-engineering

rudderlabs / rudder-server
elt,Privacy and Security focused Segment-alternative, in Golang and React
Organization: rudderlabs
Home Page: https://www.rudderstack.com/
bigquery cdp customer-data customer-data-lake customer-data-pipeline customer-data-platform data-engineering data-integration data-pipeline data-synchronization data-warehouse elt etl event-streaming privacy redshift segment-alternative snowflake warehouse-management warehouse-native

slingdata-io / sling-cli
elt,Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.
Organization: slingdata-io
Home Page: https://docs.slingdata.io
elt etl extract load

tobikodata / sqlmesh
elt,Efficient data transformation and modeling framework that is backwards compatible with dbt.
Organization: tobikodata
Home Page: https://sqlmesh.com
dataops elt etl sql python dataengineering transformation dbt

umitkaanusta / reddit-detective
elt,Play detective on Reddit: Discover political disinformation campaigns, secret influencers and more
User: umitkaanusta
Home Page: https://reddit-detective.readthedocs.io
analysis analytics api data database elt etl graph graph-database neo4j network politics reddit social social-media social-network

unytics / airbyte_serverless
elt,Airbyte made simple (no UI, no database, no cluster)
User: unytics
airbyte bigquery data-warehouse elt etl data data-analysis data-engineering pipeline

vmware / versatile-data-kit
elt,One framework to develop, deploy and operate data workflows with Python and SQL.
Organization: vmware
data-science data-engineering sql trino data-lineage etl elt data-pipelines data-engineer data-warehouse warehouse analytics snowflake dataops data-engineering-pipeline python data pipeline data-structures database

yokawasa / databricks-notebooks
elt,Collection of Sample Databricks Spark Notebooks ( mostly for Azure Databricks )
User: yokawasa
databricks azure azuredatabricks python spark elt streaming

zsvoboda / dbd
elt,dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
User: zsvoboda
elt sql database database-schemas etl csv json parquet xlsx xls excel python python3 postgresql mysql sqlite snowflake bigquery redshift