yxfff / falcon_data_pipeline Goto Github PK
View Code? Open in Web Editor NEWThis project forked from treselle-systems/falcon_data_pipeline
In our use case, we have used Apache Falcon to centrally define data pipelines, and then Falcon uses those definitions to auto-generate workflows in Apache Oozie. Falcon data flows are sinking with Atlas through Kafka topics so Atlas knows about Falcon metadata. Atlas provides Falcon feed lineage and it can tell what table was the source for another table.