Chetan Khobragade's Projects
it takes file to dataflow to bigquery then to cloud storage bucket
Cloud function which gets triggered on new file upload on GCS, this function specifically triggers data flow pipeline which processes the file
Google IT Automation with Python Professional Certificate - Practice files
It is a mini implementation of case study of MountKirk Games on GCP.
labs guide for completing #juaragcp challenge
This repo is for demonstration purposes only.
Extract
Example Terraform configuration
You will now build a Dataflow pipeline with a TextIO source and a BigQueryIO destination to ingest data into BigQuery. More specifically, you will: Ingest the files from Cloud Storage. Convert the lines read to dictionary objects. Transform the data which contains the year to a format BigQuery understands as a date. Output the rows to BigQuery.