ayang903/cloud-computing

Google Cloud Platform Assignments

These are python-based implementations of GCP products and services. Done as homework assignments for the class DS561 offered at Boston University instructed by Professor Leonidas Kontothanassis.

Here are brief summaries of each assignment:

HW1: Cloud Storage and VM Management

Perform operations on cloud services like creating and managing storage buckets, directories, and files. Manage Virtual Machines (VMs) including creation, pausing, and deletion. Get used to working with service accounts and permissions.

HW2: Data Processing and Analysis

Generate a large directory of files and analyze them for web graph properties. Implement and compute PageRank for files, and identify the top files by PageRank score. Optimize cloud resource usage and document expenses.

HW3: Cloud Functions for HTTP Requests

Develop a cloud function to handle HTTP GET requests and serve files. Implement error logging and handling for various HTTP status codes. Use a secondary app to track requests from banned countries.

HW4: Virtual Machine-based Web Server

Transition from cloud functions to a VM-based web server to handle HTTP requests. Implement logging, error handling, and request tracking from banned countries. Conduct a stress test on the server and report performance and costs.

HW5: Database Integration and Analysis

Integrate a Cloud SQL database to record detailed request information. Modify the server to process and log request data. Perform an extensive load test and analyze request data.

HW6: Predictive Modeling with Data

Retrieve data from the database and build predictive models for client IP and income. Achieve specified accuracy levels and document any challenges.

HW7: Data Processing with Apache Beam

Use Apache Beam and Cloud Data Flow for data processing tasks on cloud storage files. Find and print top files based on incoming and outgoing links. Compare runtimes between local and cloud execution.

HW8: Load Balancing and Server Failover

Set up multiple VMs as web servers in different zones behind a load balancer. Conduct failover tests and measure load balancer responsiveness.

HW9: Containerization with GKE

Port the web server to run in a container managed by Kubernetes (GKE). Ensure logging and request handling are correctly implemented. Document the containerization process, server behavior, and Kubernetes setup.

HW10: Infrastructure Automation with Google Deployment Manager

Use Google Deployment Manager to automate the deployment of the components from HW5. Modify the web server to support automated schema creation. Demonstrate functionality with an HTTP client and curl. Turn in the YAML configuration, a comprehensive deployment guide, and results documentation.