mk6502/spark-scala-boilerplate

Example Apache Spark 2.0.0-2.4.5 + Scala application.

Hi 👋

Hi there, I'm Mike.

• 🔭 I’m currently working on big data engineering with Spark on k8s on AWS at iSpot.tv.
• 🌱 I’m focusing on mentoring and coaching my team to improve their skills and release awesome products.
• 📚 I love to read - check out my list of books to see what I'm reading.
• 🌎 I occasionally write blog posts about software engineering and other topics.

Management and Software Engineering

I consider myself to be a software engineer at heart. Nowadays I'm trying to do less code-writing and more of everything else:

• Reading and learning.
• Writing, writing, and more writing.
• Thinking about design patterns and architecture (how do you scale k8s? what should we be researching?)
• Mentoring and leading a team building products and shipping features
• Hiring (!)

Stuff I've Worked On

• Django! My first internship and full-time job was at a startup that was built on top of Django.
• Built an Android app for fun. It was much easier than multiple failed attempts of creating an iOS app.
• Machine Learning. Just scratched the surface. Data cleansing and feature engineering really is 95% of the job.
• Built a CHIP-8 emulator. It plays Pong and that's pretty much it. Maybe I'll work on a Gameboy emulator next.
• Spark, Hadoop, Hive. On-prem, on Azure, on GCP, on AWS - you name it. Now we're running Spark on k8s. Also AWS EC2, EMR, ECS, EKS, Lambda, API Gateway, CodePipeline, VPC, IAM, CloudFormation, and plenty more acronyms.
• Now, I'm an engineering manager, managing a team that does a lot of what I wrote about above!

Everything Else

My wife and I enjoy hiking, biking, traveling, and playing with our Shih Tzu.