pruet/261411

261411 Parallel Processing and Distributed Systems

Course Overview

This course introduces the theoretical and practical views of parallel programming and distributed system. On one hand, parallel processing explores the idea that there will be multiple processing units inside a computer, thus, multiple task can be executed concurrently. This will extend the model of Von Neumann architecture and need new model for algorithm analysis and design. On the other hand, distributed systems focus on interaction among multiple computers with a common goal. This course will provides both theoretical idea and practical details for both subjects.

Course Learning Objectives

1. Apply the concepts of parallel processing
2. Apply the concepts of distributed system

Prerequisites

1. C# and .Net Core Programming Skill (please install .Net Core 2.0 or later with Visual Studio Code or Visual Studio 2017)
2. Github account
3. Some math knowledge (matrix, linear algebra, no calculus)
4. Basic algorithm and data structure

Attendance Sheet

Please see it here

Assignments

1. HW1 is in elearing.cmu.ac.th
2. HW2 https://gist.github.com/pruet/07510645b41e8fc549a0996b1ad75250
3. HW3 instruction, due date 28/2/2562.

Course Topics

Week 1: Introduction and state of the art of parallel processing and distributed systems.

• Slide: Week 1 Introduction
• Intro to Parallel, Distributed and Concurrent Computing
• Parallelism
• State of the art
• Class Progress: (7/1/2562) Slide 1 done.

Week 2: Parallel processing, architectures, paradigms and issues

• Slide: Week 2 Paradim and Issues
• Paradgim
• Issues
• Class progress: (10/1/2562 - 14/1/2562) Slide 2 done

Week 3: Massage passing and shared memory

• Slide: Week 3 Message Passing and Shared Memory
• Shared Memory
• Threading
• Synchronization
• Class progress: (17/1/62) Slide 3 page 1 -19
• Slide: Week 3 supplement Concurrent with Go
• Message Passing
• Go's Channel
• Class progress: (21/1/62 - 24/1/62) Slide 3, Slide 3 supplement done

Week 4: Introduction to threading

• Thread Class
• Thread Synchronization
• Monitor/Lock/Reader,Writer Lock
• Class progress: (27/1/62 - 31/1/62) Done

Week 5: Advance threading

• Thread with multi-core
• Issues with multiple-lock
• Class progress: (27/1/62 - 31/1/62) Done

Week 6: Operating systems of parallel processing

• OS Model
• Thread Model
• Thread in Popular OSes
• Class Progress: (7/2/62) Done

Week 7: Parallel Algorithm

• Class Progress: (18/2/62) Done

Week 8: OpenCL Programming

• OpenCL + TensorFlow
• Class Progress: (7/3/62) Done

Midterm exam

• March 8, 2019, 12:00-15:00

Week 9: Distributed systems, architectures, paradigms and issues

• TBA

Week 10: Tier architecture

• TBA

Week 11: Hadoop

• TBA

Week 12: Docker and microservice, part 1

• TBA

Week 13: Docker and microservice, part 2

• TBA

Week 14: Peer-to-peer archiecture

• TBA

Week 15: Cloud computing

• TBA

Required and Recommended Textbooks

Required Textbooks

None

Recommended Textbooks

1. Topics in Parallel and Distributed Computing: Introducing Concurrency in Undergraduate Courses, Sushil K Prasad (Editor), Anshul Gupta (Editor), Arnold L Rosenberg (Editor), Alan Sussman (Editor), Charles C Weems (Editor), Morgan Kaufmann, 2015
2. Distributed Systems: Principles and Paradigms, Andrew S. Tanenbaum and Maarten van Steen, Prentice Hall, 2002

Recommended Reading

1. Multicore vs. SMP 1, 2
2. Internal Windows Thread.

Grading Rubrics

Professional Conduct

We expect our students to conduct themselves professionally, at least, for the following areas:

• Taking personal responsibility for class prerequisites and coming to class prepared
• Taking personal responsibility for scheduling and managing the homework
• Treating others in a respectful manner
• Plagiarism is not acceptable in any cases
• Accepting things you cannot change, working respectfully within the system to change things you can, and choosing your battles wisely

Attendance including Active & Enthusiastic Participation

• Prepares for class by watching and studying the textbooks and readings.
• Answers in-class questions frequently
• Actively participates and frequently takes a leadership role in break outs

Making Up Missed Classes

• Whether excused or unexcused absence, missed classes must be made up in order to earn a grade of B or higher for the semester
• To make up a class, students study the slide, work any in-class exercises on their own, write up an executive summary, and submit the executive summary to the instructor using CMU Elearning or email ([email protected]) before the next class meeting start time
• Executive Summary should be a PDF file, 1 to 1.5 pages in length, professionally formatted,12 point font.

Assignments

• 8 Assignments, count as 20% toward final score
• 7 of 8 will count
  • 1 lowest will be dropped
  • However, to be dropped, the assignment must be turned in with a reasonable attempt
• Each assignment is graded in whole numbers between 0 and 10 points
  • Subjective = 2 of 10 points
    • Most students will score a 1 on subjective on most assignments
    • A score of 2 on subjective indicates exceptional performance and typically only 10% of students in a section will make this
  • Objective = 8 of 10 points
• Late Submissions
  • Assignments 1 through 7
    • When grading, a zero will be put in as a "placeholder" for missing submissions and changed when they are submitted
    • 1 of 10 points penalty for 1 second late
    • Additional 1 point penalty for each additional 24 hours late
    • After 10 days late, the assignment will be worth zero points
  • Assignment 8
    • Due to end of semester time constraints, late submissions for assignment 8 will have the following late penalties
    • 5 of 10 points for 1 second lateZero for the assignment if 24 hours late

Exams

• Midterm exam
  • Cover the content of week 1 - 7
  • Count 40% toward final score
  • Open book
• Final exam
  • Cover the content of week 8 - 14
  • Count 40% toward final score
  • Open book

Semester Letter Grade Rubrics

Grade	Rubrics
A	Professional conduct Attendance (or make ups) in all 15 classes All 8 assignments submitted with a reasonable attempt The average of the highest 7 of 8 assignments should be 9.0 or higher Obtain at least 35% of midterm and final exam score Top 5 in both midterm and final exam Completed the course
B+	Professional conduct Attendance (or make ups) in all 15 classes All 8 assignments submitted with a reasonable attempt The average of the highest 7 of 8 assignments should be 8.5 or higher Obtain at least 30% of midterm and final exam score Top 10 in both midterm and final exam Completed the course
B	Professional conduct Attendance (or make ups) in all 15 classes All 8 assignments submitted with a reasonable attempt The average of the highest 7 of 8 assignments should be 8.0 or higher Obtain at least 30% of midterm and final exam score Completed the course
C+	Professional conduct All 8 assignments submitted with a reasonable attempt The average of the highest 7 of 8 assignments should be 7.5 or higher Obtain at least 25% of midterm and final exam score Completed the course
C	Professional conduct All 8 assignments submitted with a reasonable attempt The average of the highest 7 of 8 assignments should be 7.0 or higher Obtain at least 20% of midterm and final exam score Completed the course
D+	Professional conduct The average of the highest 7 of 8 assignments should be 6.5 or higher Obtain at least 15% of midterm and final exam score
D	Professional conduct The average of the highest 7 of 8 assignments should be 6.0 or higher Obtain at least 10% of midterm and final exam score
F	Otherwise