15-319: Introduction to Cloud Computing Majd F. Sakr Computer Science Carnegie Mellon University Spring 2010 1 Organization Instructors: Majd F. Sakr [email protected] M1007, 454-8625 Tue, 3-5pm Teaching Assistant: Suhail Rehman, Office TBA, Phone TBA [email protected] Office Hours: To be decided in class and by appointment. Lecture: Tuesdays and Thursdays 10:30-11:50am Room 2035 Class Web Page: http://www.qatar.cmu.edu/˜msakr/15319-s10/ Class AFS space: /afs/qatar.cmu.edu/course/15/319/ 2 Objectives Our aim in CS 319 is to introduce you to the basics of the emerging cloud computing paradigm. It is important for you to learn how this paradigm came about and understand its enabling technologies. For this to happen we start by an overview of basic systems ideas as well as an introduction to parallel and distributed computing. It is important to cover parallel and distributed systems, their advantages and disadvantages. Understanding these systems is critical to understanding cloud computing systems. We want you to learn how the cloud is organized, provisioned and programmed. Further, we would like you to understand the computer systems constraints, tradeoffs and techniques in setting up and using the cloud to best serve the computing needs for different types of data and applications. Why would a cloud computing system be preferable to existing computing systems? Cloud computing provides the flexibility to offer any platform that a user or company requires. Further, 1it is highly scalable which means the resources can be expanded or shrunk based on current needs. This along with a pay-as-you-go economic model enables companies to have access to the platform of choice with the needed scale that is affordable to them. Further, the human resources required to maintain the cloud are reduced by the user or company utilizing these services. The data-center hosting the cloud will typically consolidate all requests on a large heterogeneous infrastructure to provide reliability, and in turn increase utilization as well as decrease its carbon footprint. This consolidation translates into efficient staffing needs for maintenance and backups. In summary, cloud computing offers the following advantages: 1. Scalability in terms of the resources. A company can start small and increase its hardware resources as it needs. This gives the illusion of having a large number of resources available on demand for the cloud user. 2. Flexibility in terms of the different software packages, multiple instance types, operating systems. As well as ease of system, application and data access from any networked computer. 3. Pay-as-you-go economic model borrowed from utility computing. 4. Consolidation of system maintenance and management, this overhead is shifted from the cloud users to its providers and is consolidated across many requests and systems. 5. Reliability where the system’s fault tolerance is managed by the cloud providers and users no longer need to worry about it. 6. High utilization & reduced carbon footprint since typically a large number of custom servers is consolidated into a smaller number of shared servers. Cloud services can be offered as three different flavors, platform as a service (PaaS) Software as a service (SaaS) or Infrastructure as a service (IaaS). An example of IaaS as cloud services is Amazon’s Web Services (AWS). By the end of the course you will understand the system perspective of the aforementioned advantages and will come to appreciate the benefits that cloud computing provides. You will also be able to design and implement parallel algorithms to efficiently distribute the intensive computation over the cloud machines and let them compute in parallel. You will get introduced to topics on parallel, distributed and large-scale data-intensive computing systems. You will have the foundation needed to match the future needs in this emerging programming paradigm. 3 Learning Outcomes - To understand the emerging cloud computing paradigm, how it came about and how it relates to traditional models of computing. - To understand the different technologies that enable cloud computing. - To gain competence in Hadoop/MapReduce as a programming model for distributed processing of large datasets. - To understand how different algorithms can be implemented and executed in the Hadoop framework. - To gain competence in evaluating the performance and identifying bottlenecks when mapping applications to the cloud. 24 Textbook The primary textbook for this course is: - Tom White, Hadoop: The Definitive Guide, O'Reilly Media, 2009. We have several reference books in the library covering most of the topics of the course. We will be mostly reading journal and conference publications on the subject. In addition, it will be useful to have the following reference books: - Tanenbaum and van Steen, Distributed Systems: Principles and Paradigms, Pearson, 2007. - Jean Dollimore, Tim Kindberg, George Coulouris Distributed Systems: Concepts and Design, Fourth Edition, Addison Wesley, 2005. - Randal E. Bryant and David R. O'Hallaron, Computer Systems: A Programmer's Perspective, Prentice Hall, 2003. - Patterson and Hennessy, Computer Organization and Design: The Hardware/Software Interface, Fourth Edition, Morgan Kaufmann/Elsvier. - Jason Venner, Pro Hadoop, Apress, 2009. 5 Course Organization Your participation in the course will involve five forms of activity: 1. Attending and participating in the lectures 2. Assignments (including reading technical papers) 3. Projects 4. Exam 5. In-class Presentations and Discussions Attendance will be taken at the beginning of each lectures, it will be worth 5% of your grade. You will be considered responsible for all material presented at the lectures. 36 Getting Help For urgent communication with the teaching staff, it is best to send electronic mail (preferred) or to phone. If you want to talk to a staff member in person, remember that our posted office hours are merely nominal times when we guarantee that we will be in our offices. You are always welcome to visit us outside of office hours if you need help or want to talk about the course. However, we ask that you follow a few simple guidelines: - Prof. Sakr normally works with his office door open and welcomes visits from students whenever his doors are open. However, if his door is closed, he