Big Data and Hadoop Developer Lesson 4 Hadoop Deployment Copyright 2014 Simplilearn All rights reserved Copyright 2014 Simplilearn All rights reserved Objectives Objectives By the end of this lesson you will be able to Explain how to install Ubuntu Server 12 04 LTS for Hadoop Describe how to install and configure Hadoop Identify the steps to install and configure Hadoop in a cluster environment 2 Copyright 2014 Simplilearn All rights reserved Ubuntu Server Introduction Ubuntu Server Introduction Ubuntu is a leading open source platform for scale out Ubuntu helps in utilizing the infrastructure at its optimum level irrespective of whether users want to deploy a cloud a web farm or a Hadoop cluster Following are the benefits of Ubuntu server It has the required versatility and performance to help users get the most out of the infrastructure Ubuntu services ensure an efficient system administration with Landscape These services provide access to Ubuntu experts as and when required and enable fast resolution of a problem 3 Copyright 2014 Simplilearn All rights reserved Installation of Ubuntu Server 12 04 You will be using Ubuntu Server 12 04 version as a dedicated server for installing Hadoop The prerequisites for installing Ubuntu Server 12 04 are VMware Workstation and Ubuntu Server 12 04 LTS The VMware Workstation or VMware Player will be required to make a Virtual Machine VM in which you will install the Ubuntu Server 12 04 LTS operating system You can download the Ubuntu Server 12 04 LTS ISO file from www ubuntu com 4 Copyright 2014 Simplilearn All rights reserved Business Scenario Business Scenario Nutri Worldwide Inc has adopted Hadoop for its operations and Olivia the EVP at the company has been assigned the task of setting up the infrastructure for Hadoop Olivia s team has listed the following tasks to be performed for this purpose Installation of Ubuntu Server 12 04 Installation of Hadoop in Ubuntu Server 12 04 Performing clustering of the Hadoop environment The demos in this lesson illustrate how to install Ubuntu Server 12 04 how to install Hadoop in Ubuntu Server 12 04 create a clone of Hadoop virtual machine and how to perform the clustering of Hadoop environment 5 Copyright 2014 Simplilearn All rights reserved Demo Demo 11 Install Ubuntu Server 12 04 6 Copyright 2014 Simplilearn All rights reserved Hadoop Installation Prerequisites Hadoop Installation Prerequisites Following are the prerequisites for installing Hadoop Ubuntu Server 12 04 Operating System High speed internet connection for example 512 kbps and above 7 Copyright 2014 Simplilearn All rights reserved Hadoop Installation You need to perform the following steps to install Hadoop 2 1 Create a password less login in ssh Install the ssh server 5 Copy the tar file to usr local Extract the Hadoop tar file 10 Open and set the core site xml file 8 7 6 9 3 11 Open and set the mapred site xml file 4 Install Java 6 or a newer version of Java Open and set bashrc and add the required paths Create a tmp directory and provide access rights Download and install Hadoop 8 Open and set the hadoop env sh script 12 Start Hadoop services Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 1 Install the ssh server using the command given below sudo apt get install openssh server 9 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 2 Create a password less login in ssh server using the commands below 10 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 3 Install Java 6 or a newer version of Java using the command given below 11 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 4 Download and install Hadoop Select closest mirror and download the current or a stable release from the link http www apache org dyn closer cgi hadoop common wget http mirror cc columbia edu pub software apache hadoop common current hadoop 2 4 1 tar gz wget http mirror cc columbia edu pub software apache hadoop common current hadoop2 4 1 tar gz 12 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 5 Extract the Hadoop tar file using the command displayed by the image 13 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 6 Copy the tar file to usr local as shown below 14 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 7 Set bashrc and add the required paths At the end of the line type the lines that are highlighted below 15 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 7 contd Reset and commit changes in bash 16 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 8 Open hadoop env sh from the usr local hadoop conf directory 17 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 8 contd Uncomment the JAVA HOME line and set the path for Java in hadoop env sh as shown below 18 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 8 contd Uncomment the export HADOOP OPTS line and enable IPv4 in hadoop env sh using the command shown below 19 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 9 Open the core site xml file in vi editor using the command displayed below 20 Copyright 2014 Simplilearn All rights reserved Hadoop Installation Step 9 contd Set the core site xml file Following are the two parameters used fs default name Used to set the access path for Hadoop Distributed File System HDFS hadoop tmp dir Used to set the default tmp directory where HDFS is installed 21 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 10 Open the mapred site xml file using the command displayed here 22 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 10 contd Set the mapred site xml file using the command mentioned below configuration property name mapred job tracker name value localhost 9001 value property configuration 23 Copyright 2014 Simplilearn All rights reserved Hadoop Installation contd Hadoop Installation Step 11 Create a tmp directory in usr local hadoop and set permissions on the same 24 Copyright 2014
View Full Document
Unlocking...