Copyright 2014, Simplilearn, All rights reserved. Copyright 2014, Simplilearn, All rights reserved. Lesson 4—Hadoop Deployment Big Data and Hadoop DeveloperCopyright 2014, Simplilearn, All rights reserved. Objectives ● Explain how to install Ubuntu Server 12.04 LTS for Hadoop ● Describe how to install and configure Hadoop ● Identify the steps to install and configure Hadoop in a cluster environment Objectives By the end of this lesson, you will be able to: 2Copyright 2014, Simplilearn, All rights reserved. Ubuntu Server – Introduction Ubuntu is a leading open-source platform for scale-out. Ubuntu Server—Introduction Ubuntu helps in utilizing the infrastructure at its optimum level irrespective of whether users want to deploy a cloud, a web farm, or a Hadoop cluster. Following are the benefits of Ubuntu server: ● It has the required versatility and performance to help users get the most out of the infrastructure. ● Ubuntu services ensure an efficient system administration with Landscape. ● These services provide access to Ubuntu experts as and when required, and enable fast resolution of a problem. 3Copyright 2014, Simplilearn, All rights reserved. You will be using Ubuntu Server 12.04 version as a dedicated server for installing Hadoop. The prerequisites for installing Ubuntu Server 12.04 are: ● VMware Workstation; and ● Ubuntu Server 12.04 LTS. The VMware Workstation or VMware Player will be required to make a Virtual Machine (VM) in which you will install the Ubuntu Server 12.04 LTS operating system. You can download the Ubuntu Server 12.04 LTS ISO file from www.ubuntu.com. Installation of Ubuntu Server 12.04 4Copyright 2014, Simplilearn, All rights reserved. Business Scenario The demos in this lesson illustrate how to install Ubuntu Server 12.04, how to install Hadoop in Ubuntu Server 12.04, create a clone of Hadoop virtual machine, and how to perform the clustering of Hadoop environment. Nutri Worldwide Inc. has adopted Hadoop for its operations and Olivia, the EVP at the company has been assigned the task of setting up the infrastructure for Hadoop. Olivia’s team has listed the following tasks to be performed for this purpose: ● Installation of Ubuntu Server 12.04 ● Installation of Hadoop in Ubuntu Server 12.04 ● Performing clustering of the Hadoop environment Business Scenario 5Copyright 2014, Simplilearn, All rights reserved. Demo 1 Demo 1 Install Ubuntu Server 12.04 6Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation – Prerequisites Following are the prerequisites for installing Hadoop: Hadoop Installation—Prerequisites Ubuntu Server 12.04 Operating System High-speed internet connection, for example, 512 kbps and above 7Copyright 2014, Simplilearn, All rights reserved. You need to perform the following steps to install Hadoop: Hadoop Installation Create a tmp directory and provide access rights. 10 11 12 9 5 1 2 6 7 8 4 3 Install the ssh server. Create a password-less login in ssh. Install Java 6 or a newer version of Java. Download and install Hadoop. Extract the Hadoop tar file. Copy the tar file to /usr/local. Open and set .bashrc, and add the required paths. Open and set the hadoop-env.sh script. Open and set the core-site.xml file. Open and set the mapred-site.xml file. Start Hadoop services. 8Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Install the ssh server using the command given below. Hadoop Installation—Step 1 $ sudo apt-get install openssh-server 9Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Create a password-less login in ssh server using the commands below. Hadoop Installation—Step 2 10Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Install Java 6 or a newer version of Java using the command given below. Hadoop Installation—Step 3 11Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Download and install Hadoop. Select closest mirror and download the current or a stable release from the link: http://www.apache.org/dyn/closer.cgi/hadoop/common/. Hadoop Installation—Step 4 $ wget http://mirror.cc.columbia.edu/pub/software/apache/hadoop/common/current/hadoop-2.4.1.tar.gz $ wget http://mirror.cc.columbia.edu/pub/software/apache/hadoop/common/current/hadoop-2.4.1.tar.gz 12Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Extract the Hadoop tar file using the command displayed by the image. Hadoop Installation—Step 5 13Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Copy the tar file to/usr/local as shown below. Hadoop Installation—Step 6 14Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) At the end of the line, type the lines that are highlighted below. Set .bashrc and add the required paths. Hadoop Installation—Step 7 15Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Reset and commit changes in bash. Hadoop Installation—Step 7 (contd.) 16Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Open hadoop-env.sh from the /usr/local/hadoop/conf directory. Hadoop Installation—Step 8 17Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Uncomment the JAVA_HOME line and set the path for Java in hadoop-env.sh as shown below. Hadoop Installation—Step 8 (contd.) 18Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Uncomment the export HADOOP_OPTS line and enable IPv4 in hadoop-env.sh using the command shown below. Hadoop Installation—Step 8 (contd.) 19Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Open the core-site.xml file in vi editor using the command displayed below. Hadoop Installation—Step 9 20Copyright 2014, Simplilearn, All rights reserved. Set the core-site.xml file. Following are the two parameters used: ● fs.default.name: Used to set the access path for Hadoop Distributed File System (HDFS) ● hadoop.tmp.dir: Used to set the default tmp directory where HDFS is installed Hadoop Installation—Step 9 (contd.) 21Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Open the mapred-site.xml file using the command displayed here. Hadoop Installation—Step 10 22Copyright 2014, Simplilearn, All rights reserved. Hadoop Installation (contd.) Set the
View Full Document
Unlocking...