Apache Sqoop Installation

The objective of this tutorial is to describe step by step process to install Sqoop (Version 1.4.7 ) on Hadoop 3.1.2 version and the OS which we are using is Ubuntu 18.04.4 LTS (Bionic Beaver), once the installation is completed you can play with Sqoop.

Platform

  • Operating System (OS). You can use Ubuntu 18.04.4 LTS version or later version, also you can use other flavors of Linux systems like Redhat, CentOS, etc.
  • Hadoop. We have already installed Hadoop 3.1.2 version on which we will run Sqoop (Please refer to the "Hadoop Installation on Single Node” tutorial and install Hadoop first before proceeding for Sqoop installation.)
  • Sqoop. We have used Apache Sqoop-1.4.7 version for installation.

Download Software

  • Sqoop.

    • https://downloads.apache.org/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz


Steps to Install Apache Sqoop version(1.4.7) on Ubuntu 18.04.4 LTS

Please follow the below steps to install Apache Sqoop.

    Step 1. Please ensure that Apache Hadoop is installed on your system.


    Step 2. Please ensure that Java is installed on your system.


    Step 3. Please download Sqoop 1.4.7 from the below link.

    On Linux: $wget https://downloads.apache.org/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

    On Windows: https://downloads.apache.org/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

    Sqoop installation


    Step 4. Now we will extract the tar file by using the below command and rename the folder to sqoop to make it meaningful.

    $tar -xzf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

    $mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop

    sqoop_extract_tar

    sqoop_rename_folder


    Step 5. Now edit the .bashrc file to update the environment variable of Apache Sqoop so that it can be accessed from any directory.

    $nano .bashrc

    Add below lines.

    $export SQOOP_HOME=/home/cloudduggu/sqoop

    $export PATH=$PATH:$SQOOP_HOME/bin

    bash_file_update

    Save the changes by pressing CTRL + O and exit from the nano editor by pressing CTRL + X.


    Step 6. Run source command to update changes in the same terminal.

    $source .bashrc


    Step 7. Now we will configure the Apache Sqoop environment file which is located at (/home/cloudduggu/sqoop/conf) path. Please check your Sqoop home directory.

    We will create an Apache Sqoop environment from prebuild template (sqoop-env-template.sh).

    Please follow the below command.

    $mv sqoop-env-template.sh sqoop-env.sh

    sqoop configuration

    Open the sqoop-env.sh file and add below lines.

    export HADOOP_COMMON_HOME=/home/cloudduggu/hadoop

    export HADOOP_MAPRED_HOME=/home/cloudduggu/hadoop

    sqoop env setup


    Step 8. Download Mysql connector, untar it, and then put in /$SQOOP_HOME/lib path. Use the below command.

    $wget http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-J/mysql-connector-java-8.0.24.tar.gz

    $tar -xvf mysql-connector-java-8.0.24.tar.gz

    $cd mysql-connector-java-8.0.24

    $mv mysql-connector-java-8.0.24.jar /home/cloudduggu/sqoop/lib/


    Step 9. Now verify the Apache Sqoop version to make sure installation is done properly.

    ssqoop version

    Apache Sqoop installation is completed on Ubuntu 18.04.4 LTS (Bionic Beaver).