CDH6.3.1 - installation steps

Note - all host passwords should be consistent

Prepare installation package

MySQL5.7

mysql-5.7.27-1.el7.x86_64.rpm-bundle.tar

MySQL driver package

mysql-connector-java.jar

Cloudera manager package

cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
allkeys.asc

parcel packet

CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha256
manifest.json

Initialize the installation environment

Install some dependent libraries (each host must run)

yum install -y vim wget ntp net-tools httpd createrepo

Set the host name in each machine separately (each host should run)

Hostnamectl set hostname {hostname of each machine}

Modify the hosts file of each host (each host should run)

IP1 Hostname 1
IP2 Hostname 2
IP3 Hostname 3

Turn off the firewall (each host must be running)

systemctl stop firewalld \
&& systemctl disable firewalld \
&& systemctl status firewalld

Turn off SELinux (each host must be running)

setenforce 0 #Temporarily Closed
vim /etc/selinux/config #Permanent closure
# Change SELINUX=enforcing to SELINUX=permissive
getenforce #The viewing status is permitted

Start NTP service (each host should run)

vim /etc/ntp.conf
# Change the clock server to the following four
server 0.cn.pool.ntp.org
server 1.cn.pool.ntp.org
server 2.cn.pool.ntp.org	
server 3.cn.pool.ntp.org

------------------------------------

Execute command:

systemctl start ntpd \
&& systemctl enable ntpd \
&& ntpdate -u 0.cn.pool.ntp.org \
&& hwclock --systohc \
&& date

Set VM Swappiness (every host should run)

sysctl vm.swappiness=10
echo 'vm.swappiness=10'>> /etc/sysctl.conf

Set transparent large page compression (each host should run)

echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Installing Cloudera Manager

Start httpd (installation host running)

systemctl start httpd

Set up local yum source (install and run the host)

Copy the Cloudera Manager file to the httpd access directory in the installation host, that is, / var/www/html

Files to be copied

cloudera-repos/
├── cm6
│   └── 6.3.1
│       ├── allkeys.asc
│       └── redhat7
│           ├── index.html
│           └── yum
│               ├── cloudera-manager.repo
│               ├── index.html
│               ├── repodata
│               │   ├── 3662f97de72fd44c017bb0e25cee3bc9398108c8efb745def12130a69df2ecb2-filelists.sqlite.bz2
│               │   ├── 43f3725f730ee7522712039982aa4befadae4db968c8d780c8eb15ae9872cd4d-primary.xml.gz
│               │   ├── 49e4d60647407a36819f1d8ed901258a13361749b742e3be9065025ad31feb8e-filelists.xml.gz
│               │   ├── 8afda99b921fd1538dd06355952719652654fc06b6cd14515437bda28376c03d-other.sqlite.bz2
│               │   ├── b9300879675bdbc300436c1131a910a535b8b5a5dc6f38e956d51769b6771a96-primary.sqlite.bz2
│               │   ├── e28836e19e07f71480c4dad0f7a87a804dc93970ec5277ad95614e8ffcff0d58-other.xml.gz
│               │   ├── index.html
│               │   ├── repomd.xml
│               │   ├── repomd.xml.asc
│               │   └── repomd.xml.key
│               ├── RPM-GPG-KEY-cloudera
│               └── RPMS
│                   ├── index.html
│                   └── x86_64
│                       ├── cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm
│                       ├── cloudera-manager-server-db-2-6.3.1-1466458.el7.x86_64.rpm
│                       ├── enterprise-debuginfo-6.3.1-1466458.el7.x86_64.rpm
│                       ├── index.html
│                       └── oracle-j2sdk1.8-1.8.0+update181-1.x86_64.rpm
└── robots.txt

Grant permission

sudo chmod -R ugo+rX /var/www/html/cloudera-repos/cm6

Configure yum source (running on each host)

(
cat <<EOF
[cloudera-manager]
name=Cloudera Manager 6.3.1
baseurl=http://{IP address of the installation host} / cloudera repos / CM6 / 6.3.1/redhat7/yum
gpgcheck=0
enabled=1
EOF
) >> /etc/yum.repos.d/cloudera-manager.repo

Install JDK (each host must run)

yum -y install oracle-j2sdk1.8

Install cloudera Manager (installation host running)

yum install -y cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

Configure the local Parcel repository (installation host running)

Place the local repository in the httpd accessible directory

cdh6/
└── 6.3.1
    └── parcels
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1
        ├── CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha256
        └── manifest.json

Configure permissions

sudo chmod -R ugo+rX /var/www/html/cdh6

Install MySQL

Uninstall the built-in MariaDB (the installation host is running)

yum -y remove mariadb-libs

Installation (host running)

yum install -y libaio \
&& rpm -ivh mysql-community-common-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-libs-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-client-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-server-5.7.27-1.el7.x86_64.rpm \
&& rpm -ivh mysql-community-libs-compat-5.7.27-1.el7.x86_64.rpm

Configure my CNF (installation host running)

echo character-set-server=utf8 >> /etc/my.cnf

Start MySQL (installation host running)

systemctl enable mysqld && systemctl start mysqld && grep password /var/log/mysqld.log | sed 's/.*\(............\)$/\1/'

Create database and authorize it (installation host running)

(
cat <<EOF
set password for root@localhost = password('123456Aa.');
grant all privileges on *.* to 'root'@'%' identified by '123456Aa.';
flush privileges;
CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE monitor DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON monitor.* TO 'monitor'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY '123456Aa.';
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY '123456Aa.';
SHOW DATABASES;
EOF
) >> /root/cdhinit.sql

Run script

source /root/cdhinit.sql

Configure JDBC driver package (install and run the host)

mkdir -p /usr/share/java/
cp mysql-connector-java.jar /usr/share/java/

Initialize scm Library (installation host running)

/opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm 123456Aa.

Start cm (installation host running)

systemctl start cloudera-scm-server 
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

Access interface (installation host running)

http://Installation host IP:7180

Login - the default account and password are admin

Add cluster

Configure machines that need to be added

Configure repository address

Configure packet address

Configure cluster password

After that, it can be installed normally in the next step

Resource requirements:

Link: https://pan.baidu.com/s/1cNj28m8XkQcRED47iex29Q

Extraction code: qn71

Tags: Big Data

Posted by jjmusicpro on Mon, 23 May 2022 10:17:39 +0300

Algorithm Zone

CDH6.3.1 - installation steps

CDH6.3.1 - installation steps

Note - all host passwords should be consistent

Prepare installation package

Initialize the installation environment

Install some dependent libraries (each host must run)

Set the host name in each machine separately (each host should run)

Modify the hosts file of each host (each host should run)

Turn off the firewall (each host must be running)

Turn off SELinux (each host must be running)

Start NTP service (each host should run)

Set VM Swappiness (every host should run)

Set transparent large page compression (each host should run)

Installing Cloudera Manager

Start httpd (installation host running)

Set up local yum source (install and run the host)

Files to be copied

Configure yum source (running on each host)

Install JDK (each host must run)

Install cloudera Manager (installation host running)

Configure the local Parcel repository (installation host running)

Install MySQL

Uninstall the built-in MariaDB (the installation host is running)

Installation (host running)

Configure my CNF (installation host running)

Start MySQL (installation host running)

Create database and authorize it (installation host running)

Configure JDBC driver package (install and run the host)

Initialize scm Library (installation host running)

Start cm (installation host running)

Access interface (installation host running)

Login - the default account and password are admin

Add cluster

Configure machines that need to be added

Configure repository address

Configure packet address

Configure cluster password

After that, it can be installed normally in the next step

Hot Tags