操作系统:
shell
$uname -a
Linux server2 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$cat /etc/redhat-release
CentOS Linux release 7.2.1511 (Core)
关闭防火墙:
shell
systemctl stop firewalld.service
安装 MySQL 不做阐述。我安装的 mysql 5.7,需支持远程登录。
使用如下语句创建数据库:
shell
create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database oozie DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
create database hue DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
flush privileges;
添加 cloudera cm 用户:
shell
useradd -r -d /opt/cloudera-manager/cm-5.15.2/run/cloudera-scm-server -M -c "Cloudera SCM User" cloudera-scm
id cloudera-scm
创建 cdh 软件仓库目录:
shell
mkdir -p /opt/cloudera/parcel-repo
chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo
mkdir -p /opt/cloudera/parcels
chown cloudera-scm:cloudera-scm /opt/cloudera/parcels
初始化 cloudera-manager:
shell
/opt/cloudera-manager/cm-5.15.2/share/cmf/schema/scm_prepare_database.sh mysql cm -h 192.168.30.217 root 123456 --verbose
配置 cloudera-scm-server
shell
cp /opt/cloudera-manager/cm-5.15.2/etc/init.d/cloudera-scm-server /etc/init.d/cloudera-scm-server
vim /etc/init.d/cloudera-scm-server
#找到CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}
#修改为
CMF_DEFAULTS="/opt/cloudera-manager/cm-5.15.2/etc/default"
启动 cloudera-scm-server
shell
/etc/init.d/cloudera-scm-server start
初始化 cloudera-agent:
修改 agent config 中的 server_host 参数,配置为 cloudera-scm-server IP 或者 hostname:
shell
vim /opt/cloudera-manager/cm-5.15.2/etc/cloudera-scm-agent/config.ini
server_host=server2
配置启动脚本:
shell
mkdir /opt/cloudera-manager/cm-5.15.2/run/cloudera-scm-agent
cp /opt/cloudera-manager/cm-5.15.2/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
chmod +x /etc/init.d/cloudera-scm-agent
vim /etc/init.d/cloudera-scm-agent
找到CMF_DEFAULTS=${CMF_DEFAULTS:-/etc/default}
修改成CMF_DEFAULTS="/opt/cloudera-manager/cm-5.15.2/etc/default"
启动 cloudera-manager
shell
/etc/init.d/cloudera-scm-agent start
配置 CDH:
启动和重启(需先启动MySQL):
shell
systemctl restart cloudera-scm-server
systemctl restart cloudera-scm-agent
注意(重装需要):sudo rm /var/lib/cloudera-scm-agent/cm_guid
安装后地址:http://192.168.30.140:7180/ admin/admin
文件系统权限问题:
shell
sudo -u yarn hdfs dfs -mkdir /user/yarn
mkdir: Permission denied: user=yarn, access=WRITE, inode="/user":hdfs:supergroup:drwxrwxr-x
[root@node1 data]# sudo -u hdfs hdfs dfs -mkdir /user/yarn
[root@node1 data]# sudo -u hdfs hdfs dfs -chown yarn:supergroup /user/yarn
安装配置信息:
shell
# 日志地址
/var/log/${service}
# 各种服务配置数据
/var/lib/${service}
# 实际安装地址(CDH控制)
/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/lib
配置文件地址:
/etc/${service}