Skip to content

Hadoop 升级启动 NameNode

JDK:1.7.0_72

旧 Hadoop: 2.2.0

新 Hadoop: 2.6.4

操作系统:OS X

启动时使用参数:

bin/hadoop namenode -upgrade

或者:

sbin/hadoop-daedom.sh start namenode -upgrade

(之前一直以为是--upgrade,发现日志错误,namenode 无法启动,在日志里提示要使用-upgrade)启动后,在 NameNode 的 UI 界面(50070)Overview 会出现

Upgrade in progress. Not yet finalized.

的提示警告。根据官方网站的升级提示,可以使用如下命令:

Upgrading Non-Federated Clusters

Suppose there are two namenodes NN1 and NN2, where NN1 and NN2 are respectively in active and standby states. The following are the steps for upgrading a HA cluster:

    Prepare Rolling Upgrade
        Run "hdfs dfsadmin -rollingUpgrade prepare" to create a fsimage for rollback.
        Run "hdfs dfsadmin -rollingUpgrade query" to check the status of the rollback image. Wait and re-run the command until the "Proceed with rolling upgrade" message is shown.
    Upgrade Active and Standby NNs
        Shutdown and upgrade NN2.
        Start NN2 as standby with the "-rollingUpgrade started" option.
        Failover from NN1 to NN2 so that NN2 becomes active and NN1 becomes standby.
        Shutdown and upgrade NN1.
        Start NN1 as standby with the "-rollingUpgrade started" option.
    Upgrade DNs
        Choose a small subset of datanodes (e.g. all datanodes under a particular rack).
            Run "hdfs dfsadmin -shutdownDatanode <DATANODE_HOST:IPC_PORT> upgrade" to shutdown one of the chosen datanodes.
            Run "hdfs dfsadmin -getDatanodeInfo <DATANODE_HOST:IPC_PORT>" to check and wait for the datanode to shutdown.
            Upgrade and restart the datanode.
            Perform the above steps for all the chosen datanodes in the subset in parallel.
        Repeat the above steps until all datanodes in the cluster are upgraded.
    Finalize Rolling Upgrade
        Run "hdfs dfsadmin -rollingUpgrade finalize" to finalize the rolling upgrade.

使用

bin/hdfs dfsadmin -rollingUpgrade prepare
bin/hdfs dfsadmin -rollingUpgrade query
bin/hdfs dfsadmin -rollingUpgrade finalize

执行成功,但是没有升级成功,Upgrade in progress. Not yet finalized. 仍然存在.

后打开 HDFS 的安全模式:

bin/hadoop dfsadmin -safemode enter

然后执行:

bin/hdfs dfsadmin -finalizeUpgrade

成功升级。然后恢复 HDFS 的正常模式

bin/hadoop dfsadmin -safemode leave

dfsadmin -safemode 有如下几个可选的参数:

  1. enter - 进入安全模式
  2. leave - 强制NameNode离开安全模式
  3. get - 返回安全模式是否开启的信息
  4. wait - 等待,一直到安全模式结束。