Hadoop 3.2.1 namenode 格式不起作用

Hadoop 3.2.1 namenode 格式不起作用

我也遇到了同样的问题Hadoop 3.2.1 namenode 格式

根据要求将此作为新问题提出,并提供更多详细信息:

这发生在桌面、Windows 10、64 位、应用了所有 Windows 更新的情况下。

  1. 遵循所有说明https://dev.to/awwsmm/installing-and-running-hadoop-and-spark-on-windows-33kc。例外:a) Hadoop 3.2.1(在这个问题上受阻后,尝试了 Hadoop 3.1.0 和 Hadoop 2.8.0)b) apache-hadoop-3.1.0-winutils-master.zip 用于 winutils.exe 和 hadoop.dll 补丁(无法找到更高版本?)c) 在之前的测试中,使用了 C:\Hadoop\hadoop-3.2.1\data[namenode,datanode] 文件夹,结果与下图相同。更改为 D:\Hadoop\hadoop-3.2.1\data..(如下所示)是指 D:drive 具有更多可用空间。

  2. [从 cmd 提示符,以管理员身份运行],确认 Java 和 hdfs 可用

    C:\WINDOWS\system32>hdfs -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
    
    C:\WINDOWS\system32>java -version java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)
    
  3. 跑步namenode -format C:\WINDOWS\system32>hdfs namenode -format

    2019-10-13 12:01:58,470 INFO namenode.NameNode: STARTUP_MSG:  
    C:\WINDOWS\system32>hdfs namenode -format
    2019-10-13 12:01:58,470 INFO namenode.NameNode: STARTUP_MSG:
    /************************************************************
    STARTUP_MSG: Starting NameNode
    STARTUP_MSG:   host = DESKTOP-354IRPS/10.0.0.23
    STARTUP_MSG:   args = [-format]
    STARTUP_MSG:   version = 3.2.1
    STARTUP_MSG:   classpath = C:\Hadoop\hadoop-3.2.1\etc\hadoop;C:\Hadoop\hadoop-3.2.1\share\hadoop\common;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\accessors-smart-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\animal-sniffer-annotations-1.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\asm-5.0.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\audience-annotations-0.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\avro-1.7.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\checker-qual-2.5.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-beanutils-1.9.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-cli-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-codec-1.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-collections-3.2.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-compress-1.18.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-configuration2-2.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-io-2.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-lang3-3.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-logging-1.1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-math3-3.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-net-3.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\commons-text-1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-client-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-framework-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\curator-recipes-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\dnsjava-2.1.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\error_prone_annotations-2.2.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\failureaccess-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\gson-2.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\guava-27.0-jre.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\hadoop-annotations-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\hadoop-auth-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\htrace-core4-4.1.0-incubating.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\httpclient-4.5.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\httpcore-4.4.10.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\j2objc-annotations-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-core-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-core-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-databind-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-jaxrs-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-mapper-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jackson-xc-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\javax.servlet-api-3.1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jaxb-api-2.2.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jaxb-impl-2.2.3-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jcip-annotations-1.0-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-core-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-json-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-server-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jersey-servlet-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jettison-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-http-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-io-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-security-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-server-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-servlet-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-util-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-webapp-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jetty-xml-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsch-0.1.54.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\json-smart-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsp-api-2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsr305-3.0.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jsr311-api-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\jul-to-slf4j-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-admin-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-client-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-common-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-core-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-crypto-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-identity-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-server-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-simplekdc-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerb-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-asn1-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-config-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-pkix-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\kerby-xdr-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\log4j-1.2.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\metrics-core-3.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\netty-3.10.5.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\nimbus-jose-jwt-4.41.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\paranamer-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\protobuf-java-2.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\re2j-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\slf4j-api-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\slf4j-log4j12-1.7.25.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\snappy-java-1.0.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\stax2-api-3.1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\token-provider-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\woodstox-core-5.0.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\lib\zookeeper-3.4.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-common-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-kms-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\common\hadoop-nfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\accessors-smart-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\animal-sniffer-annotations-1.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\asm-5.0.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\audience-annotations-0.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\avro-1.7.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\checker-qual-2.5.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-beanutils-1.9.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-cli-1.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-codec-1.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-collections-3.2.2.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-compress-1.18.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-configuration2-2.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-daemon-1.0.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-io-2.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-lang3-3.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-logging-1.1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-math3-3.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-net-3.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\commons-text-1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-client-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-framework-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\curator-recipes-2.13.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\dnsjava-2.1.7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\error_prone_annotations-2.2.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\failureaccess-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\gson-2.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\guava-27.0-jre.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\hadoop-annotations-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\hadoop-auth-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\htrace-core4-4.1.0-incubating.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\httpclient-4.5.6.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\httpcore-4.4.10.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\j2objc-annotations-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-core-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-core-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-databind-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-jaxrs-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-mapper-asl-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jackson-xc-1.9.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\javax.servlet-api-3.1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jaxb-api-2.2.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jaxb-impl-2.2.3-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jcip-annotations-1.0-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-core-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-json-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-server-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jersey-servlet-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jettison-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-http-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-io-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-security-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-server-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-servlet-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-util-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-util-ajax-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-webapp-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jetty-xml-9.3.24.v20180605.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsch-0.1.54.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\json-simple-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\json-smart-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsr305-3.0.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\jsr311-api-1.1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-admin-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-client-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-common-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-core-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-crypto-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-identity-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-server-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-simplekdc-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerb-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-asn1-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-config-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-pkix-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-util-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\kerby-xdr-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\leveldbjni-all-1.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\log4j-1.2.17.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\netty-3.10.5.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\netty-all-4.0.52.Final.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\nimbus-jose-jwt-4.41.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\okhttp-2.7.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\okio-1.6.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\paranamer-2.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\protobuf-java-2.5.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\re2j-1.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\snappy-java-1.0.5.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\stax2-api-3.1.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\token-provider-1.0.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\woodstox-core-5.0.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\lib\zookeeper-3.4.13.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-client-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-httpfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-native-client-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-native-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-nfs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-rbf-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\hdfs\hadoop-hdfs-rbf-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\aopalliance-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\bcpkix-jdk15on-1.60.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\bcprov-jdk15on-1.60.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\ehcache-3.3.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\fst-2.50.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\geronimo-jcache_1.0_spec-1.0-alpha-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\guice-4.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\guice-servlet-4.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\HikariCP-java7-2.4.12.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-jaxrs-base-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-jaxrs-json-provider-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jackson-module-jaxb-annotations-2.9.8.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\java-util-1.9.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\javax.inject-1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jersey-client-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\jersey-guice-1.19.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\json-io-2.5.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\metrics-core-3.2.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\mssql-jdbc-6.2.1.jre7.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\objenesis-1.0.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\snakeyaml-1.16.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\lib\swagger-annotations-1.5.4.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-api-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-applications-distributedshell-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-applications-unmanaged-am-launcher-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-client-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-registry-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-applicationhistoryservice-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-nodemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-resourcemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-router-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-sharedcachemanager-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-tests-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-timeline-pluginstorage-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-timelineservice-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-server-web-proxy-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-services-api-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-services-core-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\yarn\hadoop-yarn-submarine-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\lib\hamcrest-core-1.3.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\lib\junit-4.11.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-app-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-common-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-core-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-hs-plugins-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.2.1-tests.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-jobclient-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-nativetask-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-shuffle-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-client-uploader-3.2.1.jar;C:\Hadoop\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.2.1.jar
    STARTUP_MSG:   build = https://gitbox.apache.org/repos/asf/hadoop.git -r b3cbbb467e22ea829b3808f4b7b01d07e0bf3842; compiled by 'rohithsharmaks' on 2019-09-10T15:56Z
    STARTUP_MSG:   java = 1.8.0_221
    ************************************************************/
    2019-10-13 12:01:58,567 INFO namenode.NameNode: createNameNode [-format]
    Formatting using clusterid: CID-e07d7b19-b692-418b-a48d-d1b4cfabb51f
    2019-10-13 12:01:59,508 INFO namenode.FSEditLog: Edit logging is async:true
    2019-10-13 12:01:59,524 INFO namenode.FSNamesystem: KeyProvider: null
    2019-10-13 12:01:59,525 INFO namenode.FSNamesystem: fsLock is fair: true
    2019-10-13 12:01:59,527 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false
    2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: fsOwner             = stowl (auth:SIMPLE)
    2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: supergroup          = supergroup
    2019-10-13 12:01:59,535 INFO namenode.FSNamesystem: isPermissionEnabled = true
    2019-10-13 12:01:59,540 INFO namenode.FSNamesystem: HA Enabled: false
    2019-10-13 12:01:59,595 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling
    2019-10-13 12:01:59,611 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000
    2019-10-13 12:01:59,611 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
    2019-10-13 12:01:59,624 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
    2019-10-13 12:01:59,624 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Oct 13 12:01:59
    2019-10-13 12:01:59,628 INFO util.GSet: Computing capacity for map BlocksMap
    2019-10-13 12:01:59,632 INFO util.GSet: VM type       = 64-bit
    2019-10-13 12:01:59,634 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB
    2019-10-13 12:01:59,637 INFO util.GSet: capacity      = 2^21 = 2097152 entries
    2019-10-13 12:01:59,650 INFO blockmanagement.BlockManager: Storage policy satisfier is disabled
    2019-10-13 12:01:59,650 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false
    2019-10-13 12:01:59,658 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS
    2019-10-13 12:01:59,659 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
    2019-10-13 12:01:59,663 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0
    2019-10-13 12:01:59,667 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000
    2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: defaultReplication         = 1
    2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: maxReplication             = 512
    2019-10-13 12:01:59,671 INFO blockmanagement.BlockManager: minReplication             = 1
    2019-10-13 12:01:59,672 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
    2019-10-13 12:01:59,672 INFO blockmanagement.BlockManager: redundancyRecheckInterval  = 3000ms
    2019-10-13 12:01:59,673 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
    2019-10-13 12:01:59,673 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
    2019-10-13 12:01:59,725 INFO namenode.FSDirectory: GLOBAL serial map: bits=29 maxEntries=536870911
    2019-10-13 12:01:59,726 INFO namenode.FSDirectory: USER serial map: bits=24 maxEntries=16777215
    2019-10-13 12:01:59,727 INFO namenode.FSDirectory: GROUP serial map: bits=24 maxEntries=16777215
    2019-10-13 12:01:59,732 INFO namenode.FSDirectory: XATTR serial map: bits=24 maxEntries=16777215
    2019-10-13 12:01:59,755 INFO util.GSet: Computing capacity for map INodeMap
    2019-10-13 12:01:59,755 INFO util.GSet: VM type       = 64-bit
    2019-10-13 12:01:59,757 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB
    2019-10-13 12:01:59,763 INFO util.GSet: capacity      = 2^20 = 1048576 entries
    2019-10-13 12:01:59,765 INFO namenode.FSDirectory: ACLs enabled? false
    2019-10-13 12:01:59,766 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true
    2019-10-13 12:01:59,766 INFO namenode.FSDirectory: XAttrs enabled? true
    2019-10-13 12:01:59,771 INFO namenode.NameNode: Caching file names occurring more than 10 times
    2019-10-13 12:01:59,781 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536
    2019-10-13 12:01:59,783 INFO snapshot.SnapshotManager: SkipList is disabled
    2019-10-13 12:01:59,790 INFO util.GSet: Computing capacity for map cachedBlocks
    2019-10-13 12:01:59,790 INFO util.GSet: VM type       = 64-bit
    2019-10-13 12:01:59,791 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB
    2019-10-13 12:01:59,797 INFO util.GSet: capacity      = 2^18 = 262144 entries
    2019-10-13 12:01:59,814 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
    2019-10-13 12:01:59,815 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
    2019-10-13 12:01:59,817 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
    2019-10-13 12:01:59,829 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
    2019-10-13 12:01:59,829 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
    2019-10-13 12:01:59,834 INFO util.GSet: Computing capacity for map NameNodeRetryCache
    2019-10-13 12:01:59,835 INFO util.GSet: VM type       = 64-bit
    2019-10-13 12:01:59,835 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
    2019-10-13 12:01:59,836 INFO util.GSet: capacity      = 2^15 = 32768 entries
    Re-format filesystem in Storage Directory root= D:\Hadoop\data\namenode; location= null ? (Y or N) y
    2019-10-13 12:02:02,200 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1688834052-10.0.0.23-1570986122192
    2019-10-13 12:02:02,200 INFO common.Storage: Will remove files: []
    2019-10-13 12:02:02,284 ERROR namenode.NameNode: Failed to start namenode.
    java.lang.UnsupportedOperationException
            at java.nio.file.Files.setPosixFilePermissions(Unknown Source)
            at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.clearDirectory(Storage.java:452)
            at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:591)
            at org.apache.hadoop.hdfs.server.namenode.NNStorage.format(NNStorage.java:613)
            at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:188)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1206)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1649)
            at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1759)
    2019-10-13 12:02:02,297 INFO util.ExitUtil: Exiting with status 1: java.lang.UnsupportedOperationException
    2019-10-13 12:02:02,306 INFO namenode.NameNode: SHUTDOWN_MSG:
    /************************************************************
    SHUTDOWN_MSG: Shutting down NameNode at DESKTOP-354IRPS/10.0.0.23
    ************************************************************/
    

答案1

这是一个错误,我已经努力了几天,然后发现了这个:https://issues.apache.org/jira/browse/HDFS-14890

在非 posix 兼容文件系统上设置名称目录的权限会失败,并影响 Windows 上的 Hadoop 3.2.1,该问题已在版本 3.3.0 和 3.2.2 中得到解决 - 这两个版本都尚未发布,因此您需要安装不同版本的 Hadoop

答案2

这是一个错误。如果您确实想立即在 Windows 上安装 3.2.1,请参考我的以下帖子:

基本上,我签出 3.2.1 分支并手动修复文件权限代码,然后在我的 Windows 10 机器上本地重建 Hadoop。

在 Windows 10 上安装 Hadoop 3.2.1

在 Windows 10 上编译并构建 Hadoop 3.2.1

相关内容