我正在尝试在具有 1 个名称节点和 3 个数据节点的 Hadoop(3.2.0)多节点集群上安装 Apache Hive(3.1.1)。
我已经关注入门教程在 apache 网站上一步一步地操作,但是当运行“hive”命令时,我在 hive.log 文件中得到了一个非常长的堆栈跟踪,同时在实际控制台中还出现了一些其他问题。
控制台错误与多个 SLF4J 绑定有关:
> SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found
> binding in
> [jar:file:/home/hadoop/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/home/hadoop/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation. SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory] Hive Session ID =
> ebd3ca4d-0910-47d7-96a1-e7d1d369f919
>
> Logging initialized using configuration in
> jar:file:/home/hadoop/hive/lib/hive-common-3.1.1.jar!/hive-log4j2.properties
> Async: true
Hive.log 错误似乎与设置 metastore 数据库有关。但是,我是这个领域的新手,并不完全熟悉所有使用的术语,因此我发现这个堆栈跟踪非常难以理解,并且找不到修复它的方法。下面是堆栈跟踪,但格式经过缩减,以适合这个问题的主体。
2019-07-05T10:24:28,966 INFO [main] conf.HiveConf: Found configuration file file:/home/hadoop/hive/conf/hive-site.xml
2019-07-05T10:24:29,990 INFO [main] SessionState: Hive Session ID = ebd3ca4d-0910-47d7-96a1-e7d1d369f919
2019-07-05T10:24:30,133 INFO [main] SessionState:
Logging initialized using configuration in jar:file:/home/hadoop/hive/lib/hive-common-3.1.1.jar!/hive-log4j2.properties Async: true
2019-07-05T10:24:31,583 INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/ebd3ca4d-0910-47d7-96a1-e7d1d369f919
2019-07-05T10:24:31,618 INFO [main] session.SessionState: Created local directory: /tmp/hadoop/ebd3ca4d-0910-47d7-96a1-e7d1d369f919
2019-07-05T10:24:31,628 INFO [main] session.SessionState: Created HDFS directory: /tmp/hive/hadoop/ebd3ca4d-0910-47d7-96a1-e7d1d369f919/_tmp_space.db
2019-07-05T10:24:31,644 INFO [main] conf.HiveConf: Using the default value passed in for log id: ebd3ca4d-0910-47d7-96a1-e7d1d369f919
2019-07-05T10:24:31,644 INFO [main] session.SessionState: Updating thread name to ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main
2019-07-05T10:24:32,691 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore
2019-07-05T10:24:32,717 WARN [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored
2019-07-05T10:24:32,723 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.ObjectStore: ObjectStore, initialize called
2019-07-05T10:24:32,724 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] conf.MetastoreConf: Found configuration file file:/home/hadoop/hive/conf/hive-site.xml
2019-07-05T10:24:32,725 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] conf.MetastoreConf: Unable to find config file hivemetastore-site.xml
2019-07-05T10:24:32,725 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] conf.MetastoreConf: Found configuration file null
2019-07-05T10:24:32,726 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] conf.MetastoreConf: Unable to find config file metastore-site.xml
2019-07-05T10:24:32,726 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] conf.MetastoreConf: Found configuration file null
2019-07-05T10:24:33,634 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] hikari.HikariDataSource: HikariPool-1 - Starting...
2019-07-05T10:24:33,641 WARN [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] util.DriverDataSource: Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation.
2019-07-05T10:24:33,926 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] pool.PoolBase: HikariPool-1 - Driver does not support get/set network timeout for connections. (Feature not implemented: No details.)
2019-07-05T10:24:33,938 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] hikari.HikariDataSource: HikariPool-1 - Start completed.
2019-07-05T10:24:34,001 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] hikari.HikariDataSource: HikariPool-2 - Starting...
2019-07-05T10:24:34,001 WARN [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] util.DriverDataSource: Registered driver with driverClassName=org.apache.derby.jdbc.EmbeddedDriver was not found, trying direct instantiation.
2019-07-05T10:24:34,007 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] pool.PoolBase: HikariPool-2 - Driver does not support get/set network timeout for connections. (Feature not implemented: No details.)
2019-07-05T10:24:34,008 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] hikari.HikariDataSource: HikariPool-2 - Start completed.
2019-07-05T10:24:34,697 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2019-07-05T10:24:34,811 WARN [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.MetaStoreDirectSql: Self-test query [select "DB_ID" from "DBS"] failed; direct SQL is disabled
javax.jdo.JDODataStoreException: Error executing SQL query "select "DB_ID" from "DBS"".
...
Caused by: java.sql.SQLSyntaxErrorException: Table/View 'DBS' does not exist.
...
Caused by: org.apache.derby.iapi.error.StandardException: Table/View 'DBS' does not exist.
...
2019-07-05T10:24:34,817 INFO [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.ObjectStore: Initialized ObjectStore
2019-07-05T10:24:37,667 ERROR [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.RetryingHMSHandler: MetaException(message:Version information not found in metastore.)
...
2019-07-05T10:24:37,667 ERROR [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metastore.RetryingHMSHandler: HMSHandler Fatal error: MetaException(message:Version information not found in metastore.)
...
2019-07-05T10:24:37,667 WARN [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metadata.Hive: Failed to register all functions.
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: java.lang.reflect.InvocationTargetException
...
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore.
...
2019-07-05T10:24:37,711 ERROR [ebd3ca4d-0910-47d7-96a1-e7d1d369f919 main] metadata.HiveMaterializedViewsRegistry: Problem connecting to the metastore when initializing the view registry
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: java.lang.reflect.InvocationTargetException
...
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore.
我之前没有在这个设备上使用过 Derby 数据库、Java 数据库连接 (JDBC) 或 MySQL - 这可能是依赖关系问题吗?虽然 Hive 教程中没有列出任何内容。
提前致谢。
答案1
我之前遇到过同样的问题,这是我的解决方案。
在我的系统中$HIVE_HOME/scripts/metastore/upgrade/derby/hive-schema-3.1.0.derby.sql
,正在使用的数据库是“APP”(因为“ CREATE TABLE "APP"."DBS"
”)
但在我看来$HIVE_HOME/conf/hive-site.xml
,它是这样的:
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=metastore_db;create=true</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>
因此我用“APP”替换了“metastore_db”,并重新初始化了 derby 元存储。
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=APP;create=true</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>