![Sqoop 从 MySQL 导入到 Hive 引发 HIVE_CONF_DIR 错误](https://linux22.com/image/901889/Sqoop%20%E4%BB%8E%20MySQL%20%E5%AF%BC%E5%85%A5%E5%88%B0%20Hive%20%E5%BC%95%E5%8F%91%20HIVE_CONF_DIR%20%E9%94%99%E8%AF%AF.png)
我正在尝试使用 Sqoop 将我的 MySQL 表 Lab5 加载到 Hive。Sqoop 成功运行,并将数据从 mysql 导入到 hdfs。但抛出了以下HIVE_CONF_DIR
未设置的粗体错误。我已将其HIVE_CONF_DIR
设置~/.bashrc
为~/.profile
:
export HIVE_HOME=/usr/local/hive
export HIVE_CONF_DIR=$HIVE_HOME/conf
尽管这样做了,我仍然遇到以下HIVE_CONF_DIR
错误。您能尽快帮我解决这个错误吗?
18/07/16 13:27:38 **INFO mapreduce.Job: Job job_1531743854195_0001 completed successfully**
18/07/16 13:27:38 INFO mapreduce.Job: Counters: 30
File System Counters
FILE: Number of bytes read=0
FILE: Number of bytes written=566628
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=408
HDFS: Number of bytes written=179248
HDFS: Number of read operations=16
HDFS: Number of large read operations=0
HDFS: Number of write operations=8
Job Counters
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=12437
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=12437
Total vcore-milliseconds taken by all map tasks=12437
Total megabyte-milliseconds taken by all map tasks=12735488
Map-Reduce Framework
Map input records=1000
Map output records=1000
Input split bytes=408
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=338
CPU time spent (ms)=3300
Physical memory (bytes) snapshot=713752576
Virtual memory (bytes) snapshot=7925747712
Total committed heap usage (bytes)=458752000
File Input Format Counters
Bytes Read=0
File Output Format Counters
Bytes Written=179248
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Transferred 175.0469 KB in 18.8594 seconds (9.2817 KB/sec)
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Retrieved 1000 records.
18/07/16 13:27:38 INFO mapreduce.ImportJobBase: Publishing Hive/Hcat import job data to Listeners for table tableLab5
Mon Jul 16 13:27:38 IST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
18/07/16 13:27:38 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tableLab5` AS t LIMIT 1
18/07/16 13:27:38 INFO hive.HiveImport: Loading uploaded data into Hive
18/07/16 13:27:38 **ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.**
18/07/16 13:27:38 ERROR tool.ImportTool: Import failed: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:50)
at org.apache.sqoop.hive.HiveImport.getHiveArgs(HiveImport.java:392)
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:379)
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:337)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:241)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:537)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.conf.HiveConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.sqoop.hive.HiveConfig.getHiveConf(HiveConfig.java:44)
... 12 more
答案1
您可以尝试像这样导出 hadoop_classpath
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/lib/hive/lib/*
然后
source ~/.bashrc