問題描述

問題來源是這樣，我使用sqoop向Hive中導(dǎo)入mysql的表

sqoop import --connect jdbc:mysql://127.0.0.1:3306/employees_db --table titles --username root -P --hive-import -- --default-character-set=utf-8

然后發(fā)生了報(bào)錯(cuò)：

16/08/10 22:08:36 ERROR tool.ImportTool: Encountered IOException running import job: java.io.FileNotFoundException: File does not exist: hdfs://192.168.1.128:9000/usr/local/sqoop/lib/mysql-connector-java-5.1.39-bin.jar

于是我Google了一下，發(fā)現(xiàn)清一色的回答均為在hdfs中創(chuàng)建一個(gè)/usr/local/sqoop,然后將本地的sqoopput上去，即

[root@Master local]# hadoop dfs -put /usr/local/sqoop /usr/localDEPRECATED: Use of this script to execute hdfs command is deprecated.Instead use the hdfs command for it.16/08/10 22:23:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable[root@Master local]# hadoop dfs -ls /usr/localDEPRECATED: Use of this script to execute hdfs command is deprecated.Instead use the hdfs command for it.16/08/10 22:25:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicableFound 1 itemsdrwxr-xr-x - root supergroup 0 2016-08-10 22:25 /usr/local/sqoop

沒錯(cuò)，這樣是成功解決了問題但是為什么會(huì)把sqoop存到hdfs中？我看到了有些人并不需要把sqoop移到hdfs中也能成功運(yùn)行，是我哪里的配置出錯(cuò)還是只能這么做？hdfs是分布式儲(chǔ)存應(yīng)該是存數(shù)據(jù)文件，怎么會(huì)把一個(gè)程序存進(jìn)去呢？這里令我很不解，希望能得到解答。

問題解答

關(guān)于這個(gè)問題,貌似找到了一個(gè)解釋，這里報(bào)錯(cuò)是缺少hdfs://192.168.1.128:9000/usr/local/sqoop/lib/mysql-connector-java-5.1.39-bin.jar,所以我的理解就是并不是sqoop在HDFS上運(yùn)行，而是sqoop的jar要放在HDFS上，所以我們只用把jar上傳到HDFS即可