歡迎來到Linux教程網
Linux教程網
Linux教程網
Linux教程網
Linux教程網 >> Linux基礎 >> 關於Linux >> ubuntu14.04下Hadoop2.6.0+Hive1.1.1安裝

ubuntu14.04下Hadoop2.6.0+Hive1.1.1安裝

日期:2017/3/1 11:44:09   编辑:關於Linux
1.Hive是基於Hadoop構建的一套數據倉庫分析系統,它提供了豐富的SQL查詢方式來分析存儲在Hadoop 分布式文件系統中的數據。其在Hadoop的架構體系中承擔了一個SQL解析的過程,它提供了對外的入口來獲取用戶的指令然後對指令進行分析,解析出一個MapReduce程序組成可執行計劃,並按照該計劃生成對應的MapReduce任務提交給Hadoop集群處理,獲取最終的結果。元數據——如表模式——存儲在名為metastore的數據庫中。
2.下載http://mirror.bjtu.edu.cn/apache/hive/
解壓到/usr/local/hadoop/hive
3.拷貝mysql-connector-java-5.1.6-bin.jar 到hive 的lib下面
4.將hive下的新版本jline的JAR包拷貝到hadoop下:
cp /usr/local/hadoop/hive /lib/jline-2.12.jar /usr/local/hadoop /share/hadoop/yarn/lib 並且移除$HADOOP_HOME/share/hadoop/yarn/lib/下的jline-0.9.94.jar文件
5.gedit ~/.bashrc
#HADOOP VARIABLES START
export JAVA_HOME=/jdk/jdk1.8.0_91
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$JAVA_HOME/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
exportHADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
exportHADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
#HADOOP VARIABLES END
#set Hive environment
export HIVE_HOME=/usr/local/hadoop/hive
export PATH=$PATH:$HIVE_HOME/bin
export CLASSPATH=$CLASSPATH:$HIVE_HOME/bin
6.gedit /etc/profile
#set java environment
export JAVA_HOME=/jdk/jdk1.8.0_91
export JRE_HOME=$JAVA_HOME/jre
exportCLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
#set hive
export HIVE_HOME=/usr/local/hadoop/hive
export PATH=$HIVE_HOME/bin:$PATH
7.重命名配置文件
對conf 目錄下的配置文件重命名,執行如下命令:
mv hive-env.sh.template hive-env.sh
mv hive-default.xml.template hive-site.xml
8.在hive-env.sh中
HADOOP_HOME=/usr/local/hadoop
export HIVE_CONF_DIR=/usr/local/hadoop/hive/conf
9.修改hive-site.xml

hive.exec.scratchdir
/usr/local/hadoop/hive/tmp
HDFS rootscratch dir for Hive jobs which gets created with write all (733) permission.For each connecting user, an HDFS scratch dir:${hive.exec.scratchdir}/<username> is created, with${hive.scratch.dir.permission}.



hive.exec.local.scratchdir
/usr/local/hadoop/hive/tmp
Local scratchspace for Hive jobs



hive.downloaded.resources.dir
/usr/local/hadoop/hive/tmp
Maximum number of bytes ascript is allowed to emit to standard error (per map-reduce task).
This prevents runaway scriptsfrom filling logs partitions to capacity




hive.metastore.warehouse.dir
/usr/local/hadoop/hive/warehouse
location ofdefault database for the warehouse



hive.metastore.uris
thrift://127.0.0.1:9083
Thrift URIfor the remote metastore. Used by metastore client to connect to remotemetastore.



hive.metastore.local
true



javax.jdo.option.ConnectionPassword
myhive
password touse against metastore database



javax.jdo.option.ConnectionURL jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExsit=true;characterEncoding=UTF-8
JDBC connectstring for a JDBC metastore



javax.jdo.option.ConnectionUserName
myhive
Username touse against metastore database


Copyright © Linux教程網 All Rights Reserved