前言
: 实践是检验真理的唯一需求
前提
1.当前环境已安装Hadoop全分布系统 2.本地安装的有MySQL数据库 3.MySQL的JDBC安装包 4.Hive安装包
操作
1.解压安装包 tar -zvxf /安装包的安装路径 -C /解压后的安装路径 2.更改名字(将带版本号的hive 改成hive) mv /解压后hive所在路径 /更改名字后的路径 3.修改归属用户 chown -R hadoop:hadoop /hive的路径 4.环境配置 关闭防火墙 systemctl stop firewalld 设置防火墙服务开机不启动 systemctl disable firewalld 5.卸载Linux系统中自带的MariaDB 查询已安装的 MariaDB rpm -qa | grep MariaDB 卸载MariaDB rpm -e --nodeps 刚刚查询出来的软件包 6.安装 MySql 数据库 进入安装包所在路径 解压三个安装包(依据你的安装包后缀使用该命令) mysql-community-common mysql-community-libs mysql-community-client 7.安装mysql server安装包(依据你的安装包后缀使用该命令) mysql-community-server 8.修改MySQL数据库配置 vi /etc/my.cnf symbolic-links=0 设置 innodb 为默认的存储引擎 default-storage-engine=innodb 设置每个表的数据单独保存,而不是统一保存在 innodb 系统表空间中,单独保存有方便管理和提升性能两方面优势。 innodb_file_per_table 设置支持中文编码字符集 collation-server=utf8_general_ci 设置用户登录到数据库之后,在执行第一次查询之前执行 SET NAME utf8 命令,将使用的字符编码设定为utf8 init-connect='SET NAMES utf8' 将MySQL服务器字符集设定为交8 character-set-server=utf8 9.启动Mysql 数据库 systemctl start mysqld 10.查询Mysql数据库状态 systemctl status mysqld 若显示的是active(running) 则运行的正常 若mysql 的进程状态是failed,则表示 mysql 数库显示启动异常,此时需要排查 /etc/my.cnf 11.查询MySql数据库默认密码 cat /var/log/mysqld.log | grep password 12.Mysql 数据库初始化 初始化数据库 mysql_secure_installation Enter password for user root: # 输入/var/log/mysqld.log 文件中查询到的默认 root 用户登录密码 The 'validate_password' plugin is installed on the server. The subsequent steps will run with the existing configuration of the plugin. Using existing password for root. Estimated strength of the password: 100 Change the password for root ? ((Press y|Y for Yes, any other key for No) : y New password: # 输入新密码 Password123$ Re-enter new password: # 再次输入新密码 Password123$ Estimated strength of the password: 100 Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : y # 输入 y By default, a MySQL installation has an anonymous user, allowing anyone to log into MySQL without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? (Press y|Y for Yes, any other key for No) : y # 输入 y Success. Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? (Press y|Y for Yes, any other key for No) : n # 输入 n ... skipping. By default, MySQL comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y # 输入 y - Dropping test database... Success. - Removing privileges on test database... Success. Reloading the privilege tables will ensure that all changesmade so far will take effect immediately.Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y # 输入 ySuccess.All done! 13.添加root用户从本地和远程访问Mysql数据库表单的授权 mysql -uroot -p exit; 14.配置Hive 设置HIve环境变量 vi /etc/profile # set hive environment export HIVE_HOVE = /usr/local/src/hive export PATH=$PATH:$HIVE_HOME/bin 使环境变量配置生效 source /etc/profile 15.修改Hive组件配置文件 su - hadoop cp /hive的路径/conf/hive-default.xml.template /hive的路径/conf/hive-site.xml 16. 修改 hive-site.xml 文件实现 Hive 连接 Mysql 数据库,并设定Hive临时文件存储路径 vi /hive的路径/conf/hive-site.xml 1)设置 MySQL 数据库连接。 <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://master:3306/hive? createDatabaseIfNotExist=true&useSSL=false</value> <description>JDBC connect string for a JDBC metastore</description> 2)配置 MySQL 数据库 root 的密码。 <property> <name>javax.jdo.option.ConnectionPassword</name> <value>Password123$</value> <description>password to use againsts database</description> </property> 3)验证元数据存储版本一致性。若默认 false,则不用修改。 <property> <name>hive.metastore.schema.verification</name> <value>false</value> <description> Enforce metastore schema version consistency. True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic False: Warn if the version information stored in metastore doesn't match with one from in Hive jars. </description> </property> 4)配置数据库驱动。 <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> 5)配置数据库用户名 javax.jdo.option.ConnectionUserName 为 root。 <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> <description>Username to use against metastore database</description> </property> 6)将以下位置的 ${system:java.io.tmpdir}/${system:user.name} 替换为 “/usr/local/src/hive/tmp”目录及其子目录。 需要替换以下 4 处配置内容: <name>hive.querylog.location</name> <value>/usr/local/src/hive/tmp</value> <description>Location of Hive run time structured log file</description> <name>hive.exec.local.scratchdir</name> <value>/usr/local/src/hive/tmp</value> <name>hive.downloaded.resources.dir</name> <value>/usr/local/src/hive/tmp/resources</value> <name>hive.server2.logging.operation.log.location</name> <value>/usr/local/src/hive/tmp/operation_logs</value> 7)在 Hive 安装目录中创建临时文件夹 tmp。 [hadoop@master ~]$ mkdir /usr/local/src/hive/tmp 至此,Hive 组件安装和配置完成 15.初始化 hive 元数据 cp /mysql-connector-java路径 /hive的路径/lib/ 16.重新启动hadoop即可 dxc lib stop-all.sh start-all.sh dxc schematool -initSchema -dbType mysql 17.启动 hive dxc hive