ES官网 GEO

转载

技术领航探索者 2024-11-04 10:24:37

文章标签 ES官网 GEO es 大数据 elasticsearch JAVA 文章分类 架构后端开发

ES在我们的商品搜索或者一些需要频繁的模糊查询的业务场景下会经常用到。这里简单介绍下ES的安装。安装完成之后可以使用http请求的方式对数据做增删改查的操作。

ES也是使用java语言开发。所以我们使用之前打好的java的镜像服务器拉取到本地启动。然后在这个容器中安装es

1. 安装es

打开网址https://www.elastic.co/cn/downloads/elasticsearch找到我们需要安装的es的版本。我这里安装的使es7的版本。我们点击下载就可以拿到下载的链接。

wget -t 0 -c https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.1-linux-x86_64.tar.gz

使用上面的命令下载es的包

tar -zxvf elasticsearch-7.10.1-linux-x86_64.tar.gz #解压es的安装包

es的目录结构如下图所示:

ES官网 GEO_JAVA

我们启动es之前需要先修改系统的配置并且不能使用root用户启动。在启动es之前我们需要先做一些前置的工作

修改系统参数:

设置centos系统的内核参数

vi /etc/sysctl.conf #打开系统参数配置文件

在文件的最后添加下面的配置

vm.max_map_count=655360

sysctl -p #执行该命令确保修改配置生效

修改centos的资源参数:

vi /etc/security/limits.conf #打开centos资源配置文件

在该配置文件的最后加上下面的配置:

* soft nofile 65536


* hard nofile 131072


* soft nproc 65536


* hard nproc 131072

设置用户资源参数:

vi /etc/security/limits.d/20-nproc.conf #打开设置用户资源参数的配置文件

在配置文件的末尾加上如下的配置:

elk soft nproc 65536

设置elk用户可打开的线程数

2. 创建新用户启动es

由于我们es5.0以上的版本不能使用root用户启动。所以需要创建一个用户启动es

useradd elk #创建用户elk

groupadd elk #创建组elk

useradd elk -g elk #将用户添加到组

mkdir -pv /home/install/es/data/{data,logs}# 创建数据和日志目录

chown -R elk:elk /home/install/es/data/

chown -R elk:elk /home/install/es/

修改es安装目录下config/elasticsearch.yml 配置如下:

path.data: /home/install/es/data/data
path.logs: /home/install/es/data/logs
cluster.name: my-application
node.name: node-1
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["node-1"]

我们创建了elk用户并给该用户分配了相应的权限。同时修改了es的相关配置。我们需要切换到elk用户下启动es

su elk #切换到elk用户

/home/install/es/bin/elasticsearch #启动es 该命令是es启动文件的位置

我们打开浏览器访问es的http服务。如下图所示:

ES官网 GEO_大数据_02

坑1 Elasticsearch7.3.1启动指定JDK11

虚拟机开发环境的jdk是1.8，在启动Elasticsearch7.3.1的时候，启动日志会有如下信息：

uture versions of Elasticsearch will require Java 11; your Java version from [/opt/jdk1.8.0_211/jre] does not meet this requirement
1
es和jdk有着对应的依赖关系。
下载安装包

wget https://download.java.net/java/GA/jdk11/13/GPL/openjdk-11.0.1_linux-x64_bin.tar.gz
1
解压到指定目录
tar -xzvf jdk-11.0.4_linux-x64_bin.tar.gz /usr/local/
我们日常的代码开发都是使用的JDK1.8，所以这里不会把JAVA_HOME配置成JDK11，需更改Elasticsearch的启动文件，使它指向我们下载的JDK11.
修改配置文件
vi elasticsearch
添加一下几行内容

#配置自己的jdk11
export JAVA_HOME=/opt/jdk-11.0.1
export PATH=$JAVA_HOME/bin:$PATH

#添加jdk判断
if [ -x "$JAVA_HOME/bin/java" ]; then
        JAVA="/opt/jdk-11.0.1/bin/java"
else
        JAVA=`which java`
fi

完整的配置文件elasticsearch

#!/bin/bash

# CONTROLLING STARTUP:
#
# This script relies on a few environment variables to determine startup
# behavior, those variables are:
#
#   ES_PATH_CONF -- Path to config directory
#   ES_JAVA_OPTS -- External Java Opts on top of the defaults set
#
# Optionally, exact memory values can be set using the `ES_JAVA_OPTS`. Note that
# the Xms and Xmx lines in the JVM options file must be commented out. Example
# values are "512m", and "10g".
#
#   ES_JAVA_OPTS="-Xms8g -Xmx8g" ./bin/elasticsearch

#配置自己的jdk11
export JAVA_HOME=/opt/jdk-11.0.1
export PATH=$JAVA_HOME/bin:$PATH

source "`dirname "$0"`"/elasticsearch-env

if [ -z "$ES_TMPDIR" ]; then
  ES_TMPDIR=`"$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.TempDirectory`
fi

ES_JVM_OPTIONS="$ES_PATH_CONF"/jvm.options
JVM_OPTIONS=`"$JAVA" -cp "$ES_CLASSPATH" org.elasticsearch.tools.launchers.JvmOptionsParser "$ES_JVM_OPTIONS"`
ES_JAVA_OPTS="${JVM_OPTIONS//\$\{ES_TMPDIR\}/$ES_TMPDIR}"

#添加jdk判断
if [ -x "$JAVA_HOME/bin/java" ]; then
        JAVA="/opt/jdk-11.0.1/bin/java"
else
        JAVA=`which java`
fi

# manual parsing to find out, if process should be detached
if ! echo $* | grep -E '(^-d |-d$| -d |--daemonize$|--daemonize )' > /dev/null; then
  exec \
    "$JAVA" \
    $ES_JAVA_OPTS \
    -Des.path.home="$ES_HOME" \
    -Des.path.conf="$ES_PATH_CONF" \
    -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" \
    -Des.distribution.type="$ES_DISTRIBUTION_TYPE" \
    -Des.bundled_jdk="$ES_BUNDLED_JDK" \
    -cp "$ES_CLASSPATH" \
    org.elasticsearch.bootstrap.Elasticsearch \
    "$@"
else
  exec \
    "$JAVA" \
    $ES_JAVA_OPTS \
    -Des.path.home="$ES_HOME" \
    -Des.path.conf="$ES_PATH_CONF" \
    -Des.distribution.flavor="$ES_DISTRIBUTION_FLAVOR" \
    -Des.distribution.type="$ES_DISTRIBUTION_TYPE" \
    -Des.bundled_jdk="$ES_BUNDLED_JDK" \
    -cp "$ES_CLASSPATH" \
    org.elasticsearch.bootstrap.Elasticsearch \
    "$@" \
    <&- &
  retval=$?
  pid=$!
  [ $retval -eq 0 ] || exit $retval
  if [ ! -z "$ES_STARTUP_SLEEP_TIME" ]; then
    sleep $ES_STARTUP_SLEEP_TIME
  fi
  if ! ps -p $pid > /dev/null ; then
    exit 1
  fi
  exit 0
fi

exit $?

垃圾收集器警告,看下JDK11支持的垃圾回收器
修改jvm.options
将： -XX:+UseConcMarkSweepGC
改为：-XX:+UseG1GC
完整配置如下

## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
## 原有的注释了，添加了G1回收器
## -XX:+UseConcMarkSweepGC
-XX:+UseG1GC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1GC is only supported on JDK version 10 or later.
# To use G1GC uncomment the lines below.
# 10-:-XX:-UseConcMarkSweepGC
# 10-:-XX:-UseCMSInitiatingOccupancyOnly
# 10-:-XX:+UseG1GC
# 10-:-XX:InitiatingHeapOccupancyPercent=75

## DNS cache policy
# cache ttl in seconds for positive DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
-Des.networkaddress.cache.ttl=60
# cache ttl in seconds for negative DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
# forever
-Des.networkaddress.cache.negative.ttl=10

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=data

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=logs/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:logs/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

保存重新启动,警告消失.

坑2 Elasticsearch在后台启动

Elasticsearch在linux下使用命令sh elasticsearch start，按键ctrl+c的时候程序就会stop掉，如何将程序在后台启动呢？
需要使用:./elasticsearch -d
这时执行的时候会出现没有权限./elasticsearch: Permission denied
需要授权执行命令:chmod +x bin/elasticsearch
再次执行./elasticsearch -d即可启动
使用ps aux|grep elasticsearch可以查看是否启动

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。