elastic search使用总结

转载

mb5ff98246f093d 2019-04-23 16:37:00

文章标签 通用实践全文搜索 elasticsearch java xml 文章分类 运维

1. elasticsearch安装

官方下载地址：https://www.elastic.co/downloads/elasticsearch

解压文件 elasticsearch-2.4.0.zip

修改配置文件

1 2 3 4	`elasticsearch-2.4.0` `cat` `config``/elasticsearch``.yml \|``grep` `-``v` `"#"` `cluster.name: rainbow` `network.host: 127.0.0.1` `http.port: 9200`

配置说明

cluster.name表示es集群的名称，可以自定义一个自己需要的集群名称

http.port 表示对外提供http服务时的http端口。

network.host 表示本地监听绑定的ip地址，此处为测试环境，直接使用本机的ip地址 127.0.0.1.

启动说明

1	`elasticsearch-2.4.0` `nohup` `.``/bin/elasticsearch` `&`

启动后显示信息

elastic search使用总结_elasticsearch

es启动监听两个端口，9300和9200

9300端口是使用tcp客户端连接使用的端口；

9200端口是通过http协议连接es使用的端口；

2. 使用http方式增加和查询数据

增加数据（http PUT）：

1 2 3	`➜ curl -XPUT localhost:9200``/user_idx/type_tags/12` `-d\<br>` `'{"name" : "Mr.YF", "tags" : ["Go","Java","Lua","C++","Tcl","..."]}'` `{``"_index"``:``"user_idx"``,``"_type"``:``"type_tags"``,``"_id"``:``"12"``,``"_version"``:1,``"_shards"``:{``"total"``:2,``"successful"``:1,``"failed"``:0},``"created"``:``true``}`

查询数据（http GET）

elastic search使用总结_java_02

3. 使用elasticsearch客户端编写java代码访问es

编写es客户端提供者类，对es连接做简单的封装

SearchClientProvider.java

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76	`package` `org.wyf.elasticsearch.search;` `/*` ` Created by wyf on 16/9/25.` `*/` `import` `org.elasticsearch.client.transport.TransportClient;` `import` `org.elasticsearch.common.settings.Settings;` `import` `org.elasticsearch.common.transport.InetSocketTransportAddress;` `import` `org.slf4j.Logger;` `import` `org.slf4j.LoggerFactory;` `import` `org.wyf.common.property.XmlProperties;` `import` `javax.annotation.PostConstruct;` `import` `javax.annotation.PreDestroy;` `import` `java.net.InetAddress;` `import` `java.net.UnknownHostException;` `import` `java.util.HashMap;` `import` `java.util.Iterator;` `import` `java.util.Map;` `import` `static` `java.lang.System.out;` `public` `class` `SearchClientProvider {` `private` `static` `final` `Logger LOGGER = LoggerFactory.getLogger(SearchClientProvider.``class``);` `private` `TransportClient client =` `null``;` `private` `volatile` `boolean` `inited =` `false``;` `public` `TransportClient get() {` `return` `this``.client;` `}` `@PreDestroy` `public` `synchronized` `void` `close() {` `if` `(``this``.client !=` `null``) {` `this``.client.close();` `}` `}` `@PostConstruct` `public` `synchronized` `void` `init() {` `if` `(!inited) {` `try` `{` `Map<String, String> config = XmlProperties.loadFromXml(``"properties/elasticsearch.xml"``);` `if` `(config ==` `null``) {` `out.println(``"load xml err"``);` `return``;` `}` `Iterator<Map.Entry<String, String>> iterator = config.entrySet().iterator();` `Map<String, String> settingConfig =` `new` `HashMap<String, String>();` `while` `(iterator.hasNext()) {` `Map.Entry<String, String> next = iterator.next();` `if` `(!next.getKey().equals(``"transport.addresses"``)) {` `settingConfig.put(next.getKey(), next.getValue());` `}` `}` `Settings settings = Settings.builder().put(settingConfig).build();` `TransportClient client = TransportClient.builder().settings(settings).build();` `this``.client = client;` `String[] addresses = config.get(``"transport.addresses"``).split(``","``);` `for` `(String address : addresses) {` `String[] hostAndPort = address.split(``":"``);` `client.addTransportAddress(``new` `InetSocketTransportAddress(InetAddress.getByName(hostAndPort[``0``]), Integer.valueOf(hostAndPort[``1``])));` `}` `this``.inited =` `true``;` `}` `catch` `(UnknownHostException e) {` `LOGGER.error(String.format(``"init search client err:=>msg:[%s]"``, e.getMessage()), e);` `if` `(client !=` `null``) {` `this``.client.close();` `}` `}` `}` `}` `}`

编写elasticsearch的配置文件加载类

XmlProperties.java

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58	`package` `org.wyf.common.property;` `import` `org.slf4j.LoggerFactory;` `import` `org.slf4j.Logger;` `import` `java.io.File;` `import` `java.io.FileInputStream;` `import` `java.io.IOException;` `import` `java.io.InputStream;` `import` `java.util.*;` `public` `class` `XmlProperties {` `private` `static` `final` `Logger LOGGER = LoggerFactory.getLogger(XmlProperties.``class``);` `private` `XmlProperties() {` `}` `public` `static` `Map<String, String> loadFromXml(String xmlPropertiesPath) {` `try` `{` `Object in = XmlProperties.``class``.getClassLoader().getResourceAsStream(xmlPropertiesPath);` `if``(in !=` `null``) {` `LOGGER.info(``"Found the xml properties [{}] in class path,use it"``, xmlPropertiesPath);` `Map e1 = loadFromXml((InputStream)in);` `return` `e1;` `}` `Map<String, String> resMap =` `null``;` `File e =` `new` `File(xmlPropertiesPath);` `if``(!e.isFile()) {` `return` `resMap;` `}` `LOGGER.info(``"Found the xml properties [{}] in file path,use it"``, xmlPropertiesPath);` `in =` `new` `FileInputStream(``new` `File(xmlPropertiesPath));` `resMap = loadFromXml((InputStream)in);` `((InputStream) in).close();` `return` `resMap;` `}` `catch` `(Exception var7) {` `LOGGER.error(``"Load xml properties ["` `+ xmlPropertiesPath +` `"] error."``, var7);` `}` `return` `null``;` `}` `public` `static` `Map<String, String> loadFromXml(InputStream in)` `throws` `IOException {` `Properties properties =` `new` `Properties();` `properties.loadFromXML(in);` `HashMap map =` `new` `HashMap();` `Set entries = properties.entrySet();` `Iterator iter = entries.iterator();` `while``(iter.hasNext()) {` `Map.Entry entry = (Map.Entry)iter.next();` `map.put((String)entry.getKey(), (String)entry.getValue());` `}` `return` `map;` `}` `}`

编写elasticsearch简单的操作类，包含成员函数 save update get 和delete

SearchImpl.java

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92	`package` `org.wyf.elasticsearch;` `/*` ` Created by wyf on 16/9/25.` `*/` `import` `org.elasticsearch.action.delete.DeleteRequestBuilder;` `import` `org.elasticsearch.action.delete.DeleteResponse;` `import` `org.elasticsearch.action.get.GetRequestBuilder;` `import` `org.elasticsearch.action.get.GetResponse;` `import` `org.elasticsearch.action.index.IndexRequestBuilder;` `import` `org.elasticsearch.action.index.IndexResponse;` `import` `org.elasticsearch.client.transport.TransportClient;` `import` `org.elasticsearch.index.engine.VersionConflictEngineException;` `import` `org.slf4j.Logger;` `import` `org.slf4j.LoggerFactory;` `import` `org.wyf.elasticsearch.search.SearchClientProvider;` `import` `javax.annotation.PostConstruct;` `import` `java.util.Map;` `public` `class` `SearchImpl` `implements` `ISearch {` `private` `static` `final` `Logger LOGGER = LoggerFactory.getLogger(SearchImpl.``class``);` `private` `SearchClientProvider searchClientProvider;` `public` `SearchImpl() {` `searchClientProvider =` `new` `SearchClientProvider();` `searchClientProvider.init();` `}` `public` `Map<String, Object> save(String index, String type, String id, Map<String, Object> data) {` `TransportClient client = searchClientProvider.get();` `IndexRequestBuilder builder = client.prepareIndex(index, type, id);` `IndexResponse response = builder` `.setSource(data)` `.execute()` `.actionGet();` `LOGGER.info(``"save index:=>index:{}, type:{}, id:{}, data:{}, rsp:{}"``, index, type, id, data, response);` `return` `data;` `}` `public` `int` `update(String index, String type, String id, Map<String, Object> data) {` `int` `i =` `2``;` `do` `{` `try` `{` `if` `(_innerUpdate(index, type, id, data))` `return` `1``;` `}` `catch` `(VersionConflictEngineException e) {` `LOGGER.warn(String.format(``"update index:=>index:%s, type:%s, id:%s, data:%s, rsp:%s"``,` `index, type, id, data, e.getMessage()), e);` `}` `}` `while` `((i--) >` `0``);` `return` `_innerUpdate(index, type, id, data) ?` `1` `:` `0``;` `}` `public` `int` `delete(String index, String type, String id) {` `TransportClient client = searchClientProvider.get();` `DeleteRequestBuilder builder = client.prepareDelete(index, type, id);` `DeleteResponse response = builder.execute().actionGet();` `LOGGER.info(``"delete index:=>index:{}, type:{}, id:{}, rsp:{}"``,` `index, type, id, response);` `return` `response.isFound() ?` `1` `:` `0``;` `}` `public` `Map<String, Object> get(String index, String type, String id) {` `TransportClient client = searchClientProvider.get();` `GetRequestBuilder builder = client.prepareGet(index, type, id);` `GetResponse response = builder.execute().actionGet();` `return` `response.isExists() ? response.getSource() :` `null``;` `}` `private` `boolean` `_innerUpdate(String index, String type, String id, Map<String, Object> data) {` `TransportClient client = searchClientProvider.get();` `GetRequestBuilder getRequestBuilder = client.prepareGet(index, type, id);` `GetResponse getResponse = getRequestBuilder.execute().actionGet();` `if` `(getResponse.isExists()) {` `final` `long` `version = getResponse.getVersion();` `final` `Map<String, Object> source = getResponse.getSource();` `source.putAll(data);` `IndexRequestBuilder builder = client.prepareIndex(index, type, id);` `IndexResponse response = builder` `.setVersion(version)` `.setSource(source)` `.execute()` `.actionGet();` `LOGGER.info(``"update index:=>index:{}, type:{}, id:{}, data:{}, rsp:{}"``,` `index, type, id, data, response);` `return` `true``;` `}` `throw` `new` `RuntimeException(String.format(``"can not get document:=>index:%s, type:%s, id:%s "``, index, type, id));` `}` `}`

编写单元测试类

ElasticSearchTest.java

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48	`package` `org.wyf;` `import` `org.junit.Before;` `import` `org.junit.Test;` `import` `org.wyf.elasticsearch.ISearch;` `import` `org.wyf.elasticsearch.SearchImpl;` `import` `java.util.HashMap;` `import` `java.util.Iterator;` `import` `java.util.Map;` `import` `java.util.Set;` `/*` ` Created by wyf on 16/9/25.` `*/` `public` `class` `ElasticsearchTest {` `ISearch search;` `@Test` `public` `void` `get() {` `Map<String,Object> map = search.get(``"a"``,` `"type1"``,` `"123"``);` `Set<String> set = map.keySet();` `for` `(Iterator<String> it = set.iterator(); it.hasNext();) {` `String key = it.next();` `System.out.println(key +` `":"` `+ map.get(key));` `}` `}` `@Test` `public` `void` `save() {` `Map<String, Object> values =` `new` `HashMap<String, Object>();` `values.put(``"k1"``,` `"v1"``);` `values.put(``"k2"``,` `"v2"``);` `Map<String,Object> map = search.save(``"a"``,` `"type1"``,` `"123"``, values);` `Set<String> set = map.keySet();` `for` `(Iterator<String> it = set.iterator(); it.hasNext();) {` `String key = it.next();` `System.out.println(key +` `":"` `+ map.get(key));` `}` `}` `@Before` `public` `void` `before() {` `search =` `new` `SearchImpl();` `}` `}`

elasticsearch的配置文件

1 2 3 4 5 6 7 8	`<?``xml` `version="1.0" encoding="UTF-8" standalone="no"?>` `<!``DOCTYPE` `properties SYSTEM "http://java.sun.com/dtd/properties.dtd">` `<``properties``>` `<``comment``>elasticsearch的配置</``comment``>` `<``entry` `key="client.transport.sniff">true</``entry``>` `<``entry` `key="cluster.name">rainbow</``entry``>` `<``entry` `key="transport.addresses">127.0.0.1:9300</``entry``>` `</``properties``>`

测试结果：

测试save函数结果

elastic search使用总结_elasticsearch_03

http接口查询结果页面

elastic search使用总结_通用实践_04

遇到的问题

最开始调试时返回如下错误

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39	`NoNodeAvailableException[None of the configured nodes are available: [{#transport#-``1``}{``127.0``.``0.1``}{``127.0``.``0.1``:``9200``}]` `]` `at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:``290``)` `at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:``207``)` `at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:``55``)` `at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:``283``)` `at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:``347``)` `at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:``85``)` `at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:``59``)` `at org.wyf.elasticsearch.SearchImpl.save(SearchImpl.java:``36``)` `at org.wyf.ElasticsearchTest.save(ElasticsearchTest.java:``35``)` `at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)` `at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:``62``)` `at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:``43``)` `at java.lang.reflect.Method.invoke(Method.java:``498``)` `at org.junit.runners.model.FrameworkMethod$``1``.runReflectiveCall(FrameworkMethod.java:``47``)` `at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:``12``)` `at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:``44``)` `at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:``17``)` `at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:``26``)` `at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:``271``)` `at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:``70``)` `at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:``50``)` `at org.junit.runners.ParentRunner$``3``.run(ParentRunner.java:``238``)` `at org.junit.runners.ParentRunner$``1``.schedule(ParentRunner.java:``63``)` `at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:``236``)` `at org.junit.runners.ParentRunner.access$``000``(ParentRunner.java:``53``)` `at org.junit.runners.ParentRunner$``2``.evaluate(ParentRunner.java:``229``)` `at org.junit.runners.ParentRunner.run(ParentRunner.java:``309``)` `at org.junit.runner.JUnitCore.run(JUnitCore.java:``160``)` `at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:``117``)` `at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:``42``)` `at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:``253``)` `at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:``84``)` `at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)` `at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:``62``)` `at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:``43``)` `at java.lang.reflect.Method.invoke(Method.java:``498``)` `at com.intellij.rt.execution.application.AppMain.main(AppMain.java:``147``)`