先准备测试数据
创建映射:
post:http://localhost:9200/xc_course/doc/_mapping
{
"properties": {
"description": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_smart"
},
"name": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_smart"
},
"pic":{
"type":"text",
"index":false
},
"price": {
"type": "float"
},
"studymodel": {
"type": "keyword"
},
"timestamp": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
初始化文档:
http://localhost:9200/xc_course/doc/1
{
"name": "Bootstrap开发",
"description": "Bootstrap是由Twitter推出的一个前台页面开发框架,是一个非常流行的开发框架,此框架集成了多种页面效果。此开发框架包含了大量的CSS、JS程序代码,可以帮助开发者(尤其是不擅长页面开发的程序人员)轻松的实现一个不受浏览器限制的精美界面效果。",
"studymodel": "201002",
"price":38.6,
"timestamp":"2018-04-25 19:11:35",
"pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg"
}
http://localhost:9200/xc_course/doc/2
{
"name": "java编程基础",
"description": "java语言是世界第一编程语言,在软件开发领域使用人数最多。",
"studymodel": "201001",
"price":68.6,
"timestamp":"2018-03-25 19:11:35",
"pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg"
}
http://localhost:9200/xc_course/doc/3
{
"name": "spring开发基础",
"description": "spring 在java领域非常流行,java程序员都在用。",
"studymodel": "201001",
"price":88.6,
"timestamp":"2018-02-24 19:11:35",
"pic":"group1/M00/00/00/wKhlQFs6RCeAY0pHAAJx5ZjNDEM428.jpg"
}
测试数据
DSL是ES提出的基于JSON的搜索方法
1.查询所有文档
POST http://localhost:9200/xc_course/doc/_search
Body中的JSON数据
query--------搜索的方式
_source------设置要显示的字段
{
"query":{
"match_all":{}
},
"_source":["name","studymodel"]
}
返回的结果集说明:
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 3,
"max_score": 1,
"hits": [
{
"_index": "xc_course",
"_type": "doc",
"_id": "1",
"_score": 1,
"_source": {
"studymodel": "201002",
"name": "Bootstrap开发"
}
},
{
"_index": "xc_course",
"_type": "doc",
"_id": "2",
"_score": 1,
"_source": {
"studymodel": "201001",
"name": "java编程基础"
}
},
{
"_index": "xc_course",
"_type": "doc",
"_id": "3",
"_score": 1,
"_source": {
"studymodel": "201001",
"name": "spring开发基础"
}
}
]
}
}
返回的搜索结果
took:本次操作花费的时间,单位为毫秒。
timed_out:请求是否超时
_shards:说明本次操作共搜索了哪些分片
hits:搜索命中的记录
hits.total : 符合条件的文档总数
hits.hits :匹配度较高的前N个文档
hits.max_score:文档匹配得分,这里为最高分
_score:每个文档都有一个匹配度得分,按照降序排列。
_source:显示了文档的原始内容。
对应的JavaApi实现
示例:
@SpringBootTest
@RunWith(SpringRunner.class)
public class TestSearch {
@Autowired
RestHighLevelClient restHighLevelClient;
@Autowired
RestClient restClient;
/**
* 搜索全部记录
*/
@Test
public void testSearchALl() throws IOException, ParseException {
//创建搜索请求对象
SearchRequest searchRequest = new SearchRequest("xc_course");
//搜索对象指定类型
searchRequest.types("doc");
//创建搜索源构建对象
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//搜索源设置搜索方式
searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //QueryBuilders.matchAllQuery()-----搜索全部
//设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{});
//搜索请求对象设置搜索源
searchRequest.source(searchSourceBuilder);
//执行搜索
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
//获取匹配的搜索结果
SearchHits hits = searchResponse.getHits();
//匹配到的总记录数
long totalHits = hits.getTotalHits();
//匹配度高的文档
SearchHit[] searchHits = hits.getHits();
//日期格式化对象
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
//遍历输出
for (SearchHit hit:searchHits) {
//文档主键
String id = hit.getId();
//源文档内容
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
//之前设置了源文档字段过滤集,所以搜索不到
String description = (String) sourceAsMap.get("description");
String studymodel = (String) sourceAsMap.get("studymodel");
Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp"));
//打印
System.out.println(name);
System.out.println(description);
System.out.println(studymodel);
System.out.println(timestamp);
}
}
}
jave实现DSL搜索全部记录
2.分页查询
ES支持分页查询,传入两个参数:from和size
from-----表示起始文档的下标
size------查询的文档数量
POST http://localhost:9200/xc_course/doc/_search
Body的JSON数据
{
"from":0,"size":1,
"query":{
"match_all":{}
},
"_source":["name","studymodel"]
}
JaveClient示例:
/**
* 分页搜索记录
*/
@Test
public void testSearchPage() throws IOException, ParseException {
//创建搜索请求对象
SearchRequest searchRequest = new SearchRequest("xc_course");
//搜索对象指定类型
searchRequest.types("doc");
//创建搜索源构建对象
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//页码
int page = 1;
//每页记录数
int size = 1;
//计算记录起始下标
int from = (page - 1) * size;
//设置分页参数
searchSourceBuilder.from(from);
searchSourceBuilder.size(size);
//搜索源设置搜索方式
searchSourceBuilder.query(QueryBuilders.matchAllQuery()); //QueryBuilders.matchAllQuery()-----搜索全部
//设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{});
//搜索请求对象设置搜索源
searchRequest.source(searchSourceBuilder);
//执行搜索
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
//获取匹配的搜索结果
SearchHits hits = searchResponse.getHits();
//匹配到的总记录数
long totalHits = hits.getTotalHits();
//匹配度高的文档
SearchHit[] searchHits = hits.getHits();
//日期格式化对象
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
//遍历输出
for (SearchHit hit:searchHits) {
//文档主键
String id = hit.getId();
//源文档内容
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
//之前设置了源文档字段过滤集,所以搜索不到
String description = (String) sourceAsMap.get("description");
String studymodel = (String) sourceAsMap.get("studymodel");
Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp"));
//打印
System.out.println(name);
System.out.println(description);
System.out.println(studymodel);
System.out.println(timestamp);
}
}
分页搜索记录
3.TermQuery精确查询和根据ID查询
整体匹配关键字,不再将关键字分词
POST http://localhost:9200/xc_course/doc/_search
整体与分词器里的词去匹配
精确查询Body的JSON数据
{
"query":{
"term":{
"name":"spring"
}
},
"_source":["name","studymodel"]
}
根据ID查询Body的JSON数据
{
"query":{
"ids":{
"type":"doc",
"values":["3","2","100"]
}
},
"_source":["name","studymodel"]
}
JaveClient示例:
/**
* 精确查询和根据ID查询
*/
@Test
public void testSearchTermQuery() throws IOException, ParseException {
//创建搜索请求对象
SearchRequest searchRequest = new SearchRequest("xc_course");
//搜索对象指定类型
searchRequest.types("doc");
//创建搜索源构建对象
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//搜索源设置搜索方式
//termQuery精确查询
//searchSourceBuilder.query(QueryBuilders.termQuery("name","spring")); //termQuery-----精确查询
// searchSourceBuilder.query(QueryBuilders.termQuery("name","spring开发")); //因为是整体精确查询,没有相匹配的值,搜不出来
//设置源文档字段过滤------------第一个结果集包括哪些字段,第二个结果集不包括哪些字段
//根据ID查询
//设置主键
String[] ids = new String[]{"1","2"};
searchSourceBuilder.query(QueryBuilders.termsQuery("_id",ids)); //注意:这里的QueryBuilders.termsQuery 是termsQuery
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","timestamp"},new String[]{});
//搜索请求对象设置搜索源
searchRequest.source(searchSourceBuilder);
//执行搜索
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
//获取匹配的搜索结果
SearchHits hits = searchResponse.getHits();
//匹配到的总记录数
long totalHits = hits.getTotalHits();
//匹配度高的文档
SearchHit[] searchHits = hits.getHits();
//日期格式化对象
DateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
//遍历输出
for (SearchHit hit:searchHits) {
//文档主键
String id = hit.getId();
//源文档内容
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
//之前设置了源文档字段过滤集,所以搜索不到
String description = (String) sourceAsMap.get("description");
String studymodel = (String) sourceAsMap.get("studymodel");
Date timestamp = dateFormat.parse((String) sourceAsMap.get("timestamp"));
//打印
System.out.println(name);
System.out.println(description);
System.out.println(studymodel);
System.out.println(timestamp);
}
}
精确查询和根据ID查询
注意:根据ID查询这里的QueryBuilders.termsQuery 是termsQuery
4.match Query---全文检索 match----- n.比赛; 火柴; 敌手; 相配的人/物; 相似的东西
POST http://localhost:9200/xc_course/doc/_search
全文检索提交的JSON数据1
{
"query":{
"match":{
"description":{
"query":"spring开发",
"operator":"or"
}
}
}
}
query:搜索的关键字,对于英文关键字如果有多个单词则中间要用半角逗号分隔,而对于中文关键字中间可以用逗号分隔也可以不用。
operator:or 表示 只要有一个词在文档中出现则就符合条件,and表示每个词都在文档中出现则才符合条件。
全文检索提交的JSON数据2
{
"query":{
"match":{
"description":{
"query":"spring开发框架",
"minimum_should_match":"80%"
}
}
}
}
spring开发框架”会被分为三个词:spring、开发、框架
设置"minimum_should_match": "80%"表示,三个词在文档的匹配占比为80%,即3*0.8=2.4,向上取整得2,表示至少有两个词在文档中要匹配成功。
JavaClient示例:
/**
* MatchQuery
*/
@Test
public void MatchQuery() throws IOException {
//搜索请求对象
SearchRequest searchRequest = new SearchRequest("xc_course");
//搜索对象指定类型
searchRequest.types("doc");
//搜索源创建对象
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//设置搜索源搜索方式
searchSourceBuilder.query(QueryBuilders.matchQuery("description","spring开发框架")
.minimumShouldMatch("80%"));
//请求对象设置搜索源
searchRequest.source(searchSourceBuilder);
//执行搜索
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
//获取匹配的搜索结果
SearchHits hits = searchResponse.getHits();
//匹配的总数
long totalHits = hits.getTotalHits();
//匹配的搜索结果
SearchHit[] searchHits = hits.getHits();
//遍历输出
for (SearchHit hit:searchHits) {
//文档主键
String id = hit.getId();
//源文档内容
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
//之前设置了源文档字段过滤集,所以搜索不到
String description = (String) sourceAsMap.get("description");
String studymodel = (String) sourceAsMap.get("studymodel");
//打印
System.out.println(name);
System.out.println(description);
System.out.println(studymodel);
}
}
MatchQuery---全文检索
5.MultiQuery---多字段匹配
POST http://localhost:9200/xc_course/doc/_search
JSON数据:
{
"query":{
"multi_match":{
"query":"spring开发框架",
"minimum_should_match":"50%",
"fields":["name^10","description"]
}
}
}
上面拿关键字 “spring css”去匹配name 和description字段
提升boost,通常关键字匹配上name的权重要比匹配上description的权重高,这里可以对name的权重提升
name^10” 表示权重提升10倍,执行上边的查询,发现name中包括spring关键字的文档排在前边。
JavaClient:
/**
* MultiQuery --多字段匹配
*/
/* MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架",
"name", "description")
.minimumShouldMatch("50%");
multiMatchQueryBuilder.field("name",10);//提升boost*/
6.布尔查询
布尔查询对应于Lucene的BooleanQuery查询,实现将多个查询组合起来。
POST http://localhost:9200/xc_course/doc/_search
JSON数据:
{
"_source" : [ "name", "studymodel", "description"],
"from" : 0, "size" : 1,
"query": {
"bool" : {
"must":[
{
"multi_match" : {
"query" : "spring框架",
"minimum_should_match": "50%",
"fields": [ "name^10", "description" ]
}
},
{
"term":{
"studymodel" : "201001"
}
}
]
}
}
}
must:表示必须,多个查询条件必须都满足。(通常使用must)
should:表示或者,多个查询条件只要有一个满足即可。
must_not:表示非
JavaClient示例:
//BoolQuery,将搜索关键字分词,拿分词去索引库搜索
@Test
public void testBoolQuery() throws IOException {
//创建搜索请求对象
SearchRequest searchRequest = new SearchRequest("xc_course");
searchRequest.types("doc");
//创建搜索源配置对象
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.fetchSource(new String[]{"name", "pic", "studymodel"}, new String[]{});
//multiQuery
String keyword = "spring开发框架";
MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架",
"name", "description")
.minimumShouldMatch("50%");
multiMatchQueryBuilder.field("name", 10);
//TermQuery
TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("studymodel", "201001");
//布尔查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(multiMatchQueryBuilder);
boolQueryBuilder.must(termQueryBuilder);
//设置布尔查询对象
searchSourceBuilder.query(boolQueryBuilder);
searchRequest.source(searchSourceBuilder);//设置搜索源配置
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
System.out.println(sourceAsMap);
}
}
布尔查询
7.过滤器查询
过虑是针对搜索的结果进行过虑,过虑器主要判断的是文档是否匹配,不去计算和判断文档的匹配度得分,所以过虑器性能比查询要高,且方便缓存,推荐尽量使用过虑器去实现查询或者过虑器和查询共同使用。
POST http://localhost:9200/xc_course/doc/_search
JSON数据:
{
"_source" : [ "name", "studymodel", "description","price"],
"query": {
"bool":{
"must":[
{
"multi_match" : {
"query" : "spring框架",
"minimum_should_match": "50%",
"fields": [ "name^10", "description" ]
}
}
],
"filter": [
{ "term": { "studymodel": "201001" }},
{ "range": { "price": { "gte": 60 ,"lte" : 100}}}
]
}
}
}
range:范围过虑,保留大于等于60 并且小于等于100的记录。
注意:range和term一次只能对一个Field设置范围过虑。
JavaClient示例:
//布尔查询使用过虑器
@Test
public void testFilter() throws IOException {
SearchRequest searchRequest = new SearchRequest("xc_course");
searchRequest.types("doc");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//source源字段过虑
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"},
new String[]{});
searchRequest.source(searchSourceBuilder);
//匹配关键字
MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("spring框架", "name", "description");
//设置匹配占比
multiMatchQueryBuilder.minimumShouldMatch("50%");
//提升另个字段的Boost值
multiMatchQueryBuilder.field("name",10);
searchSourceBuilder.query(multiMatchQueryBuilder);
//布尔查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(searchSourceBuilder.query());
//过虑
boolQueryBuilder.filter(QueryBuilders.termQuery("studymodel", "201001"));
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(60).lte(100));
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
String index = hit.getIndex();
String type = hit.getType();
String id = hit.getId();
float score = hit.getScore();
String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
String studymodel = (String) sourceAsMap.get("studymodel");
String description = (String) sourceAsMap.get("description");
System.out.println(name);
System.out.println(studymodel);
System.out.println(description);
}
}
过滤器
8.排序
可以在字段上添加一个或多个排序,支持在keyword、date、float等类型上添加,text类型的字段上不允许添加排序。
POST http://localhost:9200/xc_course/doc/_search
JSON数据:
过虑0--10元价格范围的文档,并且对结果进行排序,先按studymodel降序,再按价格升序
{
"_source" : [ "name", "studymodel", "description","price"],
"query": {
"bool" : {
"filter": [
{ "range": { "price": { "gte": 0 ,"lte" : 100}}}
]
}
},
"sort" : [
{
"studymodel" : "desc"
},
{
"price" : "asc"
}
]
}
JavaClient示例:
/**
* 排序
* @throws IOException
*/
@Test
public void testSort() throws IOException {
SearchRequest searchRequest = new SearchRequest("xc_course");
searchRequest.types("doc");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//source源字段过虑
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"},
new String[]{});
searchRequest.source(searchSourceBuilder);
//布尔查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
//过虑
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100));
//排序
searchSourceBuilder.sort(new FieldSortBuilder("studymodel").order(SortOrder.DESC)); //降序 DESC
searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC)); //升序 ESC
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
String index = hit.getIndex();
String type = hit.getType();
String id = hit.getId();
float score = hit.getScore();
String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
String studymodel = (String) sourceAsMap.get("studymodel");
String description = (String) sourceAsMap.get("description");
System.out.println(name);
System.out.println(studymodel);
System.out.println(description);
}
}
排序
9.高亮
高亮显示可以将搜索结果一个或多个字突出显示,以便向用户展示匹配关键字的位置。
POST http://127.0.0.1:9200/xc_course/doc/_search
JSON数据:
{
"_source" : [ "name", "studymodel", "description","price"],
"query": {
"bool" : {
"must":[
{
"multi_match" : {
"query" : "开发框架",
"minimum_should_match": "50%",
"fields": [ "name^10", "description" ],
"type":"best_fields"
}
}
],
"filter": [
{ "range": { "price": { "gte": 0 ,"lte" : 100}}}
]
}
},
"highlight":{
"pre_tags": ["<tag>"],
"post_tags": ["</tag>"],
"fields": {
"name": {},
"description":{}
}
}
}
JavaClient示例:
/**
* 高亮
* @throws IOException
*/
@Test
public void testHighlight() throws IOException {
SearchRequest searchRequest = new SearchRequest("xc_course");
searchRequest.types("doc");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//source源字段过虑
searchSourceBuilder.fetchSource(new String[]{"name","studymodel","price","description"},
new String[]{});
searchRequest.source(searchSourceBuilder);
//匹配关键字
MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("开发","name", "description");
searchSourceBuilder.query(multiMatchQueryBuilder);
//布尔查询
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(searchSourceBuilder.query());
//过虑
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").gte(0).lte(100));
//排序
searchSourceBuilder.sort(new FieldSortBuilder("studymodel").order(SortOrder.DESC));
searchSourceBuilder.sort(new FieldSortBuilder("price").order(SortOrder.ASC));
//高亮设置
HighlightBuilder highlightBuilder = new HighlightBuilder();
highlightBuilder.preTags("<tag>");//设置前缀
highlightBuilder.postTags("</tag>");//设置后缀
// 设置高亮字段
highlightBuilder.fields().add(new HighlightBuilder.Field("name"));
// highlightBuilder.fields().add(new HighlightBuilder.Field("description"));
searchSourceBuilder.highlighter(highlightBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest);
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
//名称
String name = (String) sourceAsMap.get("name");
//取出高亮字段内容
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
if(highlightFields!=null){
HighlightField nameField = highlightFields.get("name");
if(nameField!=null){
Text[] fragments = nameField.getFragments();
StringBuffer stringBuffer = new StringBuffer();
for (Text str : fragments) {
stringBuffer.append(str.string());
}
name = stringBuffer.toString();
}
}
String index = hit.getIndex();
String type = hit.getType();
String id = hit.getId();
float score = hit.getScore();
String sourceAsString = hit.getSourceAsString();
String studymodel = (String) sourceAsMap.get("studymodel");
String description = (String) sourceAsMap.get("description");
System.out.println(name);
System.out.println(studymodel);
System.out.println(description);
}
}
高亮
难产难产难产