es nested类型增加mapping es nested filter

官方解释

首先来看一下官方文档：Sorting by Nested Fields
看不懂的没关系，它其实就只有两句是有用的，就是那两段代码。

索引

PUT /my_index/blogpost/2
{
  "title": "Investment secrets",
  "body":  "What they don't tell you ...",
  "tags":  [ "shares", "equities" ],
  "comments": [
    {
      "name":    "Mary Brown",
      "comment": "Lies, lies, lies",
      "age":     42,
      "stars":   1,
      "date":    "2014-10-18"
    },
    {
      "name":    "John Smith",
      "comment": "You're making it up!",
      "age":     28,
      "stars":   2,
      "date":    "2014-10-16"
    }
  ]
}

嵌套查询

GET /_search
{
  "query": {
    "nested": { 
      "path": "comments",
      "filter": {
        "range": {
          "comments.date": {
            "gte": "2014-10-01",
            "lt":  "2014-11-01"
          }
        }
      }
    }
  },
  "sort": {
    "comments.stars": { 
      "order": "asc",   
      "mode":  "min",   
      "nested_filter": { 
        "range": {
          "comments.date": {
            "gte": "2014-10-01",
            "lt":  "2014-11-01"
          }
        }
      }
    }
  }
}

以上是官方代码，方便打不开网页的同学查看。下面是我遇到的问题及解决方法。

记录

这里只提供了RestfulAPI，我在使用java的过程中遇到了很多问题。有超过一半的时间都耗在这里了，特此记录一下。

问题

项目中使用java API，所以参照ResefulAPI写出来一段代码。
首先是有一段正确的restful 代码，这一段是仿照官方文档写的，运行后测试正确无误。

{
  "sort": {
    "goods_sale_number.sale_num": {
      "order": "asc",
      "nested_filter": {
        "term": {
          "warehouse_id": "22" }
      },
      "missing": "_last"
    }
  },
  "size": "99999",
  "_source": [
    "goods_sale_number",
    "id"
  ]
}

错误的java代码，根据json转换的，运行后无法排序。

String flag = sortOrder.toString().equals("desc") ? "_last" : "_first";
FilterBuilder termFilter = FilterBuilders.termFilter("goods.goods_sale_number.warehouse_id", warehouseId);
searchRequestBuilder.addSort(SortBuilders.fieldSort("goods_sale_number.sale_num")
        .setNestedFilter(FilterBuilders.nestedFilter("goods_sale_number", termFilter))
        .setNestedPath("goods.goods_sale_number")
        .order(sortOrder).missing(flag));

乍一看，其实java代码和restful的代码并没有什么区别，但是却一直不能使用。

分析及解决问题

1. 我尝试过各种各样的写法，改变访问的路径，切换过滤的类型，能试过的方法都试过了，但是还是不可以正确排序。
2. 由于SearchRequestBuilder类有一个方法可以直接setSource(String)，当然，它只能单独使用。如果还有其它的参数，可以使用searchRequestBuilder.setExtraSource(String)。所以我尝试把json的值放在这个函数里面使用，结果是正确的，正序逆序都是正确的。
3. 猜测：可能是转换的过程中出错了。（由于elasticsearch 是基于lucene的，所以我猜测是json转换java的结果可能和我写的不一样。实际在查看代码后发现其实是把java代码转换成了json，放在了searchRequestBuilder.source里面）

elasticsearch源码分析

源码

package org.elasticsearch.action.search;
public class SearchRequestBuilder extends ActionRequestBuilder<SearchRequest, SearchResponse, SearchRequestBuilder, Client> {
...
//这个方法就是解决问题的根源
    public SearchRequest request() {
        if(this.sourceBuilder != null) {
            ((SearchRequest)this.request).source(this.sourceBuilder());
        }

        return (SearchRequest)this.request;
    }
...
}

分析

在此我们可以看到有一个this.sourceBuilder()，它里面的值，就是我们写的java代码，query，filter，sort 等等都会在这里转换成json，写入到searchRequestBuilder.source，而searchRequestBuilder.extraSource则是之前我写入的另一部分json，得到这两部分值后对比一下。

//source
"sort" : [ {
    "goods_sale_number.sale_num" : {
      "order" : "desc",
      "missing" : "_last",
      "nested_filter" : {
        "nested" : {
          "filter" : {
            "term" : {
              "goods.goods_sale_number.warehouse_id" : "22"
            }
          },
          "path" : "goods_sale_number"
        }
      },
      "nested_path" : "goods.goods_sale_number"
    }
  } ]

//extraSource
  "sort": {
    "goods_sale_number.sale_num": {
      "order": "asc",
      "nested_filter": {
        "term": {
          "warehouse_id": "22"
        }
      },
      "missing": "_last"
    }
  }

可以看到有明显的不同，不同就在于，中间多了几个nested的嵌套。所以分析出可能是由于对英文文档理解不当，sort里面的查询并非是嵌套查询，只是一个普通的查询。因为它本身在添加排序的时候就已经是嵌套了。

修改后的java代码：

String flag = sortOrder.toString().equals("desc") ? "_last" : "_first";
FilterBuilder termFilter = FilterBuilders.termFilter("warehouse_id", warehouseId);
searchRequestBuilder.addSort(SortBuilders.fieldSort("goods_sale_number.sale_num")
        .setNestedFilter(termFilter)
        .order(sortOrder).missing(flag));

再次分析json：

//source
  "sort" : [ {
    "goods_sale_number.sale_num" : {
      "order" : "desc",
      "missing" : "_last",
      "nested_filter" : {
        "term" : {
          "warehouse_id" : "22"
        }
      }
    }
  } ]

这一次的值和restful API 的json一模一样了。再次执行后发现正确无误。

最终正确的代码

正确的java代码

String flag = sortOrder.toString().equals("desc") ? "_last" : "_first";
FilterBuilder termFilter = FilterBuilders.termFilter("warehouse_id", warehouseId);
searchRequestBuilder.addSort(SortBuilders.fieldSort("goods_sale_number.sale_num")
        .setNestedFilter(termFilter)
        .order(sortOrder).missing(flag));

问题总结

1. 把英文学好
2. 遇到问题打断点
3. 看源码
4. 要有耐心
5. 要细心

其间遇到的其它问题（child query must only match non-parent docs）

一个错误日志

[2015-10-20 18:15:25,553][DEBUG][action.search.type       ] [local] [shop][4], node[uv2Ii94-R8a_Wk00s7bPew], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@1ba36675]
org.elasticsearch.search.query.QueryPhaseExecutionException: [shop][4]: query[ConstantScore(cache(+_type:goods +org.elasticsearch.index.search.nested.NonNestedDocsFilter@5005b052))],from[0],size[10],sort[<custom:"goods_sale_number.sale_num": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@31026d4e>!]: Query Failed [Failed to execute main query]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:163)
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:289)
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:300)
        at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:231)
        at org.elasticsearch.search.action.SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:228)
        at org.elasticsearch.search.action.SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:559)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalStateException: child query must only match non-parent docs, but parent docID=1 matched childScorer=class org.apache.lucene.search.ConstantScoreQuery$ConstantScorer
        at org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2203)
        at org.elasticsearch.common.cache.LocalCache.get(LocalCache.java:3937)
        at org.elasticsearch.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4739)
        at org.elasticsearch.index.cache.fixedbitset.FixedBitSetFilterCache.getAndLoadIfNotPresent(FixedBitSetFilterCache.java:136)
        at org.elasticsearch.index.cache.fixedbitset.FixedBitSetFilterCache.access$100(FixedBitSetFilterCache.java:73)
        at org.elasticsearch.index.cache.fixedbitset.FixedBitSetFilterCache$FixedBitSetFilterWrapper.getDocIdSet(FixedBitSetFilterCache.java:218)
        at org.elasticsearch.index.fielddata.IndexFieldData$XFieldComparatorSource$Nested.innerDocs(IndexFieldData.java:150)
        at org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource$1.getLongValues(LongValuesComparatorSource.java:73)
        at org.apache.lucene.search.FieldComparator$LongComparator.setNextReader(FieldComparator.java:716)
        at org.apache.lucene.search.TopFieldCollector$OneComparatorNonScoringCollector.setNextReader(TopFieldCollector.java:97)
        at org.elasticsearch.common.lucene.MultiCollector.setNextReader(MultiCollector.java:66)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:612)
        at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:191)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:581)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:533)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:510)
        at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:150)
        ... 8 more
Caused by: java.lang.IllegalStateException: child query must only match non-parent docs, but parent docID=1 matched childScorer=class org.apache.lucene.search.ConstantScoreQuery$ConstantScorer
        at org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:286)
        at org.elasticsearch.index.cache.fixedbitset.FixedBitSetFilterCache$2.call(FixedBitSetFilterCache.java:148)
        at org.elasticsearch.index.cache.fixedbitset.FixedBitSetFilterCache$2.call(FixedBitSetFilterCache.java:136)
        at org.elasticsearch.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4742)
        at org.elasticsearch.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3527)
        at org.elasticsearch.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2319)
        at org.elasticsearch.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2282)
        at org.elasticsearch.common.cache.LocalCache$Segment.get(LocalCache.java:2197)
        ... 25 more

错误原因

当前嵌套的字段不在父级下面。
也就是，有一个shop索引（index），它有goods类型（type），然后它的属性（properties）里面有一个嵌套类型字段 sale_number （nested），它有两个字段叫 sale_number和 id，此时如果你使用 sale_number.sale_number 去访问这个字段是访问不到的，需要使用 goods.sale_number.sale_number，所以加上前面的type就可以了。但是这是在代码错误的时候遇到的错误的问题，仅此记录，它并不能在实际使用中经常遇到。