Elasticsearch 入门教程 – GEO位置搜索

在ElasticSearch中,地理位置通过geo_point这个数据类型来支持。地理位置的数据需要提供经纬度信息,当经纬度不合法时,ES会拒绝新增文档。这种类型的数据支持距离计算,范围查询等。在底层,索引使用Geohash实现。

1、创建索引

PUT创建一个索引cn_large_cities,mapping为city:

{
    "mappings": {
        "city": {
            "properties": {
                "city": {"type": "string"},
                "state": {"type": "string"},
                "location": {"type": "geo_point"}            }        }    }}1234567891011

geo_point类型必须显示指定,ES无法从数据中推断。在ES中,位置数据可以通过对象,字符串,数组三种形式表示,分别如下:

# "lat,lon""location":"40.715,-74.011""location": {
  "lat":40.715,
  "lon":-74.011}

# [lon ,lat]"location":[-74.011,40.715]1234567891011

POST下面4条测试数据:

{"city": "Beijing", "state": "BJ","location": {"lat": "39.91667", "lon": "116.41667"}}

{"city": "Shanghai", "state": "SH","location": {"lat": "34.50000", "lon": "121.43333"}}

{"city": "Xiamen", "state": "FJ","location": {"lat": "24.46667", "lon": "118.10000"}}

{"city": "Fuzhou", "state": "FJ","location": {"lat": "26.08333", "lon": "119.30000"}}

{"city": "Guangzhou", "state": "GD","location": {"lat": "23.16667", "lon": "113.23333"}}123456789

查看全部文档:

curl -X GET "http://localhost:9200/cn_large_cities/city/_search?pretty=true"1

返回全部的5条数据,score均为1:

这里写图片描述

2、位置过滤

ES中有4中位置相关的过滤器,用于过滤位置信息:

  • geo_distance: 查找距离某个中心点距离在一定范围内的位置

  • geo_bounding_box: 查找某个长方形区域内的位置

  • geo_distance_range: 查找距离某个中心的距离在min和max之间的位置

  • geo_polygon: 查找位于多边形内的地点。

geo_distance

该类型过滤器查找的范围如下图:

下面是一个查询例子:

{
  "query":{
    "filtered":{
      "filter":{
        "geo_distance":"1km",
        "location":{
          "lat":40.715,
          "lon": -73.988
        }      }    }  }}12345678910111213

以下查询,查找距厦门500公里以内的城市:

{
    "query":{
        "filtered":{
          "filter":{
            "geo_distance" : {
                "distance" : "500km",
                "location" : {
                    "lat" : 24.46667,
                    "lon" : 118.10000
                }            }        }    }    } 
}12345678910111213141516

geo_distance_range

{
  "query":{
    "filtered":{
      "filter":{
        "geo_distance_range":{
        "gte": "1km",
        "lt":  "2km",
        "location":{
          "lat":40.715,
          "lon": -73.988
        }      }    }  }}123456789101112131415

geo_bounding_box

{
  "query":{
    "filtered":{
      "filter":{
        "geo_bounding_box":{
        "location":{
          "top_left":{
            "lat": 40.8,
            "lon":-74.0
          },
          "bottom_right":{
            "lat":40.715,
            "lon": -73.0
          }        }      }    }  }}1234567891011121314151617181920

3、按距离排序

接着我们按照距离厦门远近查找:

{
  "sort" : [
      {
          "_geo_distance" : {
              "location" : {
                    "lat" : 24.46667,
                    "lon" : 118.10000
              }, 
              "order" : "asc",
              "unit" : "km"
          }      }
  ],
  "query": {
    "filtered" : {
        "query" : {
            "match_all" : {}        }    }  }}123456789101112131415161718192021

结果如下,依次是厦门、福州、广州…。符合我们的常识:

{
  "took": 8,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 5,
    "max_score": null,
    "hits": [
      {
        "_index": "us_large_cities",
        "_type": "city",
        "_id": "AVaiSGXXjL0tfmRppc_p",
        "_score": null,
        "_source": {
          "city": "Xiamen",
          "state": "FJ",
          "location": {
            "lat": "24.46667",
            "lon": "118.10000"
          }        },
        "sort": [          0
        ]      },
      {
        "_index": "us_large_cities",
        "_type": "city",
        "_id": "AVaiSSuNjL0tfmRppc_r",
        "_score": null,
        "_source": {
          "city": "Fuzhou",
          "state": "FJ",
          "location": {
            "lat": "26.08333",
            "lon": "119.30000"
          }        },
        "sort": [          216.61105485607183
        ]      },
      {
        "_index": "us_large_cities",
        "_type": "city",
        "_id": "AVaiSd02jL0tfmRppc_s",
        "_score": null,
        "_source": {
          "city": "Guangzhou",
          "state": "GD",
          "location": {
            "lat": "23.16667",
            "lon": "113.23333"
          }        },
        "sort": [          515.9964950041397
        ]      },
      {
        "_index": "us_large_cities",
        "_type": "city",
        "_id": "AVaiR7_5jL0tfmRppc_o",
        "_score": null,
        "_source": {
          "city": "Shanghai",
          "state": "SH",
          "location": {
            "lat": "34.50000",
            "lon": "121.43333"
          }        },
        "sort": [          1161.512141925948
        ]      },
      {
        "_index": "us_large_cities",
        "_type": "city",
        "_id": "AVaiRwLUjL0tfmRppc_n",
        "_score": null,
        "_source": {
          "city": "Beijing",
          "state": "BJ",
          "location": {
            "lat": "39.91667",
            "lon": "116.41667"
          }        },
        "sort": [          1725.4543712286697
        ]      }
    ]  }}123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100

结果返回的sort字段是指公里数。加上限制条件,只返回最近的一个城市:

{

  "from":0,
  "size":1,
  "sort" : [
      {
          "_geo_distance" : {
              "location" : {
                    "lat" : 24.46667,
                    "lon" : 118.10000
              }, 
              "order" : "asc",
              "unit" : "km"
          }      }
  ],
  "query": {
    "filtered" : {
        "query" : {
            "match_all" : {}        }    }  }}123456789101112131415161718192021222324

4、地理位置聚合

ES提供了3种位置聚合:

  • geo_distance: 根据到特定中心点的距离聚合

  • geohash_grid: 根据Geohash的单元格(cell)聚合

  • geo_bounds: 根据区域聚合

4.1 geo_distance聚合

下面这个查询根据距离厦门的距离来聚合,返回0-500,500-8000km的聚合:

{
    "query":{
        "filtered":{
            "filter":{
                "geo_distance" : {
                    "distance" : "10000km",
                    "location" : {
                        "lat" : 24.46667,
                        "lon" : 118.10000
                    }                }           }        }    },
    "aggs":{
        "per_ring":{
            "geo_distance":{
                "field": "location",
                "unit":  "km",
                "origin":{
                    "lat" : 24.46667,
                    "lon" : 118.10000
                },
                "ranges":[
                    {"from": 0 , "to":500},
                    {"from": 500 , "to":8000}
                ]            }        }    }}12345678910111213141516171819202122232425262728293031323334

返回的聚合结果如下;

"aggregations": {
    "per_ring": {
      "buckets": [
        {
          "key": "*-500.0",
          "from": 0,
          "from_as_string": "0.0",
          "to": 500,
          "to_as_string": "500.0",
          "doc_count": 2
        },
        {
          "key": "500.0-8000.0",
          "from": 500,
          "from_as_string": "500.0",
          "to": 8000,
          "to_as_string": "8000.0",
          "doc_count": 3
        }
      ]    }  }12345678910111213141516171819202122

可以看到,距离厦门0-500km的城市有2个,500-8000km的有3个。

4.2 geohash_grid聚合

该聚合方式根据geo_point数据对应的geohash值所在的cell进行聚合,cell的划分精度通过precision属性来控制,精度是指cell划分的次数。

{
    "query":{
        "filtered":{
            "filter":{
                "geo_distance" : {
                    "distance" : "10000km",
                    "location" : {
                        "lat" : 24.46667,
                        "lon" : 118.10000
                    }                }           }        }    },
    "aggs":{
        "grid_agg":{
            "geohash_grid":{
                "field": "location",
                "precision":  2
            }        }    }}1234567891011121314151617181920212223242526

聚合结果如下:

"aggregations": {
    "grid_agg": {
      "buckets": [
        {
          "key": "ws",
          "doc_count": 3
        },
        {
          "key": "wx",
          "doc_count": 1
        },
        {
          "key": "ww",
          "doc_count": 1
        }
      ]    }  }123456789101112131415161718

可以看到,有3个城市的的geohash值为ws。将精度提高到5,聚合结果如下:

"aggregations": {
    "grid_agg": {
      "buckets": [
        {
          "key": "wx4g1",
          "doc_count": 1
        },
        {
          "key": "wwnk7",
          "doc_count": 1
        },
        {
          "key": "wssu6",
          "doc_count": 1
        },
        {
          "key": "ws7gp",
          "doc_count": 1
        },
        {
          "key": "ws0eb",
          "doc_count": 1
        }
      ]    }  }1234567891011121314151617181920212223242526

4.3 geo_bounds聚合

这个聚合操作计算能够覆盖所有查询结果中geo_point的最小区域,返回的是覆盖所有位置的最小矩形:

{
    "query":{
        "filtered":{
            "filter":{
                "geo_distance" : {
                    "distance" : "10000km",
                    "location" : {
                        "lat" : 24.46667,
                        "lon" : 118.10000
                    }                }           }        }    },
    "aggs":{
        "map-zoom":{
            "geo_bounds":{
                "field": "location"
            }        }    }}123456789101112131415161718192021222324

结果如下:

 "aggregations": {
    "map-zoom": {
      "bounds": {
        "top_left": {
          "lat": 39.91666993126273,
          "lon": 113.2333298586309
        },
        "bottom_right": {
          "lat": 23.16666992381215,
          "lon": 121.43332997336984
        }      }    }  }1234567891011121314

也就是说,这两个点构成的矩形能够包含所有到厦门距离10000km的区域。我们把距离调整为500km,此时覆盖这些城市的矩形如下:

"aggregations": {
    "map-zoom": {
      "bounds": {
        "top_left": {
          "lat": 26.083329990506172,
          "lon": 118.0999999679625
        },
        "bottom_right": {
          "lat": 24.46666999720037,
          "lon": 119.29999999701977
        }      }    }  }

Java 客户端代码:

elasticsearch 5.2.2:

/**
 * 
 * @author Tony
 *
 */
public class GEOTest {

	public static void main(String[] args) {
		 String index = "es";
		 String type = "people";
		 //init(index, type);
		 
		 double lat1 = 42;
		 double lon1 = -72;
		 double lat2 = 40;
		 double lon2 =  -74;
		 testGetNearbyPeople(index, type, lat1, lon1,lat2,lon2);
		 
		 testGetNearbyPeople2( index, type, 40, -70);
		 
		 System.out.println("---多边形查询 ----");
		 testPolygonQuery(index, type);
		 
		 System.out.println("---环形查询 ----");
		 testDistanceRangeQuery(index, type);

		/*double lat = 39.999517;
		double lon = 116.465176;
		long start = System.currentTimeMillis();
		// query("郭", index, type);
		testGetNearbyPeople2( index, type, lat, lon);
		long end = System.currentTimeMillis();
		System.out.println((end - start) + "毫秒");*/
	}

	/**
	 * 查询坐标1到坐标2矩形范围内,的用户有那些
	 * 
	 * @param indexName
	 * @param indexType
	 * @param lat
	 * @param lon
	 */
	public static void testGetNearbyPeople(String indexName, String indexType, double top, double left, double bottom,
			double right) {

		Client client = ESUtils.getClient();

		SearchRequestBuilder srb = client.prepareSearch(indexName).setTypes(indexType);

		SearchResponse searchResponse = srb
				.setQuery(QueryBuilders.geoBoundingBoxQuery("location").setCorners(top, left, bottom, right)).get();

		for (SearchHit searchHit : searchResponse.getHits().getHits()) {
			System.out.println(searchHit.getSourceAsString());
		}

		System.out.println("-----");
	}

	/**
	 * 获取坐标 附近1000米的用户信息
	 * 
	 * @param indexName
	 * @param indexType
	 * @param lat 纬度
	 * @param lon 经度
	 */
	public static void testGetNearbyPeople2(String indexName, String indexType, double lat, double lon) {

		Client client = ESUtils.getClient();

		SearchRequestBuilder srb = client.prepareSearch(indexName).setTypes(indexType);
		srb.setFrom(0).setSize(1000);// 1000人
		// lon, lat位于谦的坐标,查询距离于谦1米到1000米

		GeoDistanceQueryBuilder location1 = QueryBuilders.geoDistanceQuery("location").point(lat, lon).distance(1000,
				DistanceUnit.MILES);
		srb.setPostFilter(location1);
		
		//srb.setQuery(QueryBuilders.boolQuery().filter(location1));

		// 获取距离多少公里 这个才是获取点与点之间的距离的
		GeoDistanceSortBuilder sort = SortBuilders.geoDistanceSort("location", lat, lon);
		sort.unit(DistanceUnit.MILES);
		sort.order(SortOrder.ASC);
		sort.point(lat, lon);
		srb.addSort(sort);

		
		SearchResponse searchResponse = srb.execute().actionGet();
		
	   // System.out.println(srb.toString());

		SearchHits hits = searchResponse.getHits();
		SearchHit[] searchHists = hits.getHits();
		// 搜索耗时
		Float usetime = searchResponse.getTookInMillis() / 1000f;
		System.out.println("坐标附近的人(" + hits.getTotalHits() + "个),耗时(" + usetime + "秒):");
		for (SearchHit hit : searchHists) {
			String name = (String) hit.getSource().get("name");
			List<Double> location = (List<Double>) hit.getSource().get("location");
			// 获取距离值,并保留两位小数点
			BigDecimal geoDis = new BigDecimal((Double) hit.getSortValues()[0]);
			Map<String, Object> hitMap = hit.getSource();
			// 在创建MAPPING的时候,属性名的不可为geoDistance。
			hitMap.put("geoDistance", geoDis.setScale(0, BigDecimal.ROUND_HALF_DOWN));
			System.out.println(name + "的坐标:" + location + "他距离坐标" + hit.getSource().get("geoDistance")
					+ DistanceUnit.METERS.toString());
		}

	}
	
   /**
	    * 但是5系列好像不支持这个功能了,提示:
[geo_distance_range] queries are no longersupported for geo_point field types. Use geo_distance sort or aggregations

     * 环形查询
     */
    public static void testDistanceRangeQuery(String indexName,String indexType) {
    	
    	  
    
        GeoDistanceRangeQueryBuilder geoDistanceRangeQuery = QueryBuilders
        		.geoDistanceRangeQuery("location",new GeoPoint(42, -72))
                .from("0m")
                .to("25m")
                .includeLower(true)
                .includeUpper(true)
                .optimizeBbox("memory")
                .geoDistance(GeoDistance.SLOPPY_ARC);
        
        GeoDistanceSortBuilder sort = new GeoDistanceSortBuilder("location",new GeoPoint(42, -72));  
        sort.unit(DistanceUnit.KILOMETERS);//距离单位公里  
        sort.order(SortOrder.ASC);  
  
       
        
        SearchRequestBuilder search = ESUtils.getClient().prepareSearch(indexName).setTypes(indexType)
                .setSearchType(SearchType.DFS_QUERY_AND_FETCH)
                .setQuery(geoDistanceRangeQuery);
                //.addAggregation(AggregationBuilders.filter("agg_by_distance_range", geoDistanceRangeQuery));
        
        System.out.println(search.toString());
        
        SearchResponse response = search.execute().actionGet();
        System.out.println(response);
        System.out.println(response.getHits().totalHits());
    }
    
	
	   /**
     * 多边形查询
     */
    public static void testPolygonQuery(String indexName, String indexType) {
    	Client client = ESUtils.getClient();
    	List<GeoPoint> points=new ArrayList<GeoPoint>();
    	points.add(new GeoPoint(42, -72));
    	points.add(new GeoPoint(39, 117));
    	points.add(new GeoPoint(40, 117));
    	
    	SearchResponse response = client.prepareSearch(indexName).setTypes(indexType)
    			  .setQuery(QueryBuilders.geoPolygonQuery("location",points))
    			  .get();
    	
        System.out.println(response);
        System.err.println(response.getHits().totalHits());
    }
    
    

	private static void init(String indexName, String indexType) {
		//
		try {
		    createIndex(indexName, indexType);
			addIndexData100000(indexName, indexType);
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

	// 创建索引
	public static void createIndex(String indexName, String indexType) throws IOException {
		Client client = ESUtils.getClient();
		// 创建Mapping
		XContentBuilder mapping = createMapping(indexType);
		System.out.println("mapping:" + mapping.string());

		client.admin().indices().prepareCreate(indexName).execute().actionGet();
		PutMappingRequest putMapping = Requests.putMappingRequest(indexName).type(indexType).source(mapping);
		PutMappingResponse response = client.admin().indices().putMapping(putMapping).actionGet();
		if (!response.isAcknowledged()) {
			System.out.println("Could not define mapping for type [" + indexName + "]/[" + indexType + "].");
		} else {
			System.out.println("Mapping definition for [" + indexName + "]/[" + indexType + "] succesfully created.");
		}
	}

	// 创建mapping
	public static XContentBuilder createMapping(String indexType) {
		XContentBuilder mapping = null;
		try {
			mapping = XContentFactory.jsonBuilder().startObject()
					// 索引库名(类似数据库中的表)
					.startObject(indexType).startObject("properties")
					// ID
					.startObject("id").field("type", "long").endObject()
					// 姓名
					.startObject("name").field("type", "string").endObject()
					// 位置
					.startObject("location").field("type", "geo_point").endObject().endObject().endObject().endObject();
		} catch (IOException e) {
			e.printStackTrace();
		}
		return mapping;
	}

	public static Integer addIndexData100000(String indexName, String indexType) {

		Client client = ESUtils.getClient();
		List<String> cityList = new ArrayList<String>();

	/*	double lat = 39.929986;
	    double lon = 116.395645;
		
		
	   for (int i = 0; i < 10; i++) {
		double max = 0.00001;
		double min = 0.000001;
		Random random = new Random();
		double s = random.nextDouble() % (max - min + 1) + max;
		DecimalFormat df = new DecimalFormat("######0.000000");
		// System.out.println(s);
		String lons = df.format(s + lon);
		String lats = df.format(s + lat);
		Double dlon = Double.valueOf(lons);
		Double dlat = Double.valueOf(lats);
		User city1 = new User(i, "Tony=" + i, dlat, dlon);
		cityList.add(obj2JsonUserData(city1));
	   }*/

		User city1 = new User(1, "喜来登大酒店",41.12, -71.34);
		cityList.add(obj2JsonUserData(city1));
		
		User city2 = new User(2, "喜来登大酒店B", 42.12, -71.34);
		cityList.add(obj2JsonUserData(city2));
		
		
		// 创建索引库
		BulkRequestBuilder bulkRequest = client.prepareBulk();
		for (int i = 0; i < cityList.size(); i++) {
			bulkRequest.add(client.prepareIndex(indexName, indexType).setSource(cityList.get(i)));
		}
		BulkResponse bulkResponse = bulkRequest.execute().actionGet();
		if (bulkResponse.hasFailures()) {
			System.out.println(bulkResponse.buildFailureMessage());
			System.out.println("批量创建索引错误!");
		}

		return bulkRequest.numberOfActions();
	}

	public static String obj2JsonUserData(User user) {
		/*String jsonData = null;
		try {
			// 使用XContentBuilder创建json数据
			XContentBuilder jsonBuild = XContentFactory.jsonBuilder();
			jsonBuild.startObject()
			       .startObject("pin")
			         .startObject("location")
			         .field("lat", user.getLat())
					.field("lon", user.getLon())
					.endObject()
					.field("id", user.getId()).field("name", user.getName())
					.endObject().endObject();
			jsonData = jsonBuild.string();
			System.out.println(jsonData);
		} catch (IOException e) {
			e.printStackTrace();
		}
		return jsonData;*/
		
		  String jsonData = null;
	        try {
	            // 使用XContentBuilder创建json数据
	            XContentBuilder jsonBuild = XContentFactory.jsonBuilder();
	            jsonBuild.startObject().field("id", user.getId()).field("name", user.getName()).startArray("location").value(user.getLon()).value(user.getLat()).endArray()
	                    .endObject();
	            jsonData = jsonBuild.string();
	            System.out.println(jsonData);
	        } catch (IOException e) {
	            e.printStackTrace();
	        }
	        return jsonData;
	        
	}

}