In this post, I show how Elasticsearch and Geolocation improves search results and user experience. I assume the reader understands the basic concepts of Elasticsearch and I won’t delve too deep in these.
Geospatial abilities of Elasticsearch
Geographic data in Elasticsearch takes two forms – geo-point and geo-shape.
Geo-point represents geographic data in the form of a latitude and longitude pair. With this we can calculate the distance between two coordinates, filter data on the basis of distance, find results within bounding boxes, and/or aggregate data using aggregations.
Geo-shape is used when data represents a shape and takes the form of GeoJSON format. GeoJSON converts strings of latitude and longitude coordinates on a grid of Geohash cells. This type of representation can be used to fetch data within the shape using queries such as “intersects, disjoint, contains, within” etc.
Unfortunately, Geo-point and Geo-shape cannot be a part of the same query at the same time. Hence it is important to know the existing data type and query mechanisms prior to indexing to avoid remapping and reindexing your data.
Typical use case
In this post I will cover the Geo-point representation of geographic data. I’m a foodie and a traveller, and in this case study I will show how Elasticsearch capability is used in a hotel booking website.
My search abilities should include:
- Search hotel names by partially or completely typing the text (Autocomplete names)
- Hotels within a given distance sorted by the nearest
- Properties within a budget and sorted by nearest distance from the user
- Hotels within a bounding-box sorted by nearest
- Show the number of hotels near to me within specific distance ranges such as within 10km, slightly longer (10 to 40km) and far away (more than 40km)
Implementation
Create an index
PUT property
{
"settings": {
"analysis": {
"filter": {
"partialFilter": {
"type": "edge_ngram",
"min_gram": 2,
"max_gram": 15
}
},
"analyzer": {
"stem": {
"tokenizer": "standard",
"filter": [
"standard",
"lowercase",
"stop"
]
},
"exactWithLowerCase": {
"tokenizer": "keyword",
"filter": "lowercase"
},
"lowerCaseWithSeparator": {
"tokenizer": "whitespace",
"filter": "lowercase"
},
"partial": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"partialFilter"
]
}
}
}
},
"mappings": {
"hotel": {
"dynamic": false,
"properties": {
"description": {
"type": "text",
"analyzer": "stem",
"fields": {
"separateExact": {
"type": "text",
"analyzer": "lowerCaseWithSeparator"
}
}
},
"location": {
"type": "geo_point"
},
"averagePrice": {
"type": "float"
},
"name": {
"type": "text",
"fields": {
"partial": {
"type": "text",
"analyzer": "partial"
},
"precise": {
"type": "text",
"analyzer": "stem"
},
"exact": {
"type": "text",
"analyzer": "exactWithLowerCase"
},
"separateExact": {
"type": "text",
"analyzer": "lowerCaseWithSeparator"
}
}
},
"address": {
"type": "text",
"analyzer": "lowerCaseWithSeparator"
},
"amenities": {
"type": "text",
"analyzer": "lowerCaseWithSeparator"
},
"contactNumber": {
"type": "integer"
}
}
}
}
}
Here, I’ve created an Index “property” with type “hotel”. Take a closer look at how Filter and Analysers are added, especially “partialFilter” and “partial” analyser. This filter and analyzer will be required for partial searches.
Test Analyse
Test whether the “partial” analyser correctly tokenizes the text.
GET property/_analyze
{
"analyzer": "partial",
"text": "Text to analyze"
}
If it does, it should show the following output:
{
"tokens": [
{
"token": "te",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "tex",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "text",
"start_offset": 0,
"end_offset": 4,
"type": "<ALPHANUM>",
"position": 0
},
{
"token": "to",
"start_offset": 5,
"end_offset": 7,
"type": "<ALPHANUM>",
"position": 1
},
{
"token": "an",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "ana",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "anal",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "analy",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "analyz",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
},
{
"token": "analyze",
"start_offset": 8,
"end_offset": 15,
"type": "<ALPHANUM>",
"position": 2
}
]
}
Now let’s add a few documents to the Index:
POST /property/hotel
{
"name": " Ibis Pune Hinjewadi",
"location": " 18.597514, 73.716896",
"averagePrice": 8000,
"description": "Offering quality accommodations in the business district of Pune, ibis Pune Hinjewadi is a popular pick for both business and leisure travelers. From here, guests can make the most of all that the lively city has to offer. With its convenient location, the property offers easy access to the city's must-see destinations.",
"address": "Pune Infotech Park Hinjewadi Phase II, Maharashtra, Hinjewadi, Pune, Maharashtra, 411057, India",
"amenities": "Restaurant and bar/lounge, Breakfast available, Fitness center, 24-hour front desk, Air conditioning, Daily housekeeping, Computer station, Laundry service, Conference space, Free newspapers in the lobby, Free WiFi and free parking"
}
Note: large volumes of data is added to Elasticsearch using the bulk API.
Let’s add a few more documents in bulk here:
POST /property/hotel/_bulk
{ "index":{} }
{ "name": "Vivanta By Taj Blue Diamond", "location": "18.538266, 73.886455", "averagePrice": 5000 }
{ "index":{} }
{ "name":"Gordon House", "location": "18.538183, 73.835511", "averagePrice": 7000 }
{ "index":{} }
{ "name": "Hilton Shillim Estate Retreat & Spa Pune", "location": "18.625181, 73.450640", "averagePrice": 6000 }
{ "index":{} }
{ "name":"Double Tree By Hilton","location": "18.642451, 73.798544", "averagePrice": 4000 }
{ "index":{} }
{ "name": "Four Points By Sheraton", "location": "18.563183, 73.922255", "averagePrice": 7000 }
{ "index":{} }
{ "name":"JW Marriott Hotel Pune","location": "18.532667, 73.829842", "averagePrice": 4000 }
{ "index":{} }
{ "name": "Hyatt Regency Pune", "location": "18.560416, 73.912121", "averagePrice": 5000 }
{ "index":{} }
{ "name":"Hotel Pride","location": "18.531543, 73.847224", "averagePrice": 8000 }
{ "index":{} }
{ "name": "Hotel Woodland", "location": "18.525239, 73.875434", "averagePrice": 5000 }
{ "index":{} }
{ "name":"Sheraton Grand Pune Bund Garden Hotel","location": "18.530246, 73.871373", "averagePrice": 9000 }
{ "index":{} }
{ "name": "Hotel Blue Diamond", "location": "18.539014, 73.886293", "averagePrice": 3000 }
{ "index":{} }
{ "name":"Hotel Aurora Towers","location": "18.518660, 73.878899", "averagePrice": 5000 }
{ "index":{} }
{ "name": "Green Gate Resort", "location": "18.475221, 73.470123", "averagePrice": 7000 }
{ "index":{} }
{ "name":"Hotel Central Park","location": "18.520194, 73.844651", "averagePrice": 8000 }
{ "index":{} }
{ "name": "Hotel Kohinoor Executive", "location": "18.518343, 73.843101", "averagePrice": 1000 }
{ "index":{} }
{ "name":"Valley City Hotel","location": "18.623144, 73.397000", "averagePrice": 6000 }
{ "index":{} }
{ "name": "Avion Holiday Resort", "location": "18.754184, 73.404417", "averagePrice": 7000 }
{ "index":{} }
{ "name":"The Lagoona Resort","location": "18.771987, 73.405758", "averagePrice": 9000 }
{ "index":{} }
{ "name": "Hotel Neat Space", "location": "18.759003, 73.393159", "averagePrice": 2000 }
{ "index":{} }
{ "name":"Novotel Imagica Khopoli","location": "18.770219, 73.282127", "averagePrice": 10000 }
{ "index":{} }
{ "name": "Bella Vista Resort", "location": "17.924749, 73.694311", "averagePrice": 9000 }
{ "index":{} }
{ "name":"Evershine Keys Prima Resort", "location": "17.923065, 73.667456", "averagePrice": 4000 }
{ "index":{} }
{ "name": "Le Méridien Mahabaleshwar Resort & Spa", "location": "17.916902, 73.673692", "averagePrice": 12000 }
Search hotel names by partially or completely typing the text (Autocomplete names)
Partial Search
I wish to get all hotels whose name contains “Resort” and also see how far away are these places from my current location. My current location is “Synerzip Softech Pune, India”.
Geo-coordinates for my current location are “18.520760, 73.855410”.I can do this using this query:
GET /property/hotel/_search
{
"_source": "name",
"query": {
"bool": {
"must": [
{
"query_string": {
"default_field": "name.partial",
"query": "resort"
}
}
]
}
},
"sort": [
{
"_geo_distance": {
"location": "18.520760, 73.855410",
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}
In the above query, I’m searching for the text “Resort” in the “name.partial” field. We have created the “name.partial” field in such a way that it will be queried for partial searches.
We also wish to know how far these places are from my current location. To get that we have added “sort” in our query which will accept the geo-coordinates, the location from where distance is to be calculated.
Here, we can also specify the “Order” like “asc/desc”; “Unit” like “kilometers, meters, miles, yards, feet, inch, centimeter, millimeter, nautical miles”; “distance_type” such as “arc/plane”.
The official documentation contains more details about each parameter. This query will return documents like:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": null,
"hits": [
{
"_index": "property",
"_type": "hotel",
"_id": "_wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Green Gate Resort"
},
"sort": [
40.942914457583306
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "AwpeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Avion Holiday Resort"
},
"sort": [
54.14506029675922
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "BApeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "The Lagoona Resort"
},
"sort": [
54.997508021889125
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "BwpeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Bella Vista Resort"
},
"sort": [
68.42283078328954
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "CApeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Evershine Keys Prima Resort"
},
"sort": [
69.36215351306305
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "CQpeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Le Méridien Mahabaleshwar Resort & Spa"
},
"sort": [
69.83527119425608
]
}
]
}
}
Complete Name Search
I want to search for a hotel with the name “Hyatt Regency Pune”.This is the query you can fire for this:
{
"_source": "name",
"query": {
"bool": {
"must": [
{
"query_string": {
"default_field": "name.exact",
"query": "Hyatt Regency Pune"
}
}
]
}
},
"sort": [
{
"_geo_distance": {
"location": "18.520760, 73.855410",
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}
…and this is the result that you should get,
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": null,
"hits": [
{
"_index": "property",
"_type": "hotel",
"_id": "-QpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hyatt Regency Pune"
},
"sort": [
7.428923146415164
]
}
]
}
}
Hotels within a given distance sorted by the nearest
I want to see all the hotels which fall under 10km radius from my location and sorted with nearest first.Here is the query you can fire:
GET /property/hotel/_search
{
"size": 100,
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "10km",
"location" :
"18.520760, 73.855410"
}
}
}
},
"sort": [
{
"_geo_distance": {
"location" : "18.520760, 73.855410",
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}
Here is the result you should get,
{
"took": 20,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 12,
"max_score": null,
"hits": [
{
"_index": "property",
"_type": "hotel",
"_id": "AApeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Hotel Central Park",
"location": "18.520194, 73.844651",
"averagePrice": 8000
},
"sort": [
1.1361415903549301
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "AQpeiWsBUrsSGieBtg_c",
"_score": null,
"_source": {
"name": "Hotel Kohinoor Executive",
"location": "18.518343, 73.843101",
"averagePrice": 1000
},
"sort": [
1.3253640975360597
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-gpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Pride",
"location": "18.531543, 73.847224",
"averagePrice": 8000
},
"sort": [
1.477343312684666
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "_ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Sheraton Grand Pune Bund Garden Hotel",
"location": "18.530246, 73.871373",
"averagePrice": 9000
},
"sort": [
1.98624146374245
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Woodland",
"location": "18.525239, 73.875434",
"averagePrice": 5000
},
"sort": [
2.1691714960247315
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "_gpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Aurora Towers",
"location": "18.518660, 73.878899",
"averagePrice": 5000
},
"sort": [
2.487585015180662
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "9ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Gordon House",
"location": "18.538183, 73.835511",
"averagePrice": 7000
},
"sort": [
2.8556625672143316
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "JW Marriott Hotel Pune",
"location": "18.532667, 73.829842",
"averagePrice": 4000
},
"sort": [
3.0032955674420423
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "8wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Vivanta By Taj Blue Diamond",
"location": "18.538266, 73.886455",
"averagePrice": 5000
},
"sort": [
3.8081926331768505
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "_QpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Blue Diamond",
"location": "18.539014, 73.886293",
"averagePrice": 3000
},
"sort": [
3.8368582326373644
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-QpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hyatt Regency Pune",
"location": "18.560416, 73.912121",
"averagePrice": 5000
},
"sort": [
7.428923146415164
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "9wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Four Points By Sheraton",
"location": "18.563183, 73.922255",
"averagePrice": 7000
},
"sort": [
8.480118430819688
]
}
]
}
}
Hotels by providing a Budget and the Distance with sorted by nearest first
I want to get all hotels where the per night price is between INR 2000 – 5000 and within a 10km radius of my location. We now need a combination filter as shown below:
GET /property/hotel/_search
{
"size": 100,
"query": {
"bool": {
"must": {
"range": {
"averagePrice": {
"from": 2000,
"to": 5000
}
}
},
"filter": {
"geo_distance": {
"distance": "10km",
"location": "18.520760, 73.855410"
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": "18.520760, 73.855410",
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}
This will give a result as,
{
"took": 24,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 6,
"max_score": null,
"hits": [
{
"_index": "property",
"_type": "hotel",
"_id": "-wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Woodland",
"location": "18.525239, 73.875434",
"averagePrice": 5000
},
"sort": [
2.1691714960247315
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "_gpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Aurora Towers",
"location": "18.518660, 73.878899",
"averagePrice": 5000
},
"sort": [
2.487585015180662
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "JW Marriott Hotel Pune",
"location": "18.532667, 73.829842",
"averagePrice": 4000
},
"sort": [
3.0032955674420423
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "8wpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Vivanta By Taj Blue Diamond",
"location": "18.538266, 73.886455",
"averagePrice": 5000
},
"sort": [
3.8081926331768505
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "_QpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Blue Diamond",
"location": "18.539014, 73.886293",
"averagePrice": 3000
},
"sort": [
3.8368582326373644
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-QpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hyatt Regency Pune",
"location": "18.560416, 73.912121",
"averagePrice": 5000
},
"sort": [
7.428923146415164
]
}
]
}
}
Hotels within a bounding-box sorted by nearest first
Suppose I want to find hotels within a box.
I should know geo-coordinates for the upper left corner and lower right corner. Here is the query you can fire,
GET /property/hotel/_search
{
"_source": "name",
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_bounding_box": {
"location": {
"top_left": {
"lat": 18.625181,
"lon": 73.45064
},
"bottom_right": {
"lat": 18.52076,
"lon": 73.85541
}
}
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": "18.520760, 73.855410",
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}
Here is the result you should get,
{
"took": 82,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 4,
"max_score": null,
"hits": [
{
"_index": "property",
"_type": "hotel",
"_id": "-gpeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Hotel Pride"
},
"sort": [
1.477343312684666
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "9ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "Gordon House"
},
"sort": [
2.8556625672143316
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "-ApeiWsBUrsSGieBtg7c",
"_score": null,
"_source": {
"name": "JW Marriott Hotel Pune"
},
"sort": [
3.0032955674420423
]
},
{
"_index": "property",
"_type": "hotel",
"_id": "RYPWu2sB8wWj1PZYoJ1J",
"_score": null,
"_source": {
"name": " Ibis Pune Hinjewadi"
},
"sort": [
16.912499258116554
]
}
]
}
}
Show me the number of hotels near me (within 10km), slightly far (10-40km) and far away (more than 40km)
Here is the query that you should fire,
POST /property/hotel/_search
{
"size": 0,
"aggs": {
"distanceRanges": {
"geo_distance": {
"field": "location",
"origin": "18.520760, 73.855410",
"unit": "km",
"ranges": [
{
"key": "Near By Locations",
"to": 20
},
{
"key": "Medium Distance Locations",
"from": 20,
"to": 50
},
{
"key": "Far Away Locations",
"from": 50
}
]
}
}
}
}
Here is the result that you should get,
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 24,
"max_score": 0,
"hits": []
},
"aggregations": {
"distanceRanges": {
"buckets": [
{
"key": "Near By Locations",
"from": 0,
"to": 20,
"doc_count": 14
},
{
"key": "Medium Distance Locations",
"from": 20,
"to": 50,
"doc_count": 3
},
{
"key": "Far Away Locations",
"from": 50,
"doc_count": 7
}
]
}
}
}
These are some free ready-to-use code snippets that you can directly use in your application. I have covered a few search parameters here and you can obviously change and use it as per your own needs.
Conclusion
This article intends to show how geospatial capabilities are helpful to provide appropriate results to the user. I have taken very basic examples to make you familiar with its capabilities, however real-life cases can be a little tricky. However, Elasticsearch documentation is very detailed and the community is very helpful.If you enjoyed reading this post, here are a few more that may interest you:
