Blog Detail

preview image Programming
by Anurag Srivastava, Aug 4, 2018, 7:02:21 AM | 4 minutes

Basics of Data Search in Elasticsearch

Day by day the volume of data is increasing as we are moving towards the digital age and transforming things into Internet of things. Take a simple example of smart watch what is does, it measures the steps, BP, heart rate etc and pushing to a server from where we can access our heath related metrics. In the same way there are different smart devices which keep on sending the regular data which is stored on a server. We are dumping lots and lots of data on servers which is there to help us find a trend, analyze them through data science to solve some serious problems or apply machine learning algorithms to forecast the future trend.

So in a brief way I have explained how data is important and we should capture them regularly to get the meaningful information out of it. Now the question arrives how. What is best way to search the data as traditionally we were soring data in a RDBMS system and use to fetch them by directly applying the SQL queries but now things have changed because we want quick search response. We have no time to wait for a search result by seeing the loading icon moving in a direction. Another issue which we were facing was the uncertainty of data format and for these types of data we had a bottleneck in RDBMS system.

Now move to the search part as this blog is here introduce you about the basic Elasticsearch query construction so that a novice person can install, index and search data from an Elasticsearch cluster. These days Elasticsearch is primarily used for its search capabilities and the ELK stack which can be applied to any set of application to boost the performance and monitoring capabilities.  So lets start the process and learn the basic search query construction in Elasticsearch.

We have basically two types of search APIs in Elasticsearch: 'request URI based' and 'request body based'.  In REST request URI we use to pass the search criteria in the URL itself like:

GET /blogs/technical/_search?q=topic:kibana

In REST request body we use to construct the search block and write the search query inside the query block of Elasticsearch like:

GET /blogs/technical/_search
{
  "query": {
    "term": {
      "topic":"kibana"     
    }
  }
}

So the URI based search is quite basic search where we just want to search a keyword whereas in request body we can construct the complex queries. So we have the query language to work with request body based searches. In this blog I am not going into details to keep it simple so that everyone can understand what is going on.

Fielddata is disabled on text fields by default in Elasticsearch so we need to enable it for constructing the queries.

PUT blogs/_mapping/technical?update_all_types
{
  "properties": {
    "topic": {
      "type": "text",
      "fielddata": true
    }
  }
}

Match All:

Now lets understand the basics of query language, where first comes the match_all query:

GET /blogs/technical/_search
{
  "query": { "match_all": {} }
}

In match_all query Elasticsearch returns all the documents. So this Elasticsearch query is basically like SQL "select * from technical" query.

Limit:

Now we are going to set the offset and limit in a query to restrict the records, like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "from": 1,
  "size": 5
}

In the above query, I am fetching 5 documents starting from th second one. In the same way we can set the offset and limit in any Elasticsearch query.

Sort:

In Elasticsearch we can sort the documents as per our requirement like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "sort": { "topic": { "order": "desc" } }

}

In the above expression, we are applying the ordering on the field topic.

Field Selection:

We limit the number of columns in SQL select queries, in the same way we can do it in Elasticsearch queries, like:

GET /blogs/technical/_search
{
  "query": { "match_all": {} },
  "_source": ["category"]
}

In the above query we will only get the category field in search result and topic filed would not be displayed.

Match Queries:

We can run the match queries against the field name, like:

GET /blogs/technical/_search
{
  "query": {
    "match": {
      "topic": "kibana"
    }
  }
}

In the above query we can pass the text to search against topic field.

In this blog I have explained the basics of Elasticsearch query construction. I next blog I will cover filters, boolean queries, wild-card queries etc and then will explain aggregation and its usage.


Other Blogs on Elasticsearch:

Introduction to Elasticsearch
Elasticsearch Installation and Configuration on Ubuntu 14.04
Log analysis with Elastic stack
Elasticsearch Rest API
Wildcard and Boolean Search in Elasticsearch
Configure Logstash to push MySQL data into Elasticsearch


In case of any confusions please leave your comment, also give your suggestions to improve the blogs.




About Author

Anurag Srivastava

Author | Blogger | Tech Lead | Data Scientist | Innovator |

View Profile

Comments (0)

Leave a comment

Related Blogs

Elasticsearch Rest API

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava

Introduction to Elasticsearch Aggregations

Aug 14, 2018, 4:47:56 PM | Anurag Srivastava

Metrics Aggregation in Elasticsearch

Aug 18, 2018, 6:02:20 PM | Anurag Srivastava

Bucket Aggregation in Elasticsearch

Aug 29, 2018, 7:15:06 PM | Anurag Srivastava

Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

Create word cloud in Python

Jun 30, 2018, 6:06:45 AM | Anurag Srivastava

Typecasting in PHP for short datatype

Jun 8, 2018, 8:03:52 AM | Lovish Sharma

Built-in PHP Arrays

Feb 20, 2018, 7:11:44 PM | Anurag Srivastava

Top Blogs

Build and deploy Angular code using Python

Jun 26, 2018, 4:50:18 PM | Anurag Srivastava

Configure SonarQube Scanner with Jenkins

Jun 21, 2018, 4:58:11 AM | Anurag Srivastava

Configure Logstash to push MySQL data into Elasticsearch

Jul 7, 2018, 8:51:30 AM | Anurag Srivastava

Configure Jenkins for Automated Code Deployment

Jun 13, 2018, 3:44:01 PM | Anurag Srivastava

SonarQube installation on Ubuntu

May 12, 2018, 4:47:07 PM | Anurag Srivastava

Execute Commands on Remote Machines using sshpass

Jul 16, 2018, 5:00:02 PM | Anurag Srivastava

Why SonarQube is important for IT projects ?

Apr 24, 2018, 2:52:28 PM | Anurag Srivastava

Install Jenkins on Ubuntu

May 26, 2018, 6:42:02 PM | Anurag Srivastava

Analyze your project with SonarQube

Jun 2, 2018, 10:49:54 AM | Anurag Srivastava

Install Kafka on Ubuntu

Jul 12, 2018, 7:40:51 PM | Anurag Srivastava