Blog Detail

preview image Programming
by Anurag Srivastava, Aug 18, 2018, 6:02:20 PM | 4 minutes

Metrics Aggregation in Elasticsearch

In my previous blog, I have explained about basic aggregation. Now, let us pick the metrics aggregation and see how we can create these types of aggregations. Metrics aggregation are those aggregation where we apply different types of metrics on fields of Elasticsearch documents like min, max, avg, top, and stats etc.

Max:

I am going to explain you the max metrics using which we can get the max value for the given field. In the same way, we can use min,avg, and top etc. See below example

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "max" : {
        "field" : "views"
      }
    }
  }
}

In the above expression, I am trying to fetch the max number of views from all documents. After running this expression you may get the following result:

{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "value": 6309
    }
  }
}

In the result, you can see that the value for blog_metrics is showing as 6309. In the query, we can pass as min or avg to get minimum or the average number of views.

Stats:

Stats provides us the consolidated stats metrics for a given field. Take another example where I am going to use stats instead of max:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "stats" : {
        "field" : "views"
      }
    }
  }
}

In the above expression, I have replaced "max" keyword with "stats" rest all expression will remain the same. Now see the result of above expression.

{
  "took": 22,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "count": 54,
      "min": 11,
      "max": 6309,
      "avg": 142.85185185185185,
      "sum": 7714
    }
  }
}

When I applied the "stats" keywords the aggregation result displayed all key stats like count, min, max, avg, and the sum of the given field value. Stats can be used when we want to see the data trend and want to know the min value, max value, sum, count and the average value of a field.

Extended Stats:
We can use extended_stats to get extended stats like sum_of_squares, variance, std deviation etc. See below example:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "extended_stats" : {
        "field" : "views"
      }
    }
  }
}

After executing the above parameter we would get below response:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 54,
    "max_score": 0,
    "hits": []
  },
  "aggregations": {
    "blog_metrics": {
      "count": 54,
      "min": 11,
      "max": 1309,
      "avg": 142.85185185185185,
      "sum": 7714,
      "sum_of_squares": 3775934,
      "variance": 49518.05212620027,
      "std_deviation": 222.5265200514318,
      "std_deviation_bounds": {
        "upper": 587.9048919547154,
        "lower": -302.20118825101173
      }
    }
  }
}

In this way, we can fetch these important stats for any field and can get complete insight about the variation in the field value.

Percentile:
Percentile is again a type of metrics aggregation which shows the certain percentage of observed field value at different points. See below example:

GET bqstack/_search?size=0
{
  "aggs": {
    "blog_metrics" : {
      "percentiles" : {
        "field" : "views"
      }
    }
  }
}

After running above expression we should get the following response:

{
"took": 17,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 54,
"max_score": 0,
"hits": []
},
"aggregations": {
"blog_metrics": {
"values": {
"1.0": 11,
"5.0": 12.2,
"25.0": 31,
"50.0": 50.5,
"75.0": 148,
"95.0": 563.7999999999998,
"99.0": 1288.7200000000003
}
}
}
}

There are different other types of metrics aggregation which I am not able to cover in a blog but I have tried to explain all important metrics aggregations

Other Blogs on Elastic Stack:
Introduction to Elasticsearch

Elasticsearch Installation and Configuration on Ubuntu 14.04
Log analysis with Elastic stack 
Elasticsearch Rest API
Basics of Data Search in Elasticsearch
Elasticsearch Rest API
Wildcard and Boolean Search in Elasticsearch
Configure Logstash to push MySQL data into Elasticsearch 
Metrics Aggregation in Elasticsearch
Bucket Aggregation in Elasticsearch
How to create Elasticsearch Cluster

If you found this article interesting, you can explore  "Mastering Kibana 6.0" and "Kibana 7 Quick Start Guide" to get more insight about Kibana and how we can configure ELK to create dashboards for key performance indicators


About Author

Anurag Srivastava

Author | Blogger | Tech Lead | Elastic Stack | Innovator |

View Profile

Comments (0)

Leave a comment

Related Blogs

Basics of Data Search in Elasticsearch

Aug 4, 2018, 7:02:21 AM | Anurag Srivastava

Elasticsearch Rest API

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava

Bucket Aggregation in Elasticsearch

Aug 29, 2018, 7:15:06 PM | Anurag Srivastava

Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

Introduction to Elasticsearch Aggregations

Aug 14, 2018, 4:47:56 PM | Anurag Srivastava

Create a Pie Chart in Kibana

Dec 24, 2018, 5:25:28 PM | Anurag Srivastava

Create word cloud in Python

Jun 30, 2018, 6:06:45 AM | Anurag Srivastava

Typecasting in PHP for short datatype

Jun 8, 2018, 8:03:52 AM | Lovish Sharma

Top Blogs

Configure SonarQube Scanner with Jenkins

Jun 21, 2018, 4:58:11 AM | Anurag Srivastava

Build and deploy Angular code using Python

Jun 26, 2018, 4:50:18 PM | Anurag Srivastava

Configure Jenkins for Automated Code Deployment

Jun 13, 2018, 3:44:01 PM | Anurag Srivastava

SonarQube installation on Ubuntu

May 12, 2018, 4:47:07 PM | Anurag Srivastava

Execute Commands on Remote Machines using sshpass

Jul 16, 2018, 5:00:02 PM | Anurag Srivastava

Why SonarQube is important for IT projects ?

Apr 24, 2018, 2:52:28 PM | Anurag Srivastava

Elasticsearch Rest API

Jul 31, 2018, 6:16:42 PM | Anurag Srivastava

Analyze your project with SonarQube

Jun 2, 2018, 10:49:54 AM | Anurag Srivastava

Wildcard and Boolean Search in Elasticsearch

Aug 10, 2018, 7:14:40 PM | Anurag Srivastava

Install Jenkins on Ubuntu

May 26, 2018, 6:42:02 PM | Anurag Srivastava