Skip to main content
Version: 0.12

grouper::bucket

Bucket will perform a sliding window rate limiting based on event metadata. Limits are applied for every $class. In a $class each $dimensions is allowed to pass $rate messages per second.

This operator does not support configuration.

This operator preserves event metadata.

Metadata Variables:

  • $class - The class of an event. (String)
  • $rate - Allowed events per second per class/dimension (Number)
  • (Optional) $dimensions - The dimensions of the event. (Any)
  • (Optional)$cardinality - the maximum number of dimensions kept track of at the same time (Number, default: 1000)

Outputs:

  • out
  • error - Unprocessable events for example if $class or $rate are not set.
  • overflow - Events that exceed the rate defined for them

Example:

define operator group from grouper::bucket;

Metrics:

The bucket operator generates additional metrics. For each class the following two statistics are generated (as an example):

{"measurement":"bucketing",
"tags":{
"action":"pass",
"class":"test",
"direction":"output",
"node":"bucketing",
"pipeline":"main",
"port":"out"
},
"fields":{"count":93},
"timestamp":1553012903452340000
}
{"measurement":"bucketing",
"tags":{
"action":"overflow",
"class":"test",
"direction":"output",
"node":"bucketing",
"pipeline":"main",
"port":"out"
},
"fields":{"count":127},
"timestamp":1553012903452340000
}

This tells us the following, up until this measurement was published in the class test:

  • (pass) Passed 93 events
  • (overflow) Marked 127 events as overflow due to not fitting in the limit