gwqjava/heliosstreams

Some kafka projects

HeliosStreams

Streaming performance data all the way home.

Data Types

Data Format Specifications

Value Types

Symbol	Name	Type	Description	Parameters
A	Accumulator	Valueless	Tracking an all time total of a specific event type.	None
M	Meter	Valueless	Tracking the per seconds/minutes/hours/day rate of a specific event type	None
D	Delta	Value	The value of a given metric instance is the delta of the current and the prior value	TBD
P	Period Aggregated	Value	Values are aggregated to count, min, max and average per defined period (e.g. 15 sedonds)	TBD
S	Straight Through	Value	The metric is not aggregated and forwarded directly to endpoint	None

Plain Text

The plain text stream is typically used when extracting data from application log or some other unstructured data feed after performing an extract-and-format operation in an intermediary such as LogStash and/or Filebeat.

Data Type Routing

• Directed: The line of text is beig submitted to a stream that is specific to the data type of the metric.
• Undirected: The line of text contains a symbol representing the data type of the metric and will be routed to an appropriate stream processor according to that symbol.

Text Line Format

[<value-type>,]<timestamp>, [<value>,] <metric-name>, <host>, <app> [,<tagkey1>=<tagvalue1>,<tagkeyn*>=<tagvaluen>]*

• value-type: The value type symbol indicating how the metric should be routed. Only required if the routing is undirected.
• timestamp: The effective timestamp of the metric in seconds-since-the epoch or milliseconds-since-the epoch.
• value: The metric numeric value. Absent in valueless types.
• metric-name: The identifier of the meaning of the metric. The fully qualified (global) name is comprised of the metric-name, the host, the app and the tags.
• host : The name of the host the metric originated from
• app : The name of the application the metric originated from
• tags : Key value pairs that qualify the instance of the metric-name.