could analyzer process non-continuously data？

[earthgecko]

could analyzer process non-continuously data？

[earthgecko]

Hi @litterzhang Skyline expects all data to be continous in terms of processing in general.
Analyzer does not process non-continuous data in the way you are describing, but Boundary can with a modification.

To monitor non-continuous data in this way you can use this modified version of boundary_algorithms.py with the greater_than algorithm.
https://gist.github.com/earthgecko/b77628cde9cce4c643877e2cde011fca

If you replace your
/opt/skyline/github/skyline/skyline/boundary/boundary_algorithms.py
with the boundary_algorithms.py from in the gist above you can then monitor discontinuous metrics as follows.
Here is an example Boundary setting that will trigger an anomaly on a metric named stats.app.status_code.503, if the metric had no data and then suddenly recieved any positive value once, alerts would be sent to smtp and slack in this case. To achieve this we would add the following to the BOUNDARY_METRICS tuple:

('stats.app.status_code.503', 'greater_than', 1800, 0, 0, 1, 1, 'smtp|slack'),

Where the fields are (as per https://earthgecko-skyline.readthedocs.io/en/stable/skyline.html#settings.BOUNDARY_METRICS):

('metric_name', 'algorithm', EXPIRATION_TIME, MIN_AVERAGE, MIN_AVERAGE_SECONDS, TRIGGER_VALUE, ALERT_THRESHOLD, 'ALERT_VIAS'),

With an EXPIRATION_TIME of 1800 seconds, this would alert once every 30 minutes if the condition occured.
Add the metric/s you want to monitor in this manner to settings.py and then restart boundary.

To implement the example setting above, you will also need to define this metric or metric namespace in your BOUNDARY_SMTP_OPTS:

BOUNDARY_SMTP_OPTS = {
    # This specifies the sender of email alerts.
    'sender': 'skyline@your_domain.com',
    # recipients is a dictionary mapping metric names
    # (exactly matching those listed in ALERTS) to an array of e-mail addresses
    'recipients': {
        'stats.app.status_code.503': ['you@your_domain.com'],
    },
    # This is the default recipient which acts as a catchall for alert tuple
    # that is defined
    'default_recipient': ['you@your_domain.com'],
    'embed-images': True,
    # Send graphite graphs at the most meaningful resolution if different from
    # FULL_DURATION
    'graphite_previous_hours': 7,
    'graphite_graph_line_color': 'pink',
}

And for slack, in the BOUNDARY_SLACK_OPTS e.g.

BOUNDARY_SLACK_OPTS = {
    # Bot User OAuth Access Token
    'bot_user_oauth_access_token': '<YOUR_slack_skyline_bot_user_oauth_access_token>',
    # list of slack channels to notify about each anomaly
    # (similar to BOUNDARY_SMTP_OPTS['recipients'])
    # channel names - you can either pass the channel name (#general) or encoded
    # ID (C024BE91L)
    'channels': {
        'skyline': ('#skyline',),
        'skyline_test.alerters.test': ('#skyline',),
        'stats.app.status_code.503': ('#skyline',),
        'stats': ('#skyline',),
    },
    'icon_emoji': ':chart_with_upwards_trend:',
}

[earthgecko]

I shall test this modification and release it in the next release as in terms of Boundary the length of the time series should not have an adverse effect on alerting as long as the alert EXPIRATION_TIME is set sensibly on the BOUNDARY_METRICS alert tuple. This way Skyline can handle and alert on discontinuous data, in the Boundary context at least. @litterzhang thank you for specifying a new use case.

[litterzhang]

Thanks! I'll try to use Boundary processing non-continuous data. But in my system, the datapoint never comes point by point.Maybe i need to modify Analyzer make timeseries in format before execute abnormal-detect alg.