Giter Site home page Giter Site logo

johnsusek / praeco Goto Github PK

View Code? Open in Web Editor NEW
526.0 20.0 86.0 12.86 MB

Elasticsearch alerting made simple.

License: GNU General Public License v3.0

JavaScript 46.31% HTML 0.04% Vue 51.63% Dockerfile 0.08% Shell 0.89% SCSS 1.05%
elastalert logstash elasticsearch alerting

praeco's Introduction

Praeco

⚠ Hi developer. Please contribute to the project if you find a bug or suggest an improvement / feature.

GitHub release Docker Pulls GitHub stars

Praeco is an alerting tool for Elasticsearch – a GUI for ElastAlert 2, using the ElastAlert API.

  • Interactively build alerts for your Elasticsearch data using a query builder
  • Preview results in an interactive chart
  • Test your alerts against historical data
  • Send notifications to Slack, MS Teams, Email, Telegram, Jira, Line Notify, Mattermost, Command, Gitter, Amazon SNS, Amazon SES, Zabbix, Twilio, PagerTree, Exotel, GoogleChat, Stomp, Splunk On-Call (Formerly VictorOps), ServiceNow, Chatwork, Discord, TheHive, Alerta, Datadog, Rocket.Chat, PagerDuty, Tencent SMS, Dingtalk, Alertmanager, OpsGenie, Graylog GELF, Lark, IRIS or an HTTP POST/HTTP POST 2 endpoint
  • Supports the Any, Blacklist, Whitelist, Change, Frequency, Flatline, Spike, Cardinality, New Term, and Metric Aggregation rule types
  • View logs of when your alerts check, fire and fail


👉 Praeco is a completely free GPLv3 project, in return I only ask that you fill out this simple survey about how you use it.

Quickstart

Run the app using Docker compose. Praeco includes everything you need to get started. Just provide it the IP address of your Elasticsearch instance.

mkdir -p rules rule_templates
chmod -R 777 rules rule_templates
echo "slack_webhook_url: ''"  | sudo tee -a rules/BaseRule.config >/dev/null
export PRAECO_ELASTICSEARCH=<your elasticsearch ip>
docker-compose up
  • Don't use 127.0.0.1 for PRAECO_ELASTICSEARCH. See first item under the Troubleshooting section.

  • To set up notification settings like API keys edit rules/BaseRule.config.

Praeco should now be available on http://127.0.0.1:8080

A walkthrough article is available to guide you through creating your first rule.

Upgrading

If you use docker-compose.yml published on github as it is, it will be the current latest version instead of a specific version. If you want to specify a specific version, edit it yourself and then run docker-compose up --force-recreate --build.

docker-compose down 
docker rmi praecoapp/praeco:latest
docker rmi praecoapp/elastalert-server:latest
docker pull praecoapp/praeco:latest
docker pull praecoapp/elastalert-server:latest
docker-compose up --force-recreate --build

You may need to update your config files when a new version comes out. Please see UPGRADING.md for version-specific instructions.

Configuration

Edit rules/BaseRule.config, config/api.config.json, config/elastalert.yaml, and/or public/praeco.config.json for advanced configuration options. See the api docs and the example ElastAlert 2 config for more information.

Any ElastAlert 2 option you put into rules/BaseRule.config will be applied to every rule.

The following config settings are available in praeco.config.json:

// Link back to your Praeco instance, used in Slack alerts
"appUrl": "http://praeco-app-url:8080",

// A recordatus (https://github.com/johnsusek/recordatus) instance for javascript error reporting
"errorLoggerUrl": "",

// Hide these fields when editing rules, if they are already filled in template
"hidePreconfiguredFields": []

DockerHub

Praeco

ElastAlert Server

Praeco & elastalert server docker image relations table

praecoapp/elastalert-server ChangeLog

FAQ

Is there a sample to start elasticsearch, kibana, elastalert-server, Praeco with docker-compose?

docker compose sample(telegram)

Please tell me the response status of the alert notification destination.

👉 ElastAlert 2 Alerts support status

Will elastalert-server / Praeco be supported forever?

First of all, please understand that it is open source software. If you need generous support, please consider paid support software.

  • We are not responsible for fixing bugs.
  • There is no obligation to respond to your request.
  • There is no obligation to add features.
  • We do not always answer issues.

It does not work with the combination of bitsensor/elastalert and praeco. Is it a bug?

Do not use bitsensor/elastalert as it does not implement the features required by praeco. Use praecoapp/elastalert-server.

It does not work with the combination of karql/elastalert2-server and praeco. Is it a bug?

Do not use karql/elastalert2-server as it does not implement the features required by praeco. Use praecoapp/elastalert-server.

Does yelp/elastalert support it?

yelp/elastalert is no longer supported as maintenance has been discontinued.

Main challenges of yelp/elastalert

  • Not compatible with python 3.9 or later.
  • Line Notify, PagerTree, Stomp and Zabbix alert notifications do not work due to a bug.
  • kibana Discover only guarantees operation up to kibana 7.3.

Does it support elasticsearch 5.x?

Not Support

Doesi support elasticserch 6.x?`

Elasticserch 6 is no longer supported from elastalert2 2.4.0.
Please use praecoapp/elastalert-server:20220109 with elastalert2 installed before 2.4.0.

Does it support elasticsearch 7.x?

Support

Does it support elasticsearch 8.x?

  • Supports elasticsearch 8 with praeco 1.8.11 or later.
  • elastalert-server is compatible with elasticsearch 8.

Can you support a version that is not the latest version?

Not Support

Is it possible to set Percentage Match on the screen?

Not Support

Is it possible to set Spike Aggregation on the screen?

Not Support

Is it possible to set custom format (timestamp_type, timestamp_format, timestamp_format_expr) on the screen?

Not Support

Does elasticsearch-oss?

Not Support

Does elastalert-server support Amazon OpenSearch Service (formerly Amazon Elasticsearch)?

Not Support

Does elastalert-server support OpenSearch?

Not Support

Does elastalert-server support Elastic Cloud CloudID connections?

Not Support

Does elastalert-server support Elasticsearch ApiKey authentication connections?

Not Support

Does elastalert-server support Elasticsearch Bearer authentication connections?

Not Support

Does elastalert-server support Elasticsearch proxy connections?

Not Support

Mulit-elasticsearch instances

Not Support

If the contents of BaseRule.config are empty, an error will occur.

Currently, an error will occur when BaseRule.config is empty, so please make it as described below.

echo "slack_webhook_url: ''"  | sudo tee -a rules/BaseRule.config >/dev/null

Are you planning to add a login screen?

There are no plans to support this at this time.

Are you planning to support Email_format for Email?

There are no plans to support this at this time.

I can't send an email when I specify a gmail or Microsoft 365 address in from_addr of Email. Is there a workaround?

  • For Gmail, you need to set Allow insecure apps.
  • There is no workaround for Microsoft 365.

Is it possible to raise an alert only during a specific time period?

It is possible to set within or outside the time zone specified in the Use Time Window.

2

Is it possible to execute the rule only at a specific time?

It can be set with Limit Excecution.

2

When using the field specified by alert_subject_args in alert_subject, how do you set it on the screen?

  • Click the Alert Subject Args link for the number of fields you want to add and press the "Add alert_textargs" button. Enter the field you want to add.
  • When embedding a field in Subject, specify a serial number from 0, such as {0} for the first and {1} for the second.

1

When using the field specified by alert_text_args in alert_text, how do you set it on the screen?

  • Click the AlertTextArgs link for as many fields as you want to add and press the "Add alert_textargs" button. Enter the field you want to add.
  • When embedding a field in Body text, specify a serial number from 0, such as {0} for the first and {1} for the second.

2

[Third Party Tools] ElastAlert Server & Praeco Helm Chart

ElastAlert Server Helm Chart
Praeco Helm Chart
Installing Praeco (ElastAlert GUI) into Kubernetes with Helm

How do I change ElastAlert 2 options, like SSL, user/pass, etc?

Edit config/elastalert.yaml and uncomment the appropriate lines.

[elastalert-server] How do I connect to elasticsearch using SSL?

Edit config/api.config.json and set/add "es_ssl": true.
option "ea_verify_certs", "es_ca_certs", "es_client_cert", "es_client_key".
configuration

[elastalert-server] How do I connect to elasticsearch with a username and password?

Edit es_username and es_password in config/api.config.json and config/elastalert.yaml.

How do I serve the praeco UI over https?

The praeco UI is served by an included nginx server (see Dockerfile). Configure it as you would any nginx project by editing the files in nginx_config. Then update your docker-compose.yml and add your certificate files (under webapp volumes). Another option is using a reverse proxy.

How do I serve the praeco UI under a custom base path, i.e. http://www.my-domain.com:8080/my-path/

Uncomment the declaration of the VUE_APP_BASE_URL environment variable in docker-compose.yml and define the path you want.

    environment:
      VUE_APP_BASE_URL: /my-path/

Uncomment the rewrite command in nginx.config/default.conf and define the same path as in teh environment variable above.

rewrite ^/my-path(/.*)$ $1 last;

How do I change the writeback index?

Edit config/elastalert.yaml and config/api.config.json and change the writeback_index values.

How do I run this on Windows?

First, install docker and docker-compose.

Then, using powershell, run these commands:

$Env:PRAECO_ELASTICSEARCH="1.2.3.4"
docker-compose.exe up

Replace 1.2.3.4 with your Elasticsearch IP.

Can I import my current ElastAlert 2 rules into praeco?

Unfortunately this is not a possibility for two reasons. First, praeco only supports a subset of ElastAlert 2 features, so only certain rules would work. Second, praeco cannot automatically create the query builder ui from an arbitrary ElastAlert 2 filter entry, due to the potential complexity and combinations of filters someone can put in their rule file.

Can I export my praeco rules into another elastalert 2 instance?

Yes, the praeco rule files are 100% compatible with other elastalert servers.

Troubleshooting

I am using 127.0.0.1 for PRAECO_ELASTICSEARCH and it isn't working

Praeco, running within a docker container, cannot communicate with your ES bound to localhost. You need to change your ES network.host setting to something different. The value of _site_ is suggested, that will bind to a local network IP on your machine. Then use that IP address for PRAECO_ELASTICSEARCH. Here's a working example:

elasticsearch -E network.host=_site_
export PRAECO_ELASTICSEARCH=192.168.1.145
mkdir -p rules rule_templates
chmod -R 777 rules rule_templates
docker-compose up

Replace 192.168.1.145 with the IP address your ES binds to (look for bound_addresses in the elasticsearch launch log).

I am getting high CPU usage on some of my rules

When editing a rule, click "WITH OPTIONS" and try using the "Use count query" option. This can dramatically speed up processing time for large amounts of data (tens of thousands of results).

I'm not receiving alerts even though I expect them

First of all, try to test your alert with varying time frames and see if that is returning any results.

If the test is returning results, but you are not receiving any alerts, check the error log. There may be a problem with your alerter settings. Make sure you edited rules/BaseRule.config and have correct values in there.

If the test is not returning results, even though you think it should, try reading the ElastAlert 2 docs for your rule type. Compare the yaml from praeco with the options from the docs to make sure the rule is being created as expected. If praeco is generating the wrong yaml, please file an issue.

Failed to establish a new connection: [Errno 111] Connection refused

You will see this error when launching if praeco cannot find elasticsearch at the IP address you specified at $PRAECO_ELASTICSEARCH. Please make sure you can communicate with this IP address by issuing the following command: curl http://$PRAECO_ELASTICSEARCH:9200. If the connection is refused, your machine cannot communicate with Elasticsearch, it may be a networking problem.

404 error in error log for slack webhook

Make sure the channel/username you are trying to post to exists.

How to setup TheHive?

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#thehive for how to configure your BaseRule.config file.

  • hive_connection is set in BaseRule.config.
 hive_connection:
   hive_host: http://localhost
   hive_port: <hive_port>
   hive_apikey: <hive_apikey>
   hive_proxies:
     http: ''
     https: ''
  • hive_alert_config is set on the Praeco screen.
  • Not Support hive_observable_data_mapping.

How to setup Slack?

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#slack for how to configure your BaseRule.config file.

  • Replace slack_webhook_url with the URL of your channel.
  • Describe the following settings in BaseRule.config. Please set other settings on the screen.
slack_webhook_url: 'https://hooks.slack.com/services/xxxxxxxxx/xxxxxxxx/xxxxxxxxxxxxxxxxxxx'

How to setup Telegram?

telegram_room_id can be set on the praeco screen.

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#telegram for how to configure your BaseRule.config file.

  • Replace telegram_bot_token with the your bot token.
  • Describe the following settings in BaseRule.config. Please set other settings on the screen.
telegram_bot_token: 'xxxxxxxxxx:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

How to setup Mattermost?

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#mattermost for how to configure your BaseRule.config file.

  • Replace mattermost_webhook_url with the URL of your webhook.
  • Describe the following settings in BaseRule.config. Please set other settings on the screen.
mattermost_webhook_url: 'https://xxxxxx/hooks/xxxxxxxxxxxxxxxx'

How to setup Rocket.Chat?

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#rocket-chat for how to configure your BaseRule.config file.

  • Replace rocket_chat_webhook_url with the URL of your webhook.
  • Describe the following settings in BaseRule.config. Please set other settings on the screen.
rocket_chat_webhook_url: 'https://xxxxxx/xxxxx/xxxxxxxxxxxxxxxx'

How to setup Email?

Sorry Not Support email_format.

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#email for how to configure your BaseRule.config file.

example setting

Describe the following settings in BaseRule.config. Please set other settings on the screen.

smtp_auth_file: '/opt/elastalert/pass/smtp_auth_user.yaml'

smtp_auth_user.yaml

user: [email protected]
password: xxx

Google account in advance → Apps that can access your account → Allow less secure apps: Disabled → Enabled When I specified the Gmail address with from_addr and checked the operation, it worked without problems.

smtp_auth_file: '/opt/elastalert/smtp/smtp_auth_user.yaml"

smtp_auth_user.yaml

user: xxx@gmail
password: xxx

How to setup Jira?

Please see https://elastalert2.readthedocs.io/en/latest/ruletypes.html#jira for how to configure your BaseRule.config file.

Architecture details

Praeco is a vue.js app (hosted in an nginx docker container) that communicates with the ElastAlert API (running in another docker container) to view/edit rules. The elastalert api interacts with the included ElastAlert 2 python daemon directly for various tasks including testing and silencing rules, and indirectly by modifying or creating rule files in the rules/ directory.

When you run praeco using the quickstart instructions, it runs these two docker containers, per the docker-compose.yml file.

Praeco uses a fork of the elastalert api server, which is why the docker image source is johnsusek/elastalert-server.

NOTE: Only the api server is a fork, the ElastAlert 2 daemon itself is built from the master branch whenever a new version of the johnsusek/elastalert-server docker image is created.

Please see the development section below if you're interested in running these services separately.

Manual/Dev installation

NOTE: If you're just interested in developing Praeco UI features locally (and not changing ElastAlert 2 or the api), you can skip right to Praeco setup and just run the internal ElastAlert Server with docker-compose up elastalert-server .


First, you need a local copy of the elastalert api server running, which itself needs ElastAlert 2. Start by cloning the neccessary repos

$ cd
$ git clone https://github.com/jertel/elastalert2.git
$ git clone https://github.com/johnsusek/elastalert-server.git
$ git clone https://github.com/johnsusek/praeco.git

Setting up ElastAlert 2

Python Support version

  • Compatible with 3.9 or later

Elasticsearch Support version

  • 7.x
  • 8.x

Configure the ElastAlert 2 config.yaml with:

  • Your es_host
  • A unique writeback_index
  • Change the rules_folder to rules
cd ~/elastalert2
mkdir -p rules rule_templates
chmod -R 777 rules rule_templates
echo "slack_webhook_url: ''"  | sudo tee -a rules/BaseRule.config >/dev/null
pip install "setuptools>=11.3"
python setup.py install
cp ./examples/config.yaml.example ./config.yaml
vi config.yaml

Setting up the API server

Configure the api server config.json with:

  • An absolute path to your ElastAlert 2 folder for elastalertPath
  • The address of your elasticsearch instance for es_host
  • The same writeback_index from the config.yaml
# nvm install
# https://github.com/nvm-sh/nvm#install--update-script
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
$ vi ~/.bash_profile

export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm

source ~/.bash_profile

# npm & node install
npm install -g npm
nvm install 18.19.0
cd ~/elastalert-server
vi config/config.json
nvm use "$(cat .nvmrc)"
npm install
npm run start

You should see this line if it started successfully:

INFO elastalert-server: Server:  Server started

Setting up praeco

Finally, run praeco:

# No need to implement if the environment is the same as elastalert-server
# nvm install
# https://github.com/nvm-sh/nvm#install--update-script
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
$ vi ~/.bash_profile

export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm

source ~/.bash_profile

# npm & node install
npm install -g npm
nvm install 18.19.0
cd ~/praeco
nvm use "$(cat .nvmrc)"
npm install --legacy-peer-deps
export PRAECO_ELASTICSEARCH=<your elasticsearch ip>
npm run serve

You should now see the UI running at http://localhost:8080.

If you have any difficulties please open a github issue with your problem.

Maintainers


John Susek

Naoyuki Sano

praeco's People

Contributors

abhishek-rio avatar adrienyhuel avatar danielskowronski avatar dohq avatar giedriusramas avatar harduino avatar johnsusek avatar kourg3 avatar nsano-rururu avatar osherdp avatar phil21 avatar samuelbizon avatar vtdat avatar zdykstra avatar zsomborbenedek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

praeco's Issues

Add a feature to hide some pre-configured fields

Some fields we want pre-configured, but hidden in the UI so users don't have to worry about them

For example, the Slack webhook url - once it is in a template, it shouldn't be shown when building a rule.

Add an option in the config file with a list of fields then get hidden in the rule UI

Change way terms are found

Right now terms are pulled from the test result preview. But if the alert you want to create uses a term not in this preview, it can't be inserted because it's not in the term list.

We should fetch the terms directly from ES, probably, instead of relying on the preview result

Ability to pause rules

Rules can be paused by converting them to templates, with a special name prefix that gets treated special in the GUI. They might also be paused be removing all the alerts from the rule (but how do we know which to add back when unpausing? and do they still get run on the server?).

Add a pause button to the GUI to do this.

Add view for rule

Right now, when looking at a rule, it just shows the JSON dump of the rule. Need to add a page to display this information in tables etc

Validation for rule editor

Right now there are some fields that aren't being validated and could cause a server side error. For example, if slack notification type is selected, make those fields mandatory.

Improve rule/template view

Right now rule view is pretty rough, just tables of fields. It is also missing some output related to spike alerts, which needs to be added. Finally it is using the view model of the config file, not the vuex config model. Make these changes and low hanging visual improvements to improve this view.

Error handling

There are a few places in the GUI where error handling needs to be added. Add those so users can see as much information as possible from elastalert, instead of just a generic "something went wrong" errors.

Create example videos/images

As part of showing what this application can do, some relatively short videos should be put into the README and possibly some screenshots with descriptions of what they are showing.

If voiceover is needed for the example videos we have facilities to do that @ServerCentral and we have folks who can work off a script if preferred.

Wait to apply time changes in eschart...

... until popover is closed.

When editing time range in the es chart, the changes happen immediately upon input. Instead, it should only refresh the chart once the popover closes.

Dropdowns for rule options that are enums

Some rules config options are enums, they should be dropdowns instead of free-text fields, because users don't know what to type and could put in something wrong.

Examples: Settings -> Type, or Alert -> Slack -> Message color. These should be dropdowns with the correct options.

https support

Add an endpoint to the application server to proxy https traffic to the api

Create walkthrough document...

... that explains how to install entire elastalert-praeco stack from scratch

  • pull repos
  • configure settings
  • run docker images

Create dockerfile

For people that just want to run the GUI without developing, there should be an easy command-line invocation to spin up a docker instance with a node server for serving the praeco files.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.