1. Introduction to Prometheus alert
Prometheus Alert is an open source message forwarding system of operation and maintenance alarm center, which supports the early warning messages sent by the mainstream monitoring system Prometheus, log system grayog and data visualization system Grafana. Notification channels support nailing, wechat, Huawei cloud SMS, Tencent cloud SMS, Tencent cloud phone, Alibaba cloud SMS, Alibaba cloud phone, etc.
2. Prometheus alert feature
It supports multiple sources of information, including prometheus, graylog2, graylog3 and grafana.
Support multiple types of sending targets, including nailing, wechat, Tencent SMS, Tencent voice and Huawei SMS.
The alarm level is added for Prometheus, and it supports sending messages to different target objects according to different levels.
Simplify Prometheus packet configuration and support sending to single or multiple receivers according to specific messages.
Add mobile phone number configuration item and number automatic polling configuration, which can be sent to a single individual alarm information, or sent to multiple personnel through automatic polling, and support sending to different personnel according to different dates.
Add Dashboard to temporarily support testing whether the configuration is correct
3. Deployment method
Prometheus alert can be deployed on local and cloud platforms, supporting windows, linux, public cloud, private cloud, hybrid cloud, container and kubernetes. You can choose the appropriate method to deploy promethesalert according to the actual scenario or demand:
Container deployment:
$ git clone https://github.com/feiyu563/PrometheusAlert.git $ mkdir /etc/prometheusalert-center/ $ cp PrometheusAlert/conf/app.conf /etc/prometheusalert-center/ $ docker run -d -p 8080:8080 -v /etc/prometheusalert-center:/app/conf --name prometheusalert-center feiyu563/prometheus-alert:latest
Linux system deployment
$ git clone https://github.com/feiyu563/PrometheusAlert.git $ cd PrometheusAlert/example/linux/ # Run in the background, please execute nohup/ PrometheusAlert & $ ./PrometheusAlert
Windows system deployment
$ git clone https://github.com/feiyu563/PrometheusAlert.git $ cd PrometheusAlert/example/windows/ Double click Run PrometheusAlert.exe that will do
kubernetes deployment
$ kubectl app -n monitoring -f https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/example/kubernetes/PrometheusAlert-Deployment.yaml
Helm deployment
$ git clone https://github.com/feiyu563/PrometheusAlert.git $ cd PrometheusAlert/example/helm/prometheusalert # To modify the configuration file, please update app. In config conf $ helm install -n monitoring .
After startup, you can use the browser to open the test address: http://127.0.0.1:8080
4. Modify the configuration file
Prometheualert configuration file app Conf is as follows:
#---------------------↓ global configuration-----------------------
appname = PrometheusAlert
#Listening port
httpport = 8080
runmode = dev
#Set proxy= http://123.123.123.123:8080
proxy =
#Open JSON request
copyrequestbody = true
#Alarm message title
title=PrometheusAlert
#Link to alarm platform address
#GraylogAlerturl=http://graylog.org
#Nailing alarm logo Icon address
#logourl=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png
#Nail alarm recovery logo Icon address
#logourl=https://raw.githubusercontent.com/feiyu563/PrometheusAlert/master/doc/alert-center.png
#SMS alarm level (SMS alarm if it is equal to 3) alarm level definition: 0 information, 1 warning, 2 general serious, 3 serious, 4 disaster
messagelevel=3
#Telephone alarm level (voice alarm when it is equal to 4) alarm level definition: 0 information, 1 warning, 2 general serious, 3 serious, 4 disaster
phonecalllevel=4
#Default dialing number (this item needs to be configured to test the SMS and phone functions on the page)
defaultphone=18327026557
#Whether to enable telephone notification for fault recovery. 0 is off and 1 is on
phonecallresolved=0
#Automatic alarm suppression (automatic alarm suppression is the default alarm information of the same alarm source, only the first alarm information with the highest alarm level is sent, and other messages are shielded by default. The purpose of this is to reduce the number of messages from the same alarm source and prevent alarm bombs. 0 is off and 1 is on)
silent=1
#Whether to output file or console in the foreground
logtype=file
#log file path
logpath=logs/prometheusalertcenter.log
#Convert the time zone of Prometheus and grayog alarm message to CST time zone (if it is already CST time zone by default, do not turn it on)
prometheus_cst_time=0
#---------------------↓webhook-----------------------
#Whether to open the nail alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-dingding=0
#Default nailing robot address
ddurl=https://oapi.dingtalk.com/robot/send?access_token=xxxxx
#Turn on @ owner (0 = off, 1 = on)
dd_isatall=1
#Whether to open wechat alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-weixin=0
#Default enterprise wechat robot address
wxurl=https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=xxxxx
#Whether to open the flight Book v1 alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-feishu=0
#Default address of flying Book Robot
fsurl=
#Whether to open the flybook v2 alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-feishuv2=1
#Default address of flying Book Robot
fsv2url=https://open.feishu.cn/open-apis/bot/v2/hook/xxxxxxxxxxxxxxxxxx
#---------------------↓ Tencent cloud interface-----------------------
#Whether to open Tencent cloud SMS alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-txdx=0
#Tencent cloud SMS interface key
TXY_DX_appkey=xxxxx
#Tencent cloud SMS template ID Tencent cloud SMS template configuration can refer to prometheus alarm: {1}
TXY_DX_tpl_id=xxxxx
#Tencent cloud sms sdk app id
TXY_DX_sdkappid=xxxxx
#Tencent cloud SMS signature is filled in according to the signature approved by itself
TXY_DX_sign=Tencent cloud
#Whether to open Tencent cloud phone alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-txdh=0
#Tencent cloud phone interface key
TXY_DH_phonecallappkey=xxxxx
#Tencent cloud phone template ID
#TXY_DH_phonecalltpl_id=xxxxx
#Tencent cloud phone sdk app id
TXY_DH_phonecallsdkappid=xxxxx
#---------------------↓ Huawei cloud interface-----------------------
#Whether to enable Huawei cloud SMS alarm channel. Multiple channels can be started at the same time. 0 is off and 1 is on
open-hwdx=0
#Huawei cloud SMS interface key
HWY_DX_APP_Key=xxxxxxxxxxxxxxxxxxxxxx
#Huawei cloud SMS interface Secret
HWY_DX_APP_Secret=xxxxxxxxxxxxxxxxxxxxxx
#Huawei cloud APP access address (port interface address)
HWY_DX_APP_Url=https://rtcsms.cn-north-1.myhuaweicloud.com:10743
#Huawei cloud SMS template ID
HWY_DX_Templateid=xxxxxxxxxxxxxxxxxxxxxx
#Huawei cloud signature name must be an approved signature name that is consistent with the template type and filled in according to your own actual signature
HWY_DX_Signature=Hua Weiyun
#Huawei cloud signature channel number
HWY_DX_Sender=xxxxxxxxxx
#---------------------↓ alicloud interface-----------------------
#Whether to enable the Alibaba cloud SMS alarm channel. Multiple channels can be started at the same time. 0 is off and 1 is on
open-alydx=0
#ID of Alibaba cloud SMS primary account AccessKey
ALY_DX_AccessKeyId=xxxxxxxxxxxxxxxxxxxxxx
#Alicloud SMS interface key
ALY_DX_AccessSecret=xxxxxxxxxxxxxxxxxxxxxx
#Alibaba cloud SMS signature name
ALY_DX_SignName=Alibaba cloud
#Alibaba cloud SMS template ID
ALY_DX_Template=xxxxxxxxxxxxxxxxxxxxxx
#Whether to turn on the Alibaba cloud telephone alarm channel. You can start multiple channels at the same time. 0 is off and 1 is on
open-alydh=0
#The ID of the Alibaba cloud phone primary account AccessKey
ALY_DH_AccessKeyId=xxxxxxxxxxxxxxxxxxxxxx
#Alicloud telephone interface key
ALY_DH_AccessSecret=xxxxxxxxxxxxxxxxxxxxxx
#Alibaba cloud phone is called and must be a purchased number
ALY_DX_CalledShowNumber=xxxxxxxxx
#Alibaba cloud TTS template ID
ALY_DH_TtsCode=xxxxxxxx
#---------------------↓ Ronglian cloud interface-----------------------
#Whether to open the alarm channel of Ronglian cloud phone. Multiple channels can be started at the same time. 0 is closed and 1 is open
RLY_DH_open-rlydh=0
#Ronglian cloud basic interface address
RLY_URL=https://app.cloopen.com:8883/2013-12-26/Accounts/
#Rong Lianyun background SID
RLY_ACCOUNT_SID=xxxxxxxxxxx
#Ronglianyun API token
RLY_ACCOUNT_TOKEN=xxxxxxxxxx
#Ronglianyun app_id
RLY_APP_ID=xxxxxxxxxxxxx
#---------------------↓ mail configuration-----------------------
#Open mail
open-email=0
#Mailing server address
Email_host=smtp.qq.com
#Mail server port
Email_port=465
#Mail account
Email_user=xxxxxxx@qq.com
#Mail password
Email_password=xxxxxx
#Mail title
Email_title=Operation and maintenance alarm
#Default sending mailbox
Default_emails=xxxxx@qq.com,xxxxx@qq.com
#---------------------↓ Qimo cloud interface-----------------------
#Whether to open Qimo short message alarm channel. Multiple channels can be started at the same time. 0 is closed and 1 is open
open-7moordx=0
#Qimo account ID
7MOOR_ACCOUNT_ID=Nxxx
#Qimo account APISecret
7MOOR_ACCOUNT_APISECRET=xxx
#SMS template no. of Qimo account
7MOOR_DX_TEMPLATENUM=n
#Note: Qimo SMS variable here only uses one var1, which is written dead in the code.
#-----------
#Whether to open Qimo webcall voice notification alarm channel, which can start multiple channels at the same time. 0 is closed and 1 is open
open-7moordh=0
#Please add virtual service number and text node on Qimo platform
#Virtual service number of Qimo account webcall
7MOOR_WEBCALL_SERVICENO=xxx
# For the variable replaced in the text node, I configured text. If the replaced variable is not text, please modify this configuration
7MOOR_WEBCALL_VOICE_VAR=text
Prometheus alert supports interfaces such as Alibaba cloud, Tencent cloud and Huawei cloud. Since the company uses Feishu, Feishu is used here as an alarm
5. Log in to prometheualert
Enter the url of prometheus alert to log in to the UI page of prometheus
Select the template you need here and check the log in Prometheus alert. There is a json format. After Formatting through json on the network, transfer the json content into the template, then save and test it. If you pass, you can receive the alarm information on the flybook