Containerization | building RadonDB MySQL Cluster Monitoring Platform

In the previous article, we demonstrated how to restore RadonDB MySQL Cluster Data in S3 backup. This article will demonstrate how to use Prometheus[2] + Grafana[3] in KubeSphere[1] to build a MySQL monitoring platform and enable the required monitoring indicators.

background

Prometheus text-based exposure format has become the de facto standard format in the field of cloud native monitoring.

RadonDB MySQL monitoring engine is defined based on Prometheus MySQL exporter [5]. Grab the service indicators of RadonDB MySQL through mysqld exporter, and then realize the visualization of monitoring indicators by accessing the third-party application platform.

preparation

  • Ready to use Kubernetes or KubeSphere clusters
  • Deployed RadonDB MySQL Cluster deployment document
  • RadonDB MySQL Kubernetes version 2.1.0+

Deployment steps

Step 1: configure serviceMonitor

After serviceMonitor is enabled, mysqld will be bound automatically_ Exporter and Prometheus.

The serviceMonitor parameter contains the following fields:

serviceMonitor:
  enabled: true
  ## Additional labels for the serviceMonitor. Useful if you have multiple prometheus operators running to select only specific ServiceMonitors
  # additionalLabels:
  #   prometheus: prom-internal
  interval: 10s
  scrapeTimeout: 3s
  # jobLabel:
  # targetLabels:
  # podTargetLabels:
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      app.kubernetes.io/managed-by: mysql.radondb.com
      app.kubernetes.io/name: mysql

You can use charts / MySQL operator / values Configure serviceMonitor in yaml file.

  • When the Operator is newly deployed, servicemonitor Enabled is true by default, which means it is enabled by default.
  • The cluster with Operator version less than 2.1.0 has been deployed, and the Operator needs to be redeployed.

Step 2: configure metricsOpts

metricsOpts is CRD mysqlclusters mysql. radondb. COM, which can be configured by configuring mysql_ v1alpha1_ mysqlcluster. The parameter value in yaml file enables the monitoring service.

The metricsOpts parameter contains the following fields:

metricsOpts:
    enabled: false  
    image: prom/mysqld-exporter:v0.12.1

    resources:
      requests:
        cpu: 10m
        memory: 32Mi
      limits:
        cpu: 100m
        memory: 128Mi

metricsOpts.enabled defaults to false and needs to be manually set to true.

  • Set metricsopts If enabled is true, enable the cluster monitoring function;
  • Set the resource parameter value and define the resource quota size of the monitoring container;

After modifying the file parameters, use the following instructions to apply the configuration and deploy / update the cluster echo information as follows:

$ kubectl apply -f config/sample/mysql_v1alpha1_mysqlcluster.yaml
cluster.mysql.radondb.com/sample created/configured

View monitoring services

View via client

You can view the cluster monitoring service and serviceMonitor information through the following instructions.

$ kubectl get service,servicemonitor

$ kubectl describe servicemonitor <serviceName>

anticipate result

$ kubectl get service,servicemonitor
NAME                             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/mysql-operator-metrics   ClusterIP   10.96.242.205   <none>        8443/TCP   3h25m
service/sample-follower          ClusterIP   10.96.2.234     <none>        3306/TCP   21h
service/sample-leader            ClusterIP   10.96.30.238    <none>        3306/TCP   21h
service/sample-metrics           ClusterIP   10.96.7.222     <none>        9104/TCP   3h24m
service/sample-mysql             ClusterIP   None            <none>        3306/TCP   21h

NAME                                                              AGE
servicemonitor.monitoring.coreos.com/demo-mysql-operator          3h25m

$ kubectl describe servicemonitor demo-mysql-operator 
Name:         test-radondb-mysql-metrics
Namespace:    default
Labels:       app=test-radondb-mysql
              app.kubernetes.io/managed-by=Helm
              app.kubernetes.io/vendor=kubesphere
              chart=radondb-mysql-1.0.0
              heritage=Helm
              release=test
Annotations:  kubesphere.io/creator: admin
API Version:  monitoring.coreos.com/v1
Kind:         ServiceMonitor
......
Spec:
  Endpoints:
    Interval:        1m
    Path:            /metrics
    Port:            metrics
    Scheme:          http
    Scrape Timeout:  10s
......

View on KubeSphere platform

For radondb MySQL operators and clusters deployed in KubeSphere enterprise space, after monitoring is enabled, you can view the monitoring service status on the following page.

  • On the service page under the project space application load, click < cluster name > - metrics to view the monitoring service information.

  • On the container group page under project space application load, click the name of a container to view the status of metrics resources in the container.

View monitoring content

Custom monitoring via KubeSphere

KubeSphere's monitoring engine is based on Prometheus and Prometheus Operator. Use KubeSphere's custom monitoring function to support the monitoring of RadonDB MySQL indicators in a visual form.

  • In the same cluster project, select monitoring alarm - > Custom monitoring - > create;
  • In the dialog box, select MySQL template and continue to configure the monitoring template;

  • Click Save template to create a new monitoring panel.

  • The new monitoring panel needs to wait about ten minutes to view the monitoring data.

For more details, please see KubeSphere custom monitoring introduction [6] and visual monitoring [7].

Via Prometheus + Grafana platform

Grafana is a cross platform, open source data visualization network application platform. The basic principle of monitoring through Prometheus + Grafana platform is as follows:

  • Through MySQL_ The exporter obtains the monitoring data of RadonDB MySQL service
  • Via node_ The exporter obtains the monitoring data of RadonDB MySQL server.
  • After the monitoring data is transmitted to Prometheus, by configuring the data source, rich monitoring data charts and warnings are finally presented in Grafana.

For more instructions on Grafana visual monitoring, see Grafana Dashboards[8].

Reference reference

  1. KubeSphere: https://kubesphere.com.cn
  2. Prometheus: https://prometheus.io/
  3. Grafana: https://grafana.org/
  4. RadonDB MySQL Kubernetes 2.1.3: https://radondb.com/projects/mysql/
  5. Prometheus MySQLd Exporter: https://github.com/prometheus/mysqld_exporter
  6. Custom monitoring: https://kubesphere.io/zh/docs/project-user-guide/custom-application-monitoring/introduction/
  7. Visual monitoring: https://kubesphere.io/zh/docs/project-user-guide/custom-application-monitoring/visualization/overview/
  8. Grafana Dashboards: https://grafana.com/docs/grafana/latest/dashboards/

Tags: Database

Posted by Develop_Sake on Wed, 11 May 2022 09:46:01 +0300