[Jenkins 7] static code analysis of code quality

1, Analysis and comparison of common Java static code analysis tools
Java static code analysis tool can help developers quickly and effectively locate code defects and correct these problems in time in the process of code construction, so as to greatly improve software reliability and save software development and testing costs.
Checkstyle, FindBugs and PMD are commonly used. They have their own emphasis. At present, PMD integration with Alibaba's p3c is more popular.

 

1,PMD
PMD is a java program code checking tool published by BSD protocol. The tool can check whether the Java code contains unused variables, empty grab blocks, unnecessary objects, etc. The software has powerful functions and high scanning efficiency. It is a good helper for Java programmers to debug.
PMD comes with many rules that can be used directly. Many problems of Java source program can be found by using these rules. The common types are as follows:

①Latent bug: Empty try/catch/finally/switch sentence
②Unused Code: unused local variables, parameters, private methods, etc
③Optional Code: String/StringBuffer Abuse of
④Complex expressions: not required if Statement, can use while Cycle complete for loop
⑤Duplicate code: copy/Pasting code means copying/paste bugs
⑥Create new objects from a loop: try not to for or while Instantiate a new object in the loop
⑦Resource shutdown: Connect,Result,Statement Make sure to turn it off after use

In addition, users can also define their own rules to check whether Java code conforms to some specific coding specifications, such as integrating Alibaba's p3c.

reference resources: Static analysis tool PMD

2,FindBugs

FindBug is an open source java code checking tool that follows the GNU Public License Agreement. It can check Java classes or JAR files, running Java bytecode instead of source code. The inspection principle is to compare bytecode with a set of defect patterns to find possible problems, including null pointer reference, infinite recursive loop, deadlock, etc. The types of bugs checked include:

Malicious code vulnerability: Malicious code
Dodgy code: Non compliant code
Internationalization: Internationalization related problems, such as incorrect string conversion;
Bad practice: Bad practice:Common code errors,Serialization error,Used for defect pattern matching during static code checking;
Multithreaded correctness: Correctness of multithreading:Such as common synchronization in multithreaded programming,Thread scheduling problem;
Performance: Runtime performance issues, such as defined by variables,Code inefficiency caused by method calls.
Correctness: Codes that may cause errors,Such as null pointer reference, etc;
Experimental: Possible malicious attacks,Such as the definition of access right modifier, etc;
Security: Security

reference resources:

Use the findbugs idea plug-in to find potential bugs in your code

3,CheckStyle
As a plug-in for checking code specifications, CheckStyle can not only use the default given development specifications, such as Sun's and Google's development specifications, but also import plug-ins like Alibaba's development specifications. In fact, each company has different development specification requirements, so most companies will give their own check specification and generally import the given CheckStyle XML file.

 

Javadoc Note: check the of classes and methods Javadoc notes
 Naming convention: check whether the naming conforms to the naming convention
 Title: check whether the file starts with some lines
Import Statements: checking Import Whether the statement conforms to the definition specification
 Code block size, that is, the number of lines of code blocks such as checking classes and methods
 Blank: check the blank character, such as tab,Carriage return, etc
 Modifier: inspection of modifier symbols, such as the definition order of modifiers
 Block: check for empty or invalid blocks
 Code problem: check the duplicate code, condition judgment, magic number and other problems
 Class design: check whether the class definition conforms to the specification, such as the definition of constructor

reference resources:

CheckStyle use

Installing and using CheckStyle in Eclipse

 

4. Distinction

reference resources:

Analysis and comparison of common Java static code analysis tools

Comparison of code quality detection tools FindBugs, PMD and CheckStyle

2, Static code analysis plug-in integration
Usually, these tools provide IDE plug-ins, maven plug-ins, jenkins plug-ins and sonarQube plug-ins.
In other words, Checkstyle, FindBugs and PMD include:
Eclipse and Idea have corresponding plug-ins for developers' native code inspection;
There are also corresponding plug-ins for jenkins and maven, which can generate test reports for automatic construction and avoid developers uploading code without local inspection (Maven was used to generate reports before sonarQube came out);
For sonarQube, the code management platform can be used for developers to check the code or integrated into the automatic construction platform to generate inspection reports.
Note: in order to facilitate the unification in the later automatic construction, the same analysis tool should be used in the transaction development, automatic construction and other links of a project.

The company generally involves two links:

1. Development link

1) Developers are required to use a unified IDE (eclipse, Idea) and then install a unified static code checking tool: for example, checkStyle checks the code specifications required by the company, PMD or findbugs checks the potential problem code;

2) After developers complete the development of functions, use these IDE integration plug-ins to check the code and correct the problems detected by the plug-ins;

3) The version Library (git or svn) to which the code is submitted;

2. Automatic construction link

The company builds the full amount of code every day. jenkins should install the corresponding Checkstyle, FindBugs and PMD plug-ins in advance:

1) jenkins pulls the source code from the trunk or branch of the version library;

2) Compile with maven plug-in;

3) Check the code of Checkstyle, FindBugs and PMD plug-in through maven. If the corresponding problem is detected, the construction fails, and the submitter of the problem and problem code sends an email to inform relevant personnel of modification.

In addition to using these plug-ins in the above two links, sonarQube code quality management platform is currently used for inspection. In addition to its own inspection rules, sonarQube can also integrate Checkstyle, FindBugs and PMD plug-ins, and then generate inspection reports.

reference resources:

eclipse integrated Ali code specification tool p3c

Installing and using CheckStyle in Eclipse

Eclipse installs the FindBugs plug-in

Configure findbugs, pmd and checkstyle under Jenkins to realize automatic code detection 

3, SonarQube

SonarQube is a code quality data reporting tool and a code quality management platform. Compared with Checkstyle, FindBugs and PMD, it has a better graphical interface and can query the problems that are difficult to locate by other software.

SonarQube can detect code quality from the following seven dimensions:

(1)Non compliance with code standards: SonarQube Can pass PMD,CheckStyle,Findbugs And other code rule detection tools to standardize code writing.
(2)Potential defects: SonarQube Can pass PMD,CheckStyle,Findbugs The code rule detection tool detects potential defects.
(3)Poor complexity distribution: files, classes, methods, etc. if the complexity is too high, it will be difficult to change, which will make it difficult for developers to understand them, And without automated unit testing, changes to any component in the program may lead to the need for comprehensive regression testing.
(4)Repetition: obviously, the program contains a lot of copy and paste code, which is of low quality, sonar You can show the serious repetition in the source code.
(5)Insufficient or too many comments: No comments will make the readability of the code worse, especially when there are inevitable personnel changes, the readability of the program will be greatly reduced, and too many comments will make developers spend too much energy on reading comments, which is also contrary to the original intention.
(6)Lack of unit tests: SonarQube It is convenient to count and display the unit test coverage.
(7)Bad design: Pass SonarQube You can find out the loop, show the interdependence between packages and classes, and detect the custom architecture rules sonar Can manage third-party jar Package, can use LCOM4 Detect the application of single task rules and detect coupling.

1. SonarQube installation
SonarQube is a platform to deploy a special Server application, and then manage and view it through the interface. SonarQube Server mainly includes web Server, ElasticSearch based search Server and computing engine Server. Among them, the web Server is for developers to browse, view the code analysis results, and configure them accordingly. The computing engine Server mainly processes the code analysis report and stores it in the database. The SonarQube database is used to store configuration information and code analysis reports.

There are no authoritative books about SonarQube. You can only view the documents on the official website: https://docs.sonarqube.org/ Select the corresponding version to view it.

(1) About installing jdk, mysql and maven versions

jdk: jdk1.8
mysql: requirements > = 5.6 & < 8.0. The default is H2 embedded memory type database, which is used to save metadata information. For long-term management, a persistent database must be used. mysql is selected here. Of course, other databases can also be selected.
maven: 3.6.3

(2) Download the SonarQube CE Version (charged for other versions): http://www.sonarqube.org/downloads/
SonarQube-7.8 relies on Java 8 and supports the last version of mysql management metadata. Later versions require jdk11 and do not support mysql management configuration metadata.

(3) Decompress
[root@node106 src]# unzip sonarqube-7.8.zip -d /usr/local

(4) Disposition
Modify to mysql database, url is the database connection address, username is the database user name, JDBC Password is the database password and login is the login name of sonarqube Password is the password of sonarqube

[root@node106 conf]# vim /usr/local/sonarqube-7.8/conf/sonar.properties
sonar.jdbc.url=jdbc:mysql://192.168.0.141:3306/qjfsonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance
sonar.jdbc.username=gmsd
sonar.jdbc.password=gmsdtrade
sonar.sorceEncoding=UTF-8
sonar.login=admin
sonar.password=admin

(5) Start

#The default port is sonar web. Port = 9000, modifiable

[root@node106 linux-x86-64]# /usr/local/sonarqube-7.8/bin/linux-x86-64/sonar.sh start
Starting SonarQube...
Started SonarQube.

report errors:

1),org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
There is an elastic search component in sonarqube. ES cannot be started by using root. It can be started by creating a new ordinary user.

2),[2] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
Modify / etc / security / limits Add the following content after conf:

[root@node106 conf]# vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft memlock unlimited
* hard memlock unlimited

 

3),[1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65536] is too low, increase to at least [262144]

#Modify / etc / sysctl conf
[root@node106 logs]# vim /etc/sysctl.conf
[root@node106 logs]# sysctl -p
vm.max_map_count = 655360

#Note that the modification can only take effect after re login

4),java.lang.IllegalStateException: Fail to connect to database
Modify mysql to allow access to the host where sonarQube is located
grant all privileges on *.* to root@'%' identified by "123456";

Full startup takes a long time:

 

(6) Install Chinese plugin
Enter the Marketplace to find the plug-in and install it

At present, there are Chinese plug-ins matching the latest version 8.4 in the plug-in market, which do not match the version 7.8 I installed. An error will be reported when installing:

We want to download the Chinese plug-in sonar-l10n-zh-plugin-1.28 of the corresponding version of 7.8 Jar, address: https://github.com/SonarQubeCommunity/sonar-l10n-zh/releases
Then put it into the sonarqube-7.8/extensions/plugins directory

[root@node106 plugins]# pwd
/usr/local/sonarqube-7.8/extensions/plugins
[root@node106 plugins]# ll
total 88032
-rw-r--r-- 1 sonar sonar 224 Jun 17 2019 README.txt
-rw-r--r-- 1 sonar sonar 287504 Jun 17 2019 sonar-auth-github-plugin-1.5.0.870.jar
-rw-r--r-- 1 sonar sonar 3388540 Jun 17 2019 sonar-auth-saml-plugin-1.1.0.181.jar
-rw-r--r-- 1 sonar sonar 4092977 Jun 17 2019 sonar-csharp-plugin-7.14.0.8411.jar
-rw-r--r-- 1 sonar sonar 7016445 Jun 17 2019 sonar-css-plugin-1.1.0.993.jar
-rw-r--r-- 1 sonar sonar 1551459 Jun 17 2019 sonar-flex-plugin-2.4.0.1222.jar
-rw-r--r-- 1 sonar sonar 3903342 Jun 17 2019 sonar-go-plugin-1.1.1.2000.jar
-rw-r--r-- 1 sonar sonar 1727846 Jun 17 2019 sonar-html-plugin-3.1.0.1615.jar
-rw-r--r-- 1 sonar sonar 14629 Jun 17 2019 sonar-jacoco-plugin-1.0.1.143.jar
-rw-r--r-- 1 sonar sonar 8302512 Jun 17 2019 sonar-java-plugin-5.13.0.18197.jar
-rw-r--r-- 1 sonar sonar 6866969 Jun 17 2019 sonar-javascript-plugin-5.2.1.7778.jar
-rw-r--r-- 1 sonar sonar 7595999 Jun 17 2019 sonar-kotlin-plugin-1.5.0.315.jar
-rw-r--r-- 1 sonar sonar 47581 Sep 8 09:53 sonar-l10n-zh-plugin-1.28.jar
-rw-r--r-- 1 sonar sonar 300503 Jun 17 2019 sonar-ldap-plugin-2.2.0.608.jar
-rw-r--r-- 1 sonar sonar 5107348 Jun 17 2019 sonar-php-plugin-3.1.1.4762.jar
-rw-r--r-- 1 sonar sonar 2752193 Jun 17 2019 sonar-python-plugin-1.14.0.3086.jar
-rw-r--r-- 1 sonar sonar 10036210 Jun 17 2019 sonar-ruby-plugin-1.5.0.315.jar
-rw-r--r-- 1 sonar sonar 9202024 Jun 17 2019 sonar-scala-plugin-1.5.0.315.jar
-rw-r--r-- 1 sonar sonar 2622236 Jun 17 2019 sonar-scm-git-plugin-1.8.0.1574.jar
-rw-r--r-- 1 sonar sonar 7229293 Jun 17 2019 sonar-scm-svn-plugin-1.9.0.1295.jar
-rw-r--r-- 1 sonar sonar 2239156 Jun 17 2019 sonar-typescript-plugin-1.9.0.3766.jar
-rw-r--r-- 1 sonar sonar 3576923 Jun 17 2019 sonar-vbnet-plugin-7.14.0.8411.jar
-rw-r--r-- 1 sonar sonar 2242738 Jun 17 2019 sonar-xml-plugin-2.0.1.2020.jar

Restart sonarQube after installation:

 

2. Client

See official website: Analyzing Source Code
SonarQube is divided into local analysis and remote analysis according to whether the code is local or hosted:

There are three kinds of clients corresponding to local analysis: Sonar scanner, maven(gradle) plug-in, IDE plug-in sonarLint, and Ant Task, which are less used now.

The corresponding client of remote analysis is the sonar scanner plug-in integrated into jenkins;

(1) Local analysis

The first client: Sonar scanner mode

SonarQube is the server for code checking and provides a visual interface; The sonar scanner is the client, which analyzes and sends the analysis report to the server, that is, the traditional C/S relationship.

 

When downloading a project in a language other than PHP or Python Scala + +, use one of the following languages: https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/ )Here, we use sonar-scanner-cli-4.2.0.1873-windows zip. In the early version of sonar scanner, it is called sonar runner. Note that if the version is too low, an error will be reported: Fail to download libraries from server.

Example topology:
Here, we install the SonarQube server on 192.168.118.106 linux, then install the sonar scanner on 192.168.0.141 win7, and then use the sonar scanner to scan the code in the project E:\srcs\mybatis-3-mybatis-3.5.4 on win7.
(1) Unzip sonar-scanner-cli-4.2.0.1873-windows zip

(2) Configure and connect to the same database of SonarQube D: \ sonar scanner cli-4.2.0.1873 \ conf \ sonar runner properties

#Configure here general information about the environment, such as SonarQube server connection details for example
#No information about specific project should appear here

#-----Default sonarqube server
sonar.host.url=http://192.168.118.106:9000
#----- Default source code encoding
sonar.sourceEncoding=UTF-8

(3) Setting environment variables


(4) Example project mybatis-3-mybatis-3.5.4
Under the root directory of the project for code analysis, create a new sonar project Properties file

#-----Global database settings (not used for SonarQube 5.2 +) database user and password
sonar.jdbc.username=root
sonar.jdbc.password=123456
#-----MySQL MySQL database connection should use the same database as the SonarQube server
sonar.jdbc.url=jdbc:mysql://192.168.0.141:3306/sonar?useUnicode=true&amp;characterEncoding=utf8
sonar.login=admin
sonar.password=admin

# The project key is guaranteed to be unique
sonar.projectKey=mybatis-3-mybatis-3.5.4
# Project name
sonar.projectName=mybatis-3-mybatis-3.5.4
sonar.projectVersion=3.5.4
sonar.language=java
sonar.modules=java-module

# sonar.sources is the directory where the source files are located. Note that if there are css files below, you will be required to install node js
java-module.sonar.sources=src/main/java
java-module.sonar.projectBaseDir=.
# The specified class directory starts from sonarQube 4.12. Sonar will conduct dynamic check of the program and do not configure sonar java. The binaries property will fail
sonar.java.binaries=target/classes
# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8

(5) Then open the command running window in this directory and execute the sonar scanner command. After the analysis is successful, the following is displayed:

Go to the sonarQube interface to view

The second method: use maven and gradle scanning methods
Here we show two ways of integration using maven. The premise project is maven project.

(1) Modify $MAVEN_HOME/conf/settings.xml

<?xml version="1.0" encoding="UTF-8"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

  <localRepository>E:\workspace\repository</localRepository>
  
  <pluginGroups>
    <pluginGroup>org.sonarsource.scanner.maven</pluginGroup>
  </pluginGroups>

  <proxies>
  </proxies>
  
  <servers>
    <server>
    <id>releases</id>
    <username>admin</username>
    <password>Aa123456</password>
    </server>
    <server>
    <id>snapshots</id>
    <username>admin</username>
    <password>Aa123456</password>
    </server>
  </servers>

  <mirrors>
    <mirror>
        <id>alimaven</id>
        <name>aliyun maven</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
        <mirrorOf>central</mirrorOf>
    </mirror>
  </mirrors>

  <profiles>
        <profile>
            <id>sonar</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties>
                <sonar.host.url>
                  http://192.168.118.106:9000
                </sonar.host.url>
            </properties>
        </profile>
  </profiles>
</settings>

 

(2) Go to the root directory of Maven project and execute the following command to analyze the project using SonarQube
Method 1: directly contain POM XML directory Execution Analysis

mvn clean verify sonar:sonar -Dmaven.test.skip=true

Method 2: in the case of multiple modules, execute install first to ensure that the code is up-to-date

mvn clean install -Dmaven.test.skip=true

#Then perform the analysis. Note that you need to add - dsonar java. Binaries parameter

mvn sonar:sonar

Method 3: specify the plug-in version of sonar

mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar

Suggestions can be found in the POM XML specifies the version of the sonar Maven plugin plug-in

<build>
  <pluginManagement>
    <plugins>
      <plugin>
        <groupId>org.sonarsource.scanner.maven</groupId>
        <artifactId>sonar-maven-plugin</artifactId>
        <version>3.6.0.1398</version>
      </plugin>
    </plugins>
  </pluginManagement>
</build>

The third way: use the IDE plug-in sonarLint

Take idea using sonarLint as an example. Note that the inspection report in this way is intended to be viewed in idea, but not in sonaQube

(1) Install plug-ins

Search for sonarLint in the plug-in and install

Usage (I)

(2)sonarLint global configuration sonarQube server

(3) Project settings

Configure the project information in the sonarlint -- > project setting configuration tab, check the Binding project to SonarQube / SonarCloud, select the service link just configured in the Connection drop-down box, click Search in list in the project option, select the corresponding project, and click OK;

Note that this method is to query the project, and there is sonar project Properties project

(4) View report after scan

Mode of use (II)

(1) Right click the project for which code analysis is required, and select AnaLyze - > AnaLyze with SonarLint (or click the AnaLyze All Project Files Icon in the SonarLint Report tab) and click Advanced in the pop-up dialog box to wait for the completion of project code analysis;
(2) After the code analysis is completed, the code analysis results will be displayed in the "SonarQube Report" tab. You can expand a single file to display the problems in the file. Click a result to see the detailed description of the problem in the Rule tab on the right. Double click a record to quickly jump to the location of the code corresponding to the result;
(3) The sonarlint plug-in automatically checks the code by default, but in order to ensure the real-time and effectiveness of the code check results, we analyze the code after completing a certain amount of code.

(II) remote analysis - integration of jenkins

 

3. User Interface tutorial

reference resources: Introduction to SonarQube

 

sonarQube reference:

Official website document
eclipse installs the sonarLint plug-in

Installation, configuration and use of SonarQube

SonarQube7.3 installation and operation instructions

sonarqube introduction + Architecture + Construction + detailed usage of different languages

Posted by korion on Mon, 16 May 2022 16:40:26 +0300