1, Analysis and comparison of common Java static code analysis tools
Java static code analysis tool can help developers quickly and effectively locate code defects and correct these problems in time in the process of code construction, so as to greatly improve software reliability and save software development and testing costs.
Checkstyle, FindBugs and PMD are commonly used. They have their own emphasis. At present, PMD integration with Alibaba's p3c is more popular.
PMD is a java program code checking tool published by BSD protocol. The tool can check whether the Java code contains unused variables, empty grab blocks, unnecessary objects, etc. The software has powerful functions and high scanning efficiency. It is a good helper for Java programmers to debug.
PMD comes with many rules that can be used directly. Many problems of Java source program can be found by using these rules. The common types are as follows:
①Latent bug: Empty try/catch/finally/switch sentence ②Unused Code: unused local variables, parameters, private methods, etc ③Optional Code: String/StringBuffer Abuse of ④Complex expressions: not required if Statement, can use while Cycle complete for loop ⑤Duplicate code: copy/Pasting code means copying/paste bugs ⑥Create new objects from a loop: try not to for or while Instantiate a new object in the loop ⑦Resource shutdown: Connect，Result，Statement Make sure to turn it off after use
In addition, users can also define their own rules to check whether Java code conforms to some specific coding specifications, such as integrating Alibaba's p3c.
reference resources: Static analysis tool PMD
FindBug is an open source java code checking tool that follows the GNU Public License Agreement. It can check Java classes or JAR files, running Java bytecode instead of source code. The inspection principle is to compare bytecode with a set of defect patterns to find possible problems, including null pointer reference, infinite recursive loop, deadlock, etc. The types of bugs checked include:
Malicious code vulnerability: Malicious code Dodgy code: Non compliant code Internationalization: Internationalization related problems, such as incorrect string conversion; Bad practice: Bad practice:Common code errors,Serialization error,Used for defect pattern matching during static code checking; Multithreaded correctness: Correctness of multithreading:Such as common synchronization in multithreaded programming,Thread scheduling problem; Performance: Runtime performance issues, such as defined by variables,Code inefficiency caused by method calls. Correctness: Codes that may cause errors,Such as null pointer reference, etc; Experimental: Possible malicious attacks,Such as the definition of access right modifier, etc; Security: Security
As a plug-in for checking code specifications, CheckStyle can not only use the default given development specifications, such as Sun's and Google's development specifications, but also import plug-ins like Alibaba's development specifications. In fact, each company has different development specification requirements, so most companies will give their own check specification and generally import the given CheckStyle XML file.
Javadoc Note: check the of classes and methods Javadoc notes Naming convention: check whether the naming conforms to the naming convention Title: check whether the file starts with some lines Import Statements: checking Import Whether the statement conforms to the definition specification Code block size, that is, the number of lines of code blocks such as checking classes and methods Blank: check the blank character, such as tab，Carriage return, etc Modifier: inspection of modifier symbols, such as the definition order of modifiers Block: check for empty or invalid blocks Code problem: check the duplicate code, condition judgment, magic number and other problems Class design: check whether the class definition conforms to the specification, such as the definition of constructor
2, Static code analysis plug-in integration
Usually, these tools provide IDE plug-ins, maven plug-ins, jenkins plug-ins and sonarQube plug-ins.
In other words, Checkstyle, FindBugs and PMD include:
Eclipse and Idea have corresponding plug-ins for developers' native code inspection;
There are also corresponding plug-ins for jenkins and maven, which can generate test reports for automatic construction and avoid developers uploading code without local inspection (Maven was used to generate reports before sonarQube came out);
For sonarQube, the code management platform can be used for developers to check the code or integrated into the automatic construction platform to generate inspection reports.
Note: in order to facilitate the unification in the later automatic construction, the same analysis tool should be used in the transaction development, automatic construction and other links of a project.
The company generally involves two links:
1. Development link
1) Developers are required to use a unified IDE (eclipse, Idea) and then install a unified static code checking tool: for example, checkStyle checks the code specifications required by the company, PMD or findbugs checks the potential problem code;
2) After developers complete the development of functions, use these IDE integration plug-ins to check the code and correct the problems detected by the plug-ins;
3) The version Library (git or svn) to which the code is submitted;
2. Automatic construction link
The company builds the full amount of code every day. jenkins should install the corresponding Checkstyle, FindBugs and PMD plug-ins in advance:
1) jenkins pulls the source code from the trunk or branch of the version library;
2) Compile with maven plug-in;
3) Check the code of Checkstyle, FindBugs and PMD plug-in through maven. If the corresponding problem is detected, the construction fails, and the submitter of the problem and problem code sends an email to inform relevant personnel of modification.
In addition to using these plug-ins in the above two links, sonarQube code quality management platform is currently used for inspection. In addition to its own inspection rules, sonarQube can also integrate Checkstyle, FindBugs and PMD plug-ins, and then generate inspection reports.
SonarQube is a code quality data reporting tool and a code quality management platform. Compared with Checkstyle, FindBugs and PMD, it has a better graphical interface and can query the problems that are difficult to locate by other software.
SonarQube can detect code quality from the following seven dimensions:
(1)Non compliance with code standards: SonarQube Can pass PMD,CheckStyle,Findbugs And other code rule detection tools to standardize code writing. (2)Potential defects: SonarQube Can pass PMD,CheckStyle,Findbugs The code rule detection tool detects potential defects. (3)Poor complexity distribution: files, classes, methods, etc. if the complexity is too high, it will be difficult to change, which will make it difficult for developers to understand them, And without automated unit testing, changes to any component in the program may lead to the need for comprehensive regression testing. (4)Repetition: obviously, the program contains a lot of copy and paste code, which is of low quality, sonar You can show the serious repetition in the source code. (5)Insufficient or too many comments: No comments will make the readability of the code worse, especially when there are inevitable personnel changes, the readability of the program will be greatly reduced, and too many comments will make developers spend too much energy on reading comments, which is also contrary to the original intention. (6)Lack of unit tests: SonarQube It is convenient to count and display the unit test coverage. (7)Bad design: Pass SonarQube You can find out the loop, show the interdependence between packages and classes, and detect the custom architecture rules sonar Can manage third-party jar Package, can use LCOM4 Detect the application of single task rules and detect coupling.
1. SonarQube installation
SonarQube is a platform to deploy a special Server application, and then manage and view it through the interface. SonarQube Server mainly includes web Server, ElasticSearch based search Server and computing engine Server. Among them, the web Server is for developers to browse, view the code analysis results, and configure them accordingly. The computing engine Server mainly processes the code analysis report and stores it in the database. The SonarQube database is used to store configuration information and code analysis reports.
There are no authoritative books about SonarQube. You can only view the documents on the official website: https://docs.sonarqube.org/ Select the corresponding version to view it.
(1) About installing jdk, mysql and maven versions
mysql: requirements > = 5.6 & < 8.0. The default is H2 embedded memory type database, which is used to save metadata information. For long-term management, a persistent database must be used. mysql is selected here. Of course, other databases can also be selected.
(2) Download the SonarQube CE Version (charged for other versions): http://www.sonarqube.org/downloads/
SonarQube-7.8 relies on Java 8 and supports the last version of mysql management metadata. Later versions require jdk11 and do not support mysql management configuration metadata.
[root@node106 src]# unzip sonarqube-7.8.zip -d /usr/local
Modify to mysql database, url is the database connection address, username is the database user name, JDBC Password is the database password and login is the login name of sonarqube Password is the password of sonarqube
[root@node106 conf]# vim /usr/local/sonarqube-7.8/conf/sonar.properties sonar.jdbc.url=jdbc:mysql://192.168.0.141:3306/qjfsonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance sonar.jdbc.username=gmsd sonar.jdbc.password=gmsdtrade sonar.sorceEncoding=UTF-8 sonar.login=admin sonar.password=admin
#The default port is sonar web. Port = 9000, modifiable
[root@node106 linux-x86-64]# /usr/local/sonarqube-7.8/bin/linux-x86-64/sonar.sh start Starting SonarQube... Started SonarQube.
1),org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
There is an elastic search component in sonarqube. ES cannot be started by using root. It can be started by creating a new ordinary user.
2), bootstrap checks failed
: max file descriptors  for elasticsearch process is too low, increase to at least 
: max virtual memory areas vm.max_map_count  is too low, increase to at least 
Modify / etc / security / limits Add the following content after conf:
[root@node106 conf]# vim /etc/security/limits.conf * soft nofile 65536 * hard nofile 65536 * soft memlock unlimited * hard memlock unlimited
3), bootstrap checks failed
: max virtual memory areas vm.max_map_count  is too low, increase to at least 
#Modify / etc / sysctl conf [root@node106 logs]# vim /etc/sysctl.conf [root@node106 logs]# sysctl -p vm.max_map_count = 655360
#Note that the modification can only take effect after re login
4),java.lang.IllegalStateException: Fail to connect to database
Modify mysql to allow access to the host where sonarQube is located
grant all privileges on *.* to root@'%' identified by "123456";
Full startup takes a long time:
(6) Install Chinese plugin
Enter the Marketplace to find the plug-in and install it
At present, there are Chinese plug-ins matching the latest version 8.4 in the plug-in market, which do not match the version 7.8 I installed. An error will be reported when installing:
We want to download the Chinese plug-in sonar-l10n-zh-plugin-1.28 of the corresponding version of 7.8 Jar, address: https://github.com/SonarQubeCommunity/sonar-l10n-zh/releases
Then put it into the sonarqube-7.8/extensions/plugins directory
Restart sonarQube after installation:
See official website: Analyzing Source Code
SonarQube is divided into local analysis and remote analysis according to whether the code is local or hosted:
There are three kinds of clients corresponding to local analysis: Sonar scanner, maven(gradle) plug-in, IDE plug-in sonarLint, and Ant Task, which are less used now.
The corresponding client of remote analysis is the sonar scanner plug-in integrated into jenkins;
(1) Local analysis
The first client: Sonar scanner mode
SonarQube is the server for code checking and provides a visual interface; The sonar scanner is the client, which analyzes and sends the analysis report to the server, that is, the traditional C/S relationship.
When downloading a project in a language other than PHP or Python Scala + +, use one of the following languages: https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/ )Here, we use sonar-scanner-cli-188.8.131.523-windows zip. In the early version of sonar scanner, it is called sonar runner. Note that if the version is too low, an error will be reported: Fail to download libraries from server.
Here, we install the SonarQube server on 192.168.118.106 linux, then install the sonar scanner on 192.168.0.141 win7, and then use the sonar scanner to scan the code in the project E:\srcs\mybatis-3-mybatis-3.5.4 on win7.
(1) Unzip sonar-scanner-cli-184.108.40.2063-windows zip
(2) Configure and connect to the same database of SonarQube D: \ sonar scanner cli-220.127.116.113 \ conf \ sonar runner properties
#Configure here general information about the environment, such as SonarQube server connection details for example #No information about specific project should appear here #-----Default sonarqube server sonar.host.url=http://192.168.118.106:9000 #----- Default source code encoding sonar.sourceEncoding=UTF-8
(3) Setting environment variables
(4) Example project mybatis-3-mybatis-3.5.4
Under the root directory of the project for code analysis, create a new sonar project Properties file
#-----Global database settings (not used for SonarQube 5.2 +) database user and password sonar.jdbc.username=root sonar.jdbc.password=123456 #-----MySQL MySQL database connection should use the same database as the SonarQube server sonar.jdbc.url=jdbc:mysql://192.168.0.141:3306/sonar?useUnicode=true&characterEncoding=utf8 sonar.login=admin sonar.password=admin # The project key is guaranteed to be unique sonar.projectKey=mybatis-3-mybatis-3.5.4 # Project name sonar.projectName=mybatis-3-mybatis-3.5.4 sonar.projectVersion=3.5.4 sonar.language=java sonar.modules=java-module # sonar.sources is the directory where the source files are located. Note that if there are css files below, you will be required to install node js java-module.sonar.sources=src/main/java java-module.sonar.projectBaseDir=. # The specified class directory starts from sonarQube 4.12. Sonar will conduct dynamic check of the program and do not configure sonar java. The binaries property will fail sonar.java.binaries=target/classes # Encoding of the source code. Default is default system encoding #sonar.sourceEncoding=UTF-8
(5) Then open the command running window in this directory and execute the sonar scanner command. After the analysis is successful, the following is displayed:
Go to the sonarQube interface to view
The second method: use maven and gradle scanning methods
Here we show two ways of integration using maven. The premise project is maven project.
(1) Modify $MAVEN_HOME/conf/settings.xml
<?xml version="1.0" encoding="UTF-8"?> <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd"> <localRepository>E:\workspace\repository</localRepository> <pluginGroups> <pluginGroup>org.sonarsource.scanner.maven</pluginGroup> </pluginGroups> <proxies> </proxies> <servers> <server> <id>releases</id> <username>admin</username> <password>Aa123456</password> </server> <server> <id>snapshots</id> <username>admin</username> <password>Aa123456</password> </server> </servers> <mirrors> <mirror> <id>alimaven</id> <name>aliyun maven</name> <url>http://maven.aliyun.com/nexus/content/groups/public/</url> <mirrorOf>central</mirrorOf> </mirror> </mirrors> <profiles> <profile> <id>sonar</id> <activation> <activeByDefault>true</activeByDefault> </activation> <properties> <sonar.host.url> http://192.168.118.106:9000 </sonar.host.url> </properties> </profile> </profiles> </settings>
(2) Go to the root directory of Maven project and execute the following command to analyze the project using SonarQube
Method 1: directly contain POM XML directory Execution Analysis
mvn clean verify sonar:sonar -Dmaven.test.skip=true
Method 2: in the case of multiple modules, execute install first to ensure that the code is up-to-date
mvn clean install -Dmaven.test.skip=true
#Then perform the analysis. Note that you need to add - dsonar java. Binaries parameter
Method 3: specify the plug-in version of sonar
Suggestions can be found in the POM XML specifies the version of the sonar Maven plugin plug-in
<build> <pluginManagement> <plugins> <plugin> <groupId>org.sonarsource.scanner.maven</groupId> <artifactId>sonar-maven-plugin</artifactId> <version>18.104.22.1688</version> </plugin> </plugins> </pluginManagement> </build>
The third way: use the IDE plug-in sonarLint
Take idea using sonarLint as an example. Note that the inspection report in this way is intended to be viewed in idea, but not in sonaQube
(1) Install plug-ins
Search for sonarLint in the plug-in and install
(2)sonarLint global configuration sonarQube server
(3) Project settings
Configure the project information in the sonarlint -- > project setting configuration tab, check the Binding project to SonarQube / SonarCloud, select the service link just configured in the Connection drop-down box, click Search in list in the project option, select the corresponding project, and click OK;
Note that this method is to query the project, and there is sonar project Properties project
(4) View report after scan
Mode of use (II)
(1) Right click the project for which code analysis is required, and select AnaLyze - > AnaLyze with SonarLint (or click the AnaLyze All Project Files Icon in the SonarLint Report tab) and click Advanced in the pop-up dialog box to wait for the completion of project code analysis;
(2) After the code analysis is completed, the code analysis results will be displayed in the "SonarQube Report" tab. You can expand a single file to display the problems in the file. Click a result to see the detailed description of the problem in the Rule tab on the right. Double click a record to quickly jump to the location of the code corresponding to the result;
(3) The sonarlint plug-in automatically checks the code by default, but in order to ensure the real-time and effectiveness of the code check results, we analyze the code after completing a certain amount of code.
(II) remote analysis - integration of jenkins
3. User Interface tutorial
reference resources: Introduction to SonarQube