Continuous integration: get Robot variables returned by Robot Framework plug-in in Jenkins

stay Jenkins integrated Robot Framework This paper introduces how Jenkins integrates the Robot Framework test framework. This paper introduces how to display the test results in the email report (such as pass rate, test report link and other parameters).

The Robot Framework plug-in returned the following parameters:

  • ${ROBOT_FAILEDCASES} - list of failed cases
  • ${robot_passercentage, onlyCritical} - pass rate (passed / total * 100%), onlyCritical defaults to false.
  • ${robot_pasratio, onlyCritical} - 'passed / total' format results. onlyCritical defaults to false.
  • ${ROBOT_FAILED} - number of failed cases
  • ${ROBOT_PASSED} - number of cases passed
  • ${ROBOT_TOTAL} - total use cases
  • ${ROBOT_REPORTLINK} - log.html file link.

To use these parameters in Mail reports, you also need to install Email Extension and Token Macro These two plug-ins.

Mail direct call

After the plug-in is installed, you can directly call the above parameters in the HTML template:

<hr/>(Automatically build email without reply!)<br/><hr/>

FAIL Case: $ROBOT_FAILEDCASES<br/><br/>

passed / total: $ROBOT_PASSRATIO<br/><br/>

Passed: ${ROBOT_PASSED}<br/><br/>

Failed: ${ROBOT_FAILED}<br/><br/>

Total: ${ROBOT_TOTAL}<br/><br/>

PASSPERCENTAGE: $ROBOT_PASSPERCENTAGE<br/><br/>

Report URL: <a href="${ROBOT_REPORTLINK}">${ROBOT_REPORTLINK}</a><br/><br/
<hr/>

The pipeline script (scripted pipeline syntax) for executing the Robot framework automation test is as follows:

node('win_agent') { //Test agent node
    try {
        // automated testing  
        stage('Test') {
            TEST_STATU = bat (
                script: '''robot -d %WORKSPACE%/rf_results D:/Testcases''',
                returnStatus: true
            ) == 0
        }
        
        stage('Test Report') {
            script {
              step(
                    [
                      $class              : 'RobotPublisher',
                      outputPath          : 'rf_results',
                      outputFileName      : 'output.xml',
                      reportFileName      : 'report.html',
                      logFileName         : 'log.html',
                      otherFiles          : '*.png,*.jpg',
                      disableArchiveOutput: true,
                      passThreshold       : 100,
                      unstableThreshold   : 100,
                    ]
              )
            }
        }
        
    } catch (error) {
        currentBuild.result = 'FAILURE'
        throw error        
    } finally {
        stage("email") {
            emailext (
                subject: '\'Build notification:${PROJECT_NAME} - Build # ${BUILD_NUMBER} - ${BUILD_STATUS}\'',
                to: "XXXXXXXXX@qq.com", 
                body: '${FILE,path="email.html"}',
                )
        }
    }
}


The implementation effect is as follows:

Groovy script template call

Email Extension plug-in supports HTML Groovy script template, so how to display test results in HTML Groovy script template?

Two schemes are introduced below.

Scenario 1: save Robot variables

You can first save the Robot variable to an xml file, and then parse the generated xml file in the template file.

The specific implementation ideas are introduced below.

1. Save Robot variables to environment variables

The script is as follows:

stage('Test Report') {
    script {
        step(
            [
                $class              : 'RobotPublisher',
                outputPath          : 'rf_results',
                outputFileName      : 'output.xml',
                reportFileName      : 'report.html',
                logFileName         : 'log.html',
                otherFiles          : '*.png,*.jpg',
                disableArchiveOutput: true,
                passThreshold       : 100,
                unstableThreshold   : 100,
            ]
        )
    }

    script {
        // Save test results to environment variables
        env.ROBOT_FAILEDCASES = tm('${ROBOT_FAILEDCASES}')
        env.ROBOT_PASSRATIO = tm('${ROBOT_PASSRATIO}')
        env.ROBOT_PASSED = tm('${ROBOT_PASSED}')
        env.ROBOT_FAILED = tm('${ROBOT_FAILED}')
        env.ROBOT_TOTAL = tm('${ROBOT_TOTAL}')
        env.ROBOT_PASSPERCENTAGE = tm('${ROBOT_PASSPERCENTAGE}')                
        env.ROBOT_REPORTLINK = tm('${ROBOT_REPORTLINK}')
    }
}

The method provided by the macro plugin is tm.

2. Save variables to an xml file

You can use groovy script to write the method of saving environment variables. In order to make the code more concise, put the groovy script into the pipeline shared library.

def saveEnvToXml() {
	def jobVariables = currentBuild.getBuildVariables();
	println "jobVariables:${jobVariables}"; 
	println "ROBOT_FAILEDCASES: ${ROBOT_FAILEDCASES}";  
	println "ROBOT_PASSRATIO: ${ROBOT_PASSRATIO}";  
	println "ROBOT_PASSED: ${ROBOT_PASSED}";  
	println "ROBOT_FAILED: ${ROBOT_FAILED}";  
	println "ROBOT_TOTAL: ${ROBOT_TOTAL}";  
	println "ROBOT_PASSPERCENTAGE: ${ROBOT_PASSPERCENTAGE}";               
	println "ROBOT_REPORTLINK: ${ROBOT_REPORTLINK}";  
}

In the saveEnvToXml() method, the code for saving variables to xml files is omitted here, and the implementation idea is mainly introduced.

The execution results are as follows:

3,groovy-html. Parse the xml file in the template file and read the Robot variable.

The template file is written in groovy+HTML syntax. Groovy is used to parse the xml file, and HTML shows the results.

The complete pipeline script is as follows:

@Library('pipelinelibs@1.0') _   // SVN Library
import com.hiyongz.MyLib
def mylib = new MyLib();

node('win_agent') { //Test agent node
    env.JOB_BASE_NAME = "${JOB_BASE_NAME}"
    env.JOB_URL = "${JOB_URL}"
    
    try {
        // automated testing  
        stage('Test') {
            TEST_STATU = bat (
                script: '''robot -d %WORKSPACE%/rf_results D:/Testcases''',
                returnStatus: true
            ) == 0
        }
        
        stage('Test Report') {
            script {
              step(
                    [
                      $class              : 'RobotPublisher',
                      outputPath          : 'rf_results',
                      outputFileName      : 'output.xml',
                      reportFileName      : 'report.html',
                      logFileName         : 'log.html',
                      otherFiles          : '*.png,*.jpg',
                      disableArchiveOutput: true,
                      passThreshold       : 100,
                      unstableThreshold   : 100,
                    ]
              )
            }           
            script {
                // Save test results to environment variables
                env.ROBOT_FAILEDCASES = tm('${ROBOT_FAILEDCASES}')
                env.ROBOT_PASSRATIO = tm('${ROBOT_PASSRATIO}')
                env.ROBOT_PASSED = tm('${ROBOT_PASSED}')
                env.ROBOT_FAILED = tm('${ROBOT_FAILED}')
                env.ROBOT_TOTAL = tm('${ROBOT_TOTAL}')
                env.ROBOT_PASSPERCENTAGE = tm('${ROBOT_PASSPERCENTAGE}')       
                env.ROBOT_REPORTLINK = tm('${ROBOT_REPORTLINK}')
                
            }
            script {
                mylib.saveEnvToXml();
            }
        }
        
    } catch (error) {
        currentBuild.result = 'FAILURE'
        throw error        
    } finally {
        stage("email") {
            emailext (
                subject: '\'Build notification:${PROJECT_NAME} - Build # ${BUILD_NUMBER} - ${BUILD_STATUS}\'',
                to: "XXXXXXXXX@qq.com", 
                body: '''${SCRIPT,template="groovy-html.template"}''',
                )
        }
    }
}

Scheme 2: Jenkins API reading results

You can also directly write groovy scripts in the template file to parse the output generated by the Robot Framework XML file to obtain the test results. The disadvantage of this method is output XML files can be large and complex to process.

A simpler method is to use Jenkins API to read test results. jenkins supports the following three ways:

For more information about Jenkins API, please refer to Continuous integration: detailed introduction to Jenkins API usage , here is the usage of xml api.

Enter the Robot Results interface, add / api/xml after the url, and the following xml content will be returned:

You can see the test results of the robot framework, and we can extract the results from this API. You can further use XPath syntax to obtain the desired value and obtain the pass rate passpercentage:/ api/xml?xpath=/*/passPercentage:

In the automation script, you can use the curl command to obtain the pass rate passPercentage:

$ curl -k --silent -L --user user name:API Token http://192.168.30.8:8080/view/RF/job/RF-TMVariables-demo/26/robot/api/xml?xpath=/*/passPercentage | tr '<' '\n' | egrep '^passPercentage>' | sed 's/.*>//g'
100.0

groovy script:

def WORKFLOW_API_XML = sh ( script: "API_XML=\$(curl -k --silent -L --user user name:API Token http://192.168.30.8:8080/view/RF/job/RF-TMVariables-demo/26/robot/api/xml?xpath=/*/passPercentage | tr '<' '\n' | egrep '^passPercentage>' |  sed 's/.*>//g'); echo \${API_XML}", returnStdout: true ).trim();

Using json api will return test results in json format. After obtaining these results, there are many methods to extract the value you want.

Reference documents:

  1. https://plugins.jenkins.io/robot/
  2. https://plugins.jenkins.io/email-ext/
  3. https://stackoverflow.com/questions/54751520/how-to-output-test-pass-failure-notification-to-slack-through-jenkins-when-runni
--THE END--

Forget food in anger and worry in joy. I don't know how old I am—— The Analects of Confucius

Tags: jenkins

Posted by hrosas on Sun, 03 Apr 2022 06:13:17 +0300