1. Introduction to allure

allure is a lightweight, flexible, multilingual test reporting tool. Not only does it present what has been tested very succinctly in the form of a neat Web report, but it allows everyone involved in the development process to extract maximum useful information from the day-to-day execution of tests.

From a development/QA perspective, allure reports shorten the life cycle of common defects: test failures can be divided into defects and broken tests, and you can also configure logs, steps, fixture s, attachments, time, history, and communication with TMS and Integration of defect tracking systems, so responsible developers and testers will have all the information.

Allure is a lightweight and very flexible open source test reporting framework. It supports most testing frameworks, such as TestNG, Pytest, JUint, etc. It's simple to use and easy to integrate into jenkins.

2. The results of the allure test report

2.1 Overview page

The entry point for each report is an "overview" page with dashboards and widgets

The overview page contains several default widgets that represent basic characteristics of the project and test environment.

• Statistics - Overall report statistics.
• Launches - If this report represents multiple test launches, statistics for each launch will be displayed here.
• Behavior - Result information aggregated by story and characteristics.
• Executor - Information about the test executor used to run the test.
• Historical Trend - If the test has accumulated some historical data, its trend will be calculated and displayed on the chart.
• Environments - Information about test environments (see How to define environments).

2.2 Categories

2.3 Kit

On the "Suites" tab, you can find a standard structured representation of executed tests grouped by suite and class.

2.4 Charts

View different statistics collected from test data: status breakdown or severity and duration graphs.

2.5 Timeline

The Timeline tab visualizes a review of the test execution, the allure adapter collects the precise timing of the tests, and on this tab they are arranged according to their sequential or parallel time structure.

2.6 Behavior

For behavior-driven approaches, this tab groups test results based on the Epic, Feature, and Story tags.

2.7 packages

The Packages tab represents a tree-like layout of test results, grouped by different packages.

After introducing the components of the allure report, let's learn how to install and use allure, and generate beautiful test reports.

3. The environment required for allure installation

• (1) java jdk environment
  • allure execution depends on the java environment. I will not demonstrate how to install j to open it here.
• (2)allure-commanline
  • Display test report
• (3) allure-pytest third-party library

4, allure-commandline installation

(1) Download address: https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/2.13.5/ (2.13.5 is used here as an example)
(2) Download allure.zip
(3) Unzip, enter the bin directory and add the bin directory to the Path environment variable

Configure environment variables as shown below:

5, allure-pytest installation

(1) Install using the command line (recommended)

pip install allure-pytest

(2) Installation using the interface
Install in settings, as shown below:

6. The usage of allure

Instructions

parameter value

Parameter Description

@allure.epic()

epic description

Define the project, when there are multiple projects to use, down is the Feature

@allure.feature()

module name

Use cases are distinguished by modules, and when there are multiple modules, give each a name

@allure.story()

use case name

Description of the use case

@allure.title()

use case title

use case title

@allure.testcase()

Use case related links

The address of the function storage system corresponding to the automation use case

@allure.issue()

defect address

Corresponding to the defect address in the defect management system

@allure.description()

use case description

Detailed description of the test case

@allure.step()

Steps

Operation steps of the test case

@allure.severity()

Use case class

blocker,critical,normal,minor,trivial

@allure.link()

define link

Used to define a connection that needs to be displayed in the test report

@allure.attach()

Add attachments

Add test report attachments such as data, text, pictures, videos, web pages

7. The command to generate the test report

(1) Collect results during test execution:

#--alluredir This option is used to specify the path to store the test results
pytest [test file] -s -q --alluredir=./result/  

(2) View the test report

Method 1: After the test is completed, check the actual report, watch the report online, and directly open the default browser to display the current report

#Pay attention to the writing of serve here
allure serve ./result/   

Method 2: Generate a report from the result, which is a service that starts tomcat. Two steps required: generate report, open report

• Generate report:

  #Note: Override path plus --clean
  allure generate ./result/ -o ./report/ --clean

• Open the report:

  #./report is the path where the report is located
  allure open -h 127.0.0.1 -p 8883 ./report/

example:

#Method 1: test_allure1.py is the module name, ./result is the report address
pytest test_allure_demo.py -vs -q --alluredir=./result
allure serve ./result/

# Method 2
pytest test_allure_demo.py -vs -q --alluredir=./result
allure generate ./result/ -o ./report/ --clean
allure open -h 127.0.0.1 -p 8883 ./report/

8. allure common features in practice

(1) @allure.feature() ,@allure.title()

Instructions

parameter value

Parameter Description

@allure.feature()

module name

Use cases are distinguished by modules, and when there are multiple modules, give each a name

@allure.title()

use case title

use case title

import allure

@allure.feature("search module")
class TestSearch():
    def test_case1(self):
        print("case1")
    def test_case2(self):
        print("case2")

Execute the generate report command:

pytest test_allure_demo.py -vs -q --alluredir=./result

The command to execute the test report service:

allure serve ./result/

(2)@allure.story(),@allure.step()

Instructions

parameter value

Parameter Description

@allure.story()

use case name

Description of the use case

@allure.step()

Steps

Operation steps of the test case

import allure

@allure.feature("login module")
class TestLogin():

    @allure.story("Landed successfully")
    def test_login_success(self):
        print("This is the login test case: Login succeeded")

    @allure.story("Login failed")
    def test_login_failure_a(self):
        print("This is the login test case: Login failed")

    @allure.story("Username is missing")
    def test_login_failure_b(self):
        print("Username is missing")

    @allure.story("Missing password")
    def test_login_failure_c(self):
        with allure.step("Click on username"):
            print("Enter your user name")
        with allure.step("click password"):
            print("enter password")
        print("Click to Login")
        with allure.step("Login failed after clicking login"):
            assert '1' == 1
            print("Login failed")

    @allure.story("Login failed")
    def test_login_failure_d(self):
        print("This is login: test case, login fails")

Excuting an order:

pytest test_allure_demo.py -vs -q --alluredir=./result
allure generate ./result/ -o ./report/ --clean
allure open -h 127.0.0.1 -p 8883 ./report/

Or only execute some of these use cases:

Execute use case: execute only feature Use case for login module
pytest test_allure_demo.py --allure-features="login module" -vs

Execute use case: execute only stroy for login failure use cases
pytest test_allure_demo.py --allure-stories="Login failed"

Execution use case: Execute the successful login use case under the login module
pytest test_allure_demo.py --allure-features "login module" --allure-stories "login successful" -vs

(3)@allure.link(), @allure.attach() ,@allure.issue()

Instructions

parameter value

Parameter Description

@allure.link()

define link

Used to define a connection that needs to be displayed in the test report

@allure.issue()

defect address

Corresponding to the defect address in the defect management system

@allure.attach()

Add attachments

Add test report attachments such as data, text, pictures, videos, web pages

import allure

TEST_CASE_LINK = 'https://github.com/qameta/allure-integrations/issues/8#issuecomment-268313637'


@allure.link('https://ceshiren.com')
def test_with_link():
    pass


@allure.link('https://www.baidu.com', name='Baidu address')
def test_with_named_link():
    print("This is link")


@allure.issue('140', 'bug address')
def test_with_issue_link():
    print("This is issue")


@allure.testcase(TEST_CASE_LINK, 'Test case management platform address')
def test_with_testcase_link():
    print("This is testcase")

(4)@allure.severity()

Instructions

parameter value

Parameter Description

@allure.severity()

Use case class

blocker,critical,normal,minor,trivial

Usage scenarios: Usually tests include PO, smoke test, and verification on-line test. Executed separately according to the level of importance, for example, the main process and important modules should be run once when going online.

Test cases with run level: normal, critical

pytest -s -v file name--allure-severities normal,critical --alluredir=./result

Five priorities:

level

describe

BLOCKER

Blocking defect (function not implemented, unable to go next step)

CRITICAL

Serious defect (missing function point)

NORMAL

General flaws (edge ​​cases, malformed)

MINOR

Minor defects (interface errors do not match ui requirements)

TRIVIAL

Minor defects (must have no hints, or hints are not standardized)

import allure

@allure.severity(allure.severity_level.BLOCKER)
def test_blocker():
    print("This is BLOCKER")

@allure.severity(allure.severity_level.TRIVIAL)
def test_tricial():
    print("TRIVIAL")

@allure.severity(allure.severity_level.NORMAL)
def test_normal():
    print("NORMAL")

@allure.severity(allure.severity_level.NORMAL)
class TestClassWithNormalSeverity(object):

    @allure.severity(allure.severity_level.MINOR)
    def test_minor(self):
        print("This is MINOR")
    
    @allure.severity(allure.severity_level.CRITICAL)
    def test_critical(self):
        print("This is CRITICAL")

Friends who like software testing, if my blog is helpful to you, and if you like the content of my blog, please "Like", "Comment" and "Favorite" with one click!