1. Introduction to allure
allure is a lightweight, flexible, multilingual test reporting tool. Not only does it present what has been tested very succinctly in the form of a neat Web report, but it allows everyone involved in the development process to extract maximum useful information from the day-to-day execution of tests.
From a development/QA perspective, allure reports shorten the life cycle of common defects: test failures can be divided into defects and broken tests, and you can also configure logs, steps, fixture s, attachments, time, history, and communication with TMS and Integration of defect tracking systems, so responsible developers and testers will have all the information.
Allure is a lightweight and very flexible open source test reporting framework. It supports most testing frameworks, such as TestNG, Pytest, JUint, etc. It's simple to use and easy to integrate into jenkins.
2. The results of the allure test report
2.1 Overview page
The entry point for each report is an "overview" page with dashboards and widgets
The overview page contains several default widgets that represent basic characteristics of the project and test environment.
- Statistics - Overall report statistics.
- Launches - If this report represents multiple test launches, statistics for each launch will be displayed here.
- Behavior - Result information aggregated by story and characteristics.
- Executor - Information about the test executor used to run the test.
- Historical Trend - If the test has accumulated some historical data, its trend will be calculated and displayed on the chart.
- Environments - Information about test environments (see How to define environments).
2.2 Categories
2.3 Kit
On the "Suites" tab, you can find a standard structured representation of executed tests grouped by suite and class.
2.4 Charts
View different statistics collected from test data: status breakdown or severity and duration graphs.
2.5 Timeline
The Timeline tab visualizes a review of the test execution, the allure adapter collects the precise timing of the tests, and on this tab they are arranged according to their sequential or parallel time structure.
2.6 Behavior
For behavior-driven approaches, this tab groups test results based on the Epic, Feature, and Story tags.
2.7 packages
The Packages tab represents a tree-like layout of test results, grouped by different packages.
After introducing the components of the allure report, let's learn how to install and use allure, and generate beautiful test reports.
3. The environment required for allure installation
- (1) java jdk environment
- allure execution depends on the java environment. I will not demonstrate how to install j to open it here.
- (2)allure-commanline
- Display test report
- (3) allure-pytest third-party library
4, allure-commandline installation
(1) Download address: https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/2.13.5/ (2.13.5 is used here as an example)
(2) Download allure.zip
(3) Unzip, enter the bin directory and add the bin directory to the Path environment variable
Configure environment variables as shown below:
5, allure-pytest installation
(1) Install using the command line (recommended)
pip install allure-pytest
(2) Installation using the interface
Install in settings, as shown below:
6. The usage of allure
Instructions
parameter value
Parameter Description
@allure.epic()
epic description
Define the project, when there are multiple projects to use, down is the Feature
@allure.feature()
module name
Use cases are distinguished by modules, and when there are multiple modules, give each a name
@allure.story()
use case name
Description of the use case
@allure.title()
use case title
use case title
@allure.testcase()
Use case related links
The address of the function storage system corresponding to the automation use case
@allure.issue()
defect address
Corresponding to the defect address in the defect management system
@allure.description()
use case description
Detailed description of the test case
@allure.step()
Steps
Operation steps of the test case
@allure.severity()
Use case class
blocker,critical,normal,minor,trivial
@allure.link()
define link
Used to define a connection that needs to be displayed in the test report
@allure.attach()
Add attachments
Add test report attachments such as data, text, pictures, videos, web pages
7. The command to generate the test report
(1) Collect results during test execution:
#--alluredir This option is used to specify the path to store the test results pytest [test file] -s -q --alluredir=./result/
(2) View the test report
Method 1: After the test is completed, check the actual report, watch the report online, and directly open the default browser to display the current report
#Pay attention to the writing of serve here allure serve ./result/
Method 2: Generate a report from the result, which is a service that starts tomcat. Two steps required: generate report, open report
-
Generate report:
#Note: Override path plus --clean
allure generate ./result/ -o ./report/ --clean -
Open the report:
#./report is the path where the report is located
allure open -h 127.0.0.1 -p 8883 ./report/
example:
#Method 1: test_allure1.py is the module name, ./result is the report address pytest test_allure_demo.py -vs -q --alluredir=./result allure serve ./result/ # Method 2 pytest test_allure_demo.py -vs -q --alluredir=./result allure generate ./result/ -o ./report/ --clean allure open -h 127.0.0.1 -p 8883 ./report/
8. allure common features in practice
(1) @allure.feature() ,@allure.title()
Instructions
parameter value
Parameter Description
@allure.feature()
module name
Use cases are distinguished by modules, and when there are multiple modules, give each a name
@allure.title()
use case title
use case title
import allure @allure.feature("search module") class TestSearch(): def test_case1(self): print("case1") def test_case2(self): print("case2")
Execute the generate report command:
pytest test_allure_demo.py -vs -q --alluredir=./result
The command to execute the test report service:
allure serve ./result/
(2)@allure.story(),@allure.step()
Instructions
parameter value
Parameter Description
@allure.story()
use case name
Description of the use case
@allure.step()
Steps
Operation steps of the test case
import allure @allure.feature("login module") class TestLogin(): @allure.story("Landed successfully") def test_login_success(self): print("This is the login test case: Login succeeded") @allure.story("Login failed") def test_login_failure_a(self): print("This is the login test case: Login failed") @allure.story("Username is missing") def test_login_failure_b(self): print("Username is missing") @allure.story("Missing password") def test_login_failure_c(self): with allure.step("Click on username"): print("Enter your user name") with allure.step("click password"): print("enter password") print("Click to Login") with allure.step("Login failed after clicking login"): assert '1' == 1 print("Login failed") @allure.story("Login failed") def test_login_failure_d(self): print("This is login: test case, login fails")
Excuting an order:
pytest test_allure_demo.py -vs -q --alluredir=./result allure generate ./result/ -o ./report/ --clean allure open -h 127.0.0.1 -p 8883 ./report/
Or only execute some of these use cases:
Execute use case: execute only feature Use case for login module pytest test_allure_demo.py --allure-features="login module" -vs Execute use case: execute only stroy for login failure use cases pytest test_allure_demo.py --allure-stories="Login failed" Execution use case: Execute the successful login use case under the login module pytest test_allure_demo.py --allure-features "login module" --allure-stories "login successful" -vs
(3)@allure.link(), @allure.attach() ,@allure.issue()
Instructions
parameter value
Parameter Description
@allure.link()
define link
Used to define a connection that needs to be displayed in the test report
@allure.issue()
defect address
Corresponding to the defect address in the defect management system
@allure.attach()
Add attachments
Add test report attachments such as data, text, pictures, videos, web pages
import allure TEST_CASE_LINK = 'https://github.com/qameta/allure-integrations/issues/8#issuecomment-268313637' @allure.link('https://ceshiren.com') def test_with_link(): pass @allure.link('https://www.baidu.com', name='Baidu address') def test_with_named_link(): print("This is link") @allure.issue('140', 'bug address') def test_with_issue_link(): print("This is issue") @allure.testcase(TEST_CASE_LINK, 'Test case management platform address') def test_with_testcase_link(): print("This is testcase")
(4)@allure.severity()
Instructions
parameter value
Parameter Description
@allure.severity()
Use case class
blocker,critical,normal,minor,trivial
Usage scenarios: Usually tests include PO, smoke test, and verification on-line test. Executed separately according to the level of importance, for example, the main process and important modules should be run once when going online.
Test cases with run level: normal, critical
pytest -s -v file name--allure-severities normal,critical --alluredir=./result
Five priorities:
level
describe
BLOCKER
Blocking defect (function not implemented, unable to go next step)
CRITICAL
Serious defect (missing function point)
NORMAL
General flaws (edge cases, malformed)
MINOR
Minor defects (interface errors do not match ui requirements)
TRIVIAL
Minor defects (must have no hints, or hints are not standardized)
import allure @allure.severity(allure.severity_level.BLOCKER) def test_blocker(): print("This is BLOCKER") @allure.severity(allure.severity_level.TRIVIAL) def test_tricial(): print("TRIVIAL") @allure.severity(allure.severity_level.NORMAL) def test_normal(): print("NORMAL") @allure.severity(allure.severity_level.NORMAL) class TestClassWithNormalSeverity(object): @allure.severity(allure.severity_level.MINOR) def test_minor(self): print("This is MINOR") @allure.severity(allure.severity_level.CRITICAL) def test_critical(self): print("This is CRITICAL")
Friends who like software testing, if my blog is helpful to you, and if you like the content of my blog, please "Like", "Comment" and "Favorite" with one click!