Debugging Tools: Application Validator (How to use it and how it works)

What does Application Verifier do?

Application Verifier is an official application verification tool from Microsoft. It is mainly used to help users detect and debug memory corruption, dangerous security vulnerabilities, Run-time detection, etc. It is an auxiliary development tool without modifying the source code; When reporting unreleased resources, etc., the program will report normally after exiting.

  • Obtain: You can download the latest version through the windows software development Kit Manager
  • Recommended: WinDbg and application validator are used together in many scenarios, and the articles in the reference can also be read.
  • Principle: hook the API for allocating and releasing resources, and do some checks and records
  • Prerequisite knowledge: It is best to use WinDbg, gflag.exe, to understand the basic knowledge of heap... It doesn't matter if you don't understand, Baidu will know it at a glance

After using WinDbg for so long, I didn't know AV, but I found it while reading "Software Debugging" 2nd Edition, Chapter 19. The following is a compilation of the notes at that time.

0x01 Introdution

If you are new to this tool, it is recommended to read the How to use Application Verifier section of appverif.chm, which can be used as an introduction document for application verifier, including application timing, installation and configuration, choosing 64 or 32-bit version, basic testing, result analysis, etc.

When there is a bug in a Release version software, the verification mechanism provided by the operating system can be used; there are two common tools in the verification mechanism:

  • Driver Verifier (verifier.exe): Userland Tool
  • Application Verifier (appverif.exe): Kernel Mode Tool

Application Verifier (appverif.exe) is a runtime analysis tool for windows native code. It relies on the support of the operating system to catch common system API misuses. The tool can automatically capture many hard-to-reproduce code errors; it is a setting application Tools for validating mechanism parameters

The tool mainly consists of 2 parts: the function that starts with avf implemented in dlls such as verifier.dll and a setting tool (appverif.exe)

The application verifier (appverif.exe) is essentially a tool, which is explained below through Q&A and an example

0x02 FAQ

Tip: The Frequently Asked Questions about the Application Verifier chapter on appverif.chm has many answers to newbie questions

The following is a question and answer form to explain some doubts about the use of the tool (mainly the first time you use it), in fact, you will naturally remember it after a few operations.

Q1: When to use?

Under what circumstances can the application authentication mechanism be used? There are usually 2 situations:

  • 1. Start all related processes in development, the disadvantage is that it affects the efficiency of program execution
  • 2. When modifying bug s in the product, it can be ensured that the modification of errors will not introduce new problems

Q2: How to start and stop?

The method to start and stop is given below

=========================Start==================================
#Start Testing an Application
To test an application you must first add the Application to the verifier tool. 
You can then select the tests you want enabled, save your test setup, and run your application.

1.On the File menu, select Add Application.Browse to the application and click Open
2.select the tests to run
3.When the tests you want enabled have all been selected, click Save

Note: After setting, regardless of appvrif.exe Whether it is running (mainly modifying an item in the registry, and continuing to run the program will read the registry), the application will perform the corresponding verification every time it runs 

=========================Stop==================================
#To remove an application from Application Verifier
1.Select the name of the application
2.On the File menu, select Delete Application .
3.Click Save . 

Note: Even if the corresponding program is removed, produced before log files are not deleted

Q3: How to check the test setting items?

Each test item has some properties, which can be viewed in detail in the following ways

1. Display the property setting area

#To view the Property window
On the View menu, select Property Window . A check mark while appear next to the Property Window selection.

The property setting area is closed by default and can be opened in the following ways; it will display the detailed parameter settings and description information of a specific test item

2. Right-click on the test item

  • Select Properties: Properties that allow control of test settings, note that not all test settings have

  • Select Verifier Stop Options: Set verification stop related properties

Among them: Not Continuable means that when the verification pause is triggered, it will break into the debugger and will not recover from the verification pause (the effect is to stop in the debugger)

Tip: Other specific properties do not need to be introduced in detail, you can understand at a glance; for specific information about each pause code, please refer to the Stop Codes and Definitions chapter of appverif.chm

The following are the details of the verification pause code 7, including the possible causes and the format of the results.

Q4: How to check the test results?

The normal application verifier use process: use the verifier (appverif.exe) to start the corresponding settings for a program, start the program, and the verifier (appverif.exe) will run in the background; there are two ways to observe the test results:

  • 1. View the corresponding log file

Logs under the View menu can be viewed

  • 2. Debugger viewing: that is, to debug with WinDbg and other tools, and view it in WinDbg; this will not be discussed first, and later you will see that the error information in the debugger has changed

Tip: Stop the mouse directly on the corresponding test settings to view the short description, and you will see whether the test results need to be viewed in the log or in the debugger

Q5: Full page heap and normal page heap?

Note: The full page heap is enabled by default, so you need to explain the page heap. What is the page heap?

Function: that is, the problem can be found immediately when the heap is destroyed, instead of waiting until the next time the heap function is called

Principle: The simple understanding is to add a fence page (fense page) specially used for detection after the heap block. Once the user data area overflows and triggers the fence page (fense page), an exception will be triggered immediately, instead of waiting until the next time the heap function is called. The problem will be found immediately; the fence page can also be added in front of the heap block

Information: If you want to learn more about page heap knowledge, you can see here

Documentation: The Debugging Heap Errors chapter of appverif.chm introduces page heaps. There are actually two types of page heaps: full page heaps and normal page heaps. The following is the difference between the two in the screenshots in "Software Debugging"

Q6: Modify the timing of being captured by the process?

The following is the general flow of the program after the application has verified the configuration:

  • 1. The process loads the module

The process loader will first load the executable program (exe) and the two modules of ntdll.dll. The function at the beginning of ldr in ntdll.dll is responsible for the main loading work.

  • 2. Loader initialization

Will look for the current program's executable options (capture location) from the registry key below

\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\programname.exe

  • 3. Dynamically modify the IAT based on executable decisions
  • 4. Find the EP to execute the program

0x03 Release the same piece of memory 2 times

The following uses an example of heap memory release twice to explain the use of the application validator, and build the code into an executable file named heap_test

//Free the same memory twice
//Direct running phenomenon: running alone, the program will crash; if the JIT debugger is specified, it will trigger
//Behavior in debugger: If configured by application validator, exact type of error is found (double free)
#include "StdAfx.h"
#include <stdio.h>
void main(int argc, char* args[]) {
	char *psz = NULL;
	psz = new char[10];
	delete[] psz;
	delete[] psz;			//secondary release

	return;
}

The following picture shows the debugging information without the verification function related to the application verifier. It can be seen that the instruction is abnormally interrupted to the debugger (the error code (0x80000003) and the prompt information have no valuable information)

Tip: This screenshot is a classic screenshot of using WinDbg to check bug s, remember it first; compare it with the WinDbg screenshot after opening the application verification to see which one is better?

What should I do if I have no ideas for further positioning? Usually it is a series of WinDbg such as kb, which will not be introduced here.

If you are not very familiar with WinDbg, is there any fool-proof operation to get more information when the program crashes? The application authenticator described below may be something you can try

Tip: The following two methods are used to introduce application verification

  • 1. Application Verifier (appverif.exe) method: mainly focuses on the preliminary understanding of application verification
  • 2.gflags.exe method: mainly to expand the configuration method of application verification and understand the verification hook (principle)

Summarize the difference between the two methods first: After the application verifier (appverif.exe) sets up a program, the registry keys will appear with values ​​such as GlobalFlag and VerifierFlags; gflags.exe will only affect the GlobalFlag registry value

0x04 Application Authenticator Application

First, a brief introduction to the basic use of the tool

step 1. Configuration

take effect

The default configuration screenshot of the executable file (heap_test.exe) generated by the source code is as follows

Tip: If you use it for the first time, you can generally configure Basics, and then you can configure it according to your own requirements. At this time, the page heap will also be opened by default.

Registry Impact

The configuration actually affects the global flag. The screenshots viewed in the registry are as follows. You can see that the test items set in the application validator and the registry are in one-to-one correspondence

Some of the information can also be found in global variables using WinDbg

Cancel

To disable the settings of a process, there are two simple ways:

  • Use Ctrl+D in appverif.exe

    Note: There will be a little residue in the registry at this time, it is best to delete it together to prevent problems with the subsequent use of WinDbg to open the process

  • Delete the corresponding item directly in the registry (because the tool actually affects the item in the registry)

step 2.WinDbg debugging

After configuring the verification items, re-run the program under WinDbg and find that both the page heap and the application are enabled

0:000> vercommand
command line: '"C:\Program Files (x86)\Windows Kits\10\Debuggers\x86\windbg.exe" '

#View global flags
0:000> !gflag
Current NtGlobalFlag contents: 0x02000100
    vrf - Enable application verifier					#Application validator in effect, 0x00000100
    hpa - Place heap allocations at ends of pages		#Page heap takes effect, 0x02000000

Continue to let the program run, this time to see if the prompt message will change when something goes wrong?

The error message is more specific, and it is no longer an error code (0x80000003) that is not useful. If you focus on this, you must learn how to use the application validator.

#Running, it will generate an exception to the debugger, and the information to detect the problem is more comprehensive
0:000> g
=======================================
VERIFIER STOP 00000007: pid 0x16630: Heap block already freed. 					#The heap block has been freed
	07F71000 : Heap handle for the heap owning the block.
	07F72BFC : Heap block being freed again.				
	0000000A : Size of the heap block.											#heap size
	00000000 : Not used
=======================================
This verifier stop is not continuable. Process will be terminated 
when you use the `go' debugger command.
=======================================
(16630.16634): Break instruction exception - code 80000003 (first chance)		#command exception
eax=7c4e03a0 ebx=00000000 ecx=000001a1 edx=006ff2a1 esi=07f71000 edi=7c4ddda0
eip=7c4d3bf8 esp=006ff58c ebp=006ff794 iopl=0         nv up ei pl nz na po nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000202
vrfcore!VerifierStopMessageEx+0x5c8:
7c4d3bf8 cc              int     3
  • Explanation: VERIFIER STOP 00000007, the pause code 7 represents the repeatedly released heap block; if the same heap block is released twice, this pause will be triggered (this is too direct...)

  • Root cause: Which heap block (07F72BFC) has been released by the application verifier, and why (if the same heap block is released twice), the instruction exception is displayed

Recommendation: If this secondary release is triggered in a special scenario and the program code is large, it is difficult to find when VS debugs the executable program. Therefore, it is recommended to use the application verifier to cooperate with the debugger to check the bug of the problem.

Tip: All prompts that the second release causes the program to crash, then the remaining operation is the code in the review code that can cause the second release; if there is source code, you can use commands such as k to view the stack frame

0:000> k
# ChildEBP RetAddr  
00 006ff794 7c4d83c0 vrfcore!VerifierStopMessageEx+0x5c8	        #stop message
01 006ff7b8 7935df34 vrfcore!VfCoreRedirectedStopMessage+0x80
02 006ff80c 7935bb4f verifier!VerifierStopMessage+0x84
03 006ff878 793589dd verifier!AVrfpDphReportCorruptedBlock+0x1cf	#heap block exception
04 006ff8dc 79358b35 verifier!AVrfpDphFindBusyMemoryNoCheck+0x7d
05 006ff900 7935cde1 verifier!AVrfpDphFindBusyMemory+0x15
06 006ff918 79362572 verifier!AvrfpDphCheckPageHeapAllocation+0x41
07 006ff928 7c526123 verifier!VerifierCheckPageHeapAllocation+0x12				
08 006ff970 00b61314 vfbasics!AVrfpHeapFree+0x53					#Call the release function of the vfbasics.dll version
09 006ff984 00b61016 heap_test!free+0x1c [f:\dd\vctools\crt_bld\self_x86\crt\src\free.c @ 51] 
0a 006ff998 00b61298 heap_test!main+0x16 [c:\...\heap_test.cpp @ 11] 

Extension: Format of validation result output in debugger

#The basics of the verification result output in the debugger: the parameter list in it depends on the type of verification being performed
=======================================
VERIFIER STOP <stop-code>: <porcess-ID>: Brief description of the failure 
	parameter -1 : describe
	parameter -2 : describe			
	parameter -3 : describe	
	parameter -4 : describe
=======================================

step 3. Extended command (!avrf)

The following introduces a very useful extension command in WinDbg

The NT global flag can be viewed with the extended command !gflag, and the command !avrf in WinDbg can be used to view the check flags of all debugging processes

#Partial summary of !avrf in the official website
The !avrf extension controls the settings of Application Verifier and displays a variety of output produced by Application Verifier.

#grammar:
!avrf
The !avrf command without any parameters shows the Application Verifier settings and information about the current and previous Application Verifier breaks if any.
!avrf -hp { Length | -a Address }
Displays the heap operation log. Address specifies the heap address. Records of the heap operations containing this heap address will be displayed.
...

#Comments
#Show authentication configuration
When the !avrf extension is used with no parameters, it displays the current Application Verifier options.
#Get verification stop codes and related reasons
If an Application Verifier Stop has occurred, the !avrf extension with no parameters will reveal the nature of the stop and what caused it.
#missing symbol information
If symbols for ntdll.dll and verifier.dll are missing, the !avrf extension will generate an error message. 

Tip: If the !avrf command fails to display, it is probably because of the path problem of vfbasics.dll (the old version of Microsoft's symbol server has the public symbols of this dll, but there is no type required for function expansion, so the parsing fails, the new version does not encounter To this problem); because vfbasics.dll is put into the symbol, it will be installed in the system32 directory together, so the solution is to set the symbol path, so that the path of the local system32 appears in front of Microsoft's symbol server

The following is a screenshot of the command used, which saves the process of finding the error code and directly gives the brief information

#View NT Global Flags
0:000> !gflag
Current NtGlobalFlag contents: 0x02000100
    vrf - Enable application verifier
    hpa - Place heap allocations at ends of pages

#View the checkmarks of all debugged processes (for comprehensive display, the following is the result set using the default check items of appverif.exe)
0:000> !avrf
Verifier package version >= 3.00 
Application verifier settings (81643027):
   - full page heap						#One-to-one correspondence with the default check items of appverif.exe
   - Handles
   - Locks
   - Memory
   ...
   - SRWLock
*******************************************************************************
*                        Exception Analysis                                   *
*******************************************************************************
APPLICATION_VERIFIER_HEAPS_DOUBLE_FREE (7)								#Reason: The heap was freed twice
Heap block already freed.
This situation happens if the block is freed twice. Freed blocks are marked in a
special way and are kept around for a while in a delayed free queue. ...

Arguments:
Arg1: 08051000, Heap handle for the heap owning the block. 				#get details
Arg2: 08052c64, Heap block being freed again. 
Arg3: 0000000a, Size of the heap block. 
Arg4: 00000000, Not used 

NTGLOBALFLAG:  2000100
ExceptionAddress: 799e3bf8 (vrfcore!VerifierStopMessageEx+0x000005c8)
   ExceptionCode: 80000003 (Break instruction exception)
FAILURE_BUCKET_ID:  BREAKPOINT_AVRF_80000003_vrfcore.dll!VerifierStopMessageEx

0x05 Checker supported by the system

The following mainly introduces the verifier for setting the program through gflags.exe

Tip: gflags is also a very powerful tool, it is best to learn it when you have time

step 1. Configuration

The gflags.exe tool starts the default validator supported by the operating system for the sample program

The above actually sets the NT global flag of the process image to 0x100 (vrf, application verifier), which tells the operating system that we want subsequent instances of the process to start the public application verifier hook (hook, a simple understanding is that it will hook off part of the function)

The above settings will affect 2 places:

  • 1. Global Flag GlobalFlag Image File Execution Options

  • 2. A dll called verifier.dll is loaded right after ntdll.dll (settings with appverif.exe also affect)

When running the target process under the user mode debugger, verifier.dll is loaded next to ntdll.dll and mapped into the address space of the process; this verifier.dll module contains the core verification engine of the operating system

#View NT Global Flags
0:000> !gflag
Current NtGlobalFlag contents: 0x02000100
    vrf - Enable application verifier
    hpa - Place heap allocations at ends of pages
#List modules    
0:000> lm
start    end        module name
00b70000 00b8d000   heap_test   (deferred)             
75590000 7578f000   KERNELBASE   (deferred)             
76900000 769e0000   KERNEL32   (deferred)             
77510000 776aa000   ntdll      (pdb symbols)   c:\symbols\...\wntdll.pdb
79350000 793b3000   verifier   (deferred)  				#this module application validator related modules          

#View the details of the verifier.dll module
0:000> lmv m verifier
Browse full module list
start    end        module name
79350000 793b3000   verifier   (deferred)             
    Image path: C:\WINDOWS\SysWOW64\verifier.dll
    Image name: verifier.dll
    FileDescription:  Standard application verifier provider dll

Page heap default setting?

  • hint

If you are careful, you will find that hpa (page heap) is also turned on, but the global flag GlobalFlag Image File Execution Options and the gflags /p query do not start the hpa process

  • association

In fact, the heap is selected by default in the heap in the default test settings of the application validator

step 2. System default startup items

When the application check bit in the NT global flag is set, the operating system will start the relevant check bit by default, and the processing check is the default startup item of the system

step 3. Principle of validator check

If you continue to run the program under the debugger, a verification interrupt will be hit, and the type of error displayed is block already freed.

0:000> g
===========================================================
VERIFIER STOP 00000007: pid 0x18424: block already freed 
	066E1000 : Heap handle
	06893968 : Heap block
	0000000A : Block size
	00000000 : 
===========================================================
(18424.183a4): Break instruction exception - code 80000003 (first chance)
verifier!VerifierBreakin+0x42:
7935dd72 cc              int     3
0:000> k
 # ChildEBP RetAddr      
00 007ef84c 7935de70     verifier!VerifierBreakin+0x42	#Interrupt is generated by verifier.dll
01 007efb74 7935e16d     verifier!VerifierCaptureContextAndReportStop+0xf0
02 007efbb8 7935bb4f     verifier!VerifierStopMessage+0x2bd
03 007efc24 7935c09a     verifier!AVrfpDphReportCorruptedBlock+0x1cf
04 007efc80 7935ce0b     verifier!AVrfpDphCheckNormalHeapBlock+0x11a
05 007efca0 79362572     verifier!AvrfpDphCheckPageHeapAllocation+0x6b
06 007efcb0 79377cd6     verifier!VerifierCheckPageHeapAllocation+0x12
07 007efd08 00b71314     verifier!AVrfpHeapFree+0x56	#free was changed to the function of the same name in verifier.dll
08 007efd1c 00b71016     heap_test!free+0x1c [f:\...\free.c @ 51] 
09 007efd30 00b71298     heap_test!main+0x16 [c:...\heap_test.cpp @ 8] 

Checking principle of the validator: mainly update the import table (IAT) items corresponding to some common win32 API function calls to intercept the calls to the API and verify whether it conforms to the agreed usage method, so there is a saying that the application checks the hook

The following is a screenshot of the runtime. You can see that free originally called the HeapFree function and was hook ed to the AVrfpHeapFree function that called verifier.dll

Hint: These hook methods are only for the user mode code context, the kernel driver is related to another tool (driver verification tool verifier.exe)

0x06 Summary

  • 1.hook common API

The application verification tool replaces the import table of common operating system APIs with specific calls to implement. Before executing the real function, it will perform additional verification checks to detect misuse of the API

  • 2. Local Features

The application verifier (appverif.exe) is only used for analysis when the local user mode is running, and the driver verification tool (verifier.exe) is used in the kernel mode.

  • 3. The target program is best run in a debugger

The application verification interrupt is proposed as a runtime assertion (SEH exception). If it is not running under the debugger and the JIT debugger is not set, the exception will not be handled and will simply hang up; therefore, try to make the target program run in the debugging mode. in the controller, which can catch any test and stop

0x07 reference

  • 1. Chapter 6.2 of "Windows Programming and Debugging Technology Insider"
  • 2. Relevant parts in "Windows Advanced Debugging"
  • 3. "Software Debugging" 2nd Edition, Chapter 19
  • 4. Help document for application verifier (appverif.chm)

Tags: Windows security microsoft

Posted by Idri on Wed, 25 May 2022 12:51:24 +0300