In this fast growing IT world, we are adopting new technology process and standards to achieve excellence and to save time .One of such practice these days we are using is DevOps. DevOps is the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes.DevOps is a fundamental change in the traditional structure of IT.

As of part of test automation if we are using UFT as a automation tool, one such requirement for running of UFT tests through the Azure pipeline and displaying results in pipeline.

In the below blog i will explain how we can display test suite results in azure pipeline ,even this solution would also work for any automation tool to push the results in azure pipeline after executing the automation tests.

Prerequisite

1.Azure DevOps subscription or you can try trial subscription from here.

2.UFT Tool.

Pipeline Test Result Format

Azure pipeline supported results formats include CTest, JUnit (including PHPUnit), NUnit 2, NUnit 3, Visual Studio Test (TRX), and xUnit 2. Here i am using JUnit to push the UFT results to azure pipeline.

Below table lists the fields reported in the Tests tab in a build or release summary, and the corresponding mapping with the attributes in the supported test result formats.

 *Duration is used only when Date started and Date completed are not available.

you can find Junit schema from here to see the detailed options and parameters.

below is the Junit xml as per the above format.

<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" timestamp="2020-08-17T15:01:15.00" name="UFTTestSuitResults" time="109.141" tests="2" skipped="0" failures="1">
   <testcase name="UFT_TC01_Scenario"  time="69.0" owner="Mohd Nazim" />
    <testcase name="UFT_TC02_Scenario" time="40.00" owner="Mohd Nazim">
	 <error type="error_message" message="failure msg to be displayed"/>
	</testcase>
</testsuite>

Where,

name=UFTTestSuitResults is the name of test suit displayed in result.

time=109.141 is total time taken by test suite to execute

tests=2 total number of tests inside test suite

name=UFT_TC01_Scenario name of test /sctipt

owner is name of user responsible for execution

message is used for displaying the failure reason in case of failed test case.

We need of generate this Junit result xml at the end of test suite execution in UFT with data/detailed required by xml to push the result in Azure pipeline.

Publish Test Results task

The published test results task are used to displayed in the Tests tab in the pipeline summary and help you to measure pipeline quality, review traceability, troubleshoot failures, and drive failure ownership.

Add publish test result task just after the task which execute UFT tests in pipeline.

The following example shows the task configured to publish test results.

Where,

Test Result format should be JUnit

Test results files is the name Junit test result xml file.

Search Folder is the path of Junit xml file.

UFT Implementation

Use below UFT code snippet to generate Junit xml at run time.

testsuite_junit_result_xml = "<?xml version='1.0' encoding='UTF-8'?><testsuite xmlns:xsi="&Chr(34)&"http://www.w3.org/2001/XMLSchema-instance"&Chr(34)&" timestamp="&Chr(34)&testsuite_timestamp&Chr(34)&" name="&Chr(34)&testsuite_name&Chr(34)&" time="&Chr(34)&"TS_DURATION_PLACEHOLDER"&Chr(34)&" tests="&Chr(34)&"TS_TESTS_PLACEHOLDER"&Chr(34)&" failure="&Chr(34)&"TS_FAILED_TC_PLACEHOLDER"&Chr(34)&">"

testsuite_junit_result_xml is the string for Junit xml header. Using FSO ,create a new Junit xml and write the header string ,below code will generate the Junit xml with header before the start of test suite execution.

       'Generating junit xml file
	Const ForWriting = 2
	Set fso = CreateObject("Scripting.FileSystemObject")
	
	'Path of json file 
	junitXmlFileDir = "C:\UFTAzureFramework\TestReults\JunitResult"
	junitXmlFileName = "junit_test_suite"
        testsuite_junit_result_xml = junitXmlFileDir&"\"&junitXmlFileName&".xml"
	Environment.Value("JunitXmlFilePath") = testsuite_junit_result_xml
	'Creating & writing xml data to a xml file
	Set junitXmlFile = fso.CreateTextFile(junitXmlFileDir ,ForWriting, False)
	
	junitXmlFile.Write(testsuite_junit_result_xml)
	
	Set junitXmlFile = Nothing
	Set fso = Nothing

Below is the function used to add the individual test result to JUnit xml after execution of each test.

Function fn_AppendResultToJunitXml(TC_Name,TC_Duration,TC_Status,TC_Failure_Msg)

	Dim fso
	Dim junitXmlFile
	Dim testcase_junit_result_xml : testcase_junit_result_xml = ""
	Dim testcase_name : testcase_name = ""
	Dim testcase_owner : testcase_owner = ""
	Dim testcase_duration : testcase_duration = ""
	Dim testcase_failure_msg : testcase_failure_msg = ""
	
	testcase_name = TC_Name
	testcase_duration = TC_Duration
	testcase_failure_msg = TC_Failure_Msg
	testcase_owner = Environment.Value("TesterName")

	If UCase(TC_Status) ="PASS" Then
		testcase_junit_result_xml = "<testcase name="&Chr(34)&testcase_name&Chr(34)&"  time="&Chr(34)&testcase_duration&Chr(34)&" owner="&Chr(34)&testcase_owner&Chr(34)&" />"
	Else
		testcase_junit_result_xml = "<testcase name="&Chr(34)&testcase_name&Chr(34)&"  time="&Chr(34)&testcase_duration&Chr(34)&" owner="&Chr(34)&testcase_owner&Chr(34)&"><error type="&Chr(34)&"error_message"&Chr(34)&" message="&Chr(34)&testcase_failure_msg&Chr(34)&"/></testcase>"		
	End If
	
	'For appending data to junit xml file
	Const ForAppending = 8
	Const TriStateFalse = 0
	
	Set fso = CreateObject("Scripting.FileSystemObject")
			
	'for appending test case result to xml file
	Set junitXmlFile = fso.OpenTextFile (Environment.Value("JunitXmlFilePath"),ForAppending, TriStateFalse)
	junitXmlFile.Write(testcase_junit_result_xml)
   	 junitXmlFile.Close
    
   	 Set junitXmlFile = Nothing
	 Set fso = Nothing

End Function

use below function to end the Junit xml after execution of test suite .

Function fn_EndSuiteJunitXmlFile()

	Dim fso
	Dim junitXmlFile
	
	Const ForAppending = 8
	Const TriStateFalse = 0
	
	Set fso = CreateObject("Scripting.FileSystemObject")
	
	'for appending test case result to xml file
	Set junitXmlFile = fso.OpenTextFile (Environment.Value("JunitXmlFilePath"),ForAppending, TriStateFalse)
	junitXmlFile.Write("</testsuite>")
	junitXmlFile.Close
	
	Set junitXmlFile = Nothing
	Set fso = Nothing
	
End Function

Update the Junit xml file and replace TS_DURATION_PLACEHOLDER, TS_TESTS_PLACEHOLDER and TS_FAILED_TC_PLACEHOLDER with actual values.

Function fn_UpdateJunitXmlFile(strTestSuiteDuration,strTestSuitTestCases,strTestSuiteFailures)

    Const ForReading = 1, ForWriting = 2, ForAppending = 8
	Const TristateUseDefault = -2, TristateTrue = -1, TristateFalse = 0	
	Dim oFSO, oFolder, oFile, oTextStream, report_all_text
	
	Set oFSO = CreateObject("Scripting.FileSystemObject")
	Set oFile = oFSO.GetFile(Environment.Value("JunitXmlFilePath"))
	Set oTextStream = oFile.OpenAsTextStream(ForReading, TristateUseDefault)
	report_all_text = oTextStream.ReadAll
	oTextStream.Close  
	Set oTextStream = oFile.OpenAsTextStream(ForWriting, TristateUseDefault)
	report_all_text = Replace(report_all_text,"TS_DURATION_PLACEHOLDER",Cstr(strTestSuiteDuration))
	report_all_text = Replace(report_all_text,"TS_TESTS_PLACEHOLDER",Cstr(strTestSuitTestCases))
	report_all_text = Replace(report_all_text,"TS_FAILED_TC_PLACEHOLDER",Cstr(strTestSuiteFailures))
	oTextStream.Write ""
	oTextStream.Write report_all_text
	oTextStream.Close
	
	Set oTextStream = Nothing
	Set oFile = Nothing
	Set oFSO = Nothing
	
End Function

Use above code as per your UFT framework and execute test suite by azure pipeline ,you will see the UFT tests result in pipeline under Tests Tab.

Reference: