I have seen many automation projects using TestNG as their Test Engine layer but they fail to generalize it.

So they end up creating multiple .xml files in their project and hardcoding of classes or methods to run.

They manually comment methods to be ignored in the .xml file or add enabled=false to the @Test annotation and then initiate the execution.

In this post, we’ll focus on executing a set of tests using the .xml file, and then we will try to generalize it by creating TestNG.xml at runtime. So we don’t need to worry about manually changing our .xml files before starting our execution.

Project Package Structure

This is the package structure we’ll be referring to create and run test cases from an .xml file.

Cases are separated into three types: Regression, Sanity, and Smoke and each class have three methods method1(), method2(), and method3().

Class Reg1.java contains three methods and methods name starting with reg1 . Similarly, other classes follow the same structure and the method name starts with smoke1, sanity1, reg2, and so on.

You can refer code for above structure here.

Now, to execute all the cases from this structure, either we can create three separate .xml files for Regression, Sanity, and Smoke or we can add all the cases in a single .xml file.

For now, I am just adding all the cases in a single file. So my TestNG.xml file will look like this:

After executing all @Test methods from .xml file, code output will look like:

Now, suppose I just want to run only regression cases from this .xml file then my only option is to comment Smoke and Sanity <test> tags.

Similarly, if I want to run only sanity cases then I need to comment rest of the test, and this cycle keeps going on and so far we have not even talked about handling method-level execution.

To avoid all the redundant work, We can create our TestNG at runtime and we can control all the cases we want to execute from an external resource (Excel/DB/etc). So no more .xml or .xml's in our project.

Creating Runtime TestNG

Creating testNG at runtime is the same as creating an .xml file. We just need to follow the proper hierarchy.

TestNG.xml can contain multiple <test>, <class>, and <include> tags, so we need to write the code in the same way i.e. we need to create a List object of above tags and then we need to bind all the Lists together.

Here is the code to create TestNG at runtime:

One catch with generating TestNG at runtime is, we don’t know exactly which class contains methods with @Test annotations and we cannot search for methods in all the classes present at the classpath as it’ll take a lot of time.

To overcome this issue, we take the help of Reflections. Reflection scans classpath, indexes the metadata, allows us to query it on runtime, and may save and collect that information for many modules within your project.

We created a method to get all the classes present inside a particular path i.e. com.learn.testng.cases.

When we initiate the run, our XML file will look like below as we’re not applying any execution restrictions on methods. So all the methods present inside the class will be added to the XML.

You can also refer entire code on my GitHub repository.

Continuing this post, I’ll be writing another post on how to handle execution from an external file and filtering logic while adding it to the TestNG.

To learn more about TestNG execution order and priority click here.