Executing specific testng group using build.gradle

Question

I have checked following questions but none of them helped -

Gradle + TestNG Only Running Specified Group

Gradle command syntax for executing TESTNG tests as a group

The project I am using is available at - https://github.com/tarun3kumar/gradle-demo It is standard maven project and I am not using testng.xml file. Test method - com.org.corpsite.LandingPageTest is grouped as - smoke

I am running test as - gradle clean test and test is executed. Test fails due to genuine reason and let's ignore it.

Then I passed test group from command line as - gradle clean test -P testGroups='doesnotexist' Notice that 'doesnotexist' is not a valid group but it still executes test.

Following this I added includeGroups in build.gradle as -

test {
    useTestNG() {
        includeGroups 'smoke'
    }
}

and now gradle clean test -P testGroups='doesnotexist' fails with NPE on one of the java class - java.lang.NullPointerException at com.org.pageobjects.BasePage.findElements(BasePage.java:24)

Questions -

1. What is right flag to specify test group from command line? Seems -P is wrong else gradle clean test -P testGroups='doesnotexist' would not execute test.
2. What is wrong with specifying includeGroups 'smoke'?

I am using Gradle 5.1 on macbook pro

Answer 1

Here are the set of things that need to be done to get this to work.

1. You need to add the attribute alwaysRun=true to your @BeforeMethod and @AfterMethod annotations from your base class com.org.core.SelTestCase. This is to ensure that TestNG executes these configuration methods all the time irrespective of what group is chosen.
2. Alter the test task in your build.gradle to look like below:

test {
    def groups = System.getProperty('groups', 'smoke')
    useTestNG() {
        includeGroups groups
    }
}

This ensures that we try to extract the JVM argument groups value. If its not specified we default to smoke.

We now execute the tests by specifying the groups needed using the below command:

./gradlew clean test --info -Dgroups=smoke

Now if we execute the below command, you would notice that no tests are executed.

./gradlew clean test --info -Dgroups=smoke1

Here's a patch that you can apply to your project

From 25133a5d2a0f96d4a305f34e1f5a17e70be2bb54 Mon Sep 17 00:00:00 2001
From: Krishnan Mahadevan <krishnan.mahadevan@stackoverflow.com>
Date: Mon, 14 Jan 2019 22:38:27 +0530
Subject: [PATCH] Fixing the bug

---
 build.gradle                                | 2 ++
 src/main/java/com/org/core/SelTestCase.java | 5 +++--
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/build.gradle b/build.gradle
index 10ba91d..2d08991 100644
--- a/build.gradle
+++ b/build.gradle
@@ -38,7 +38,9 @@ task smokeTests(type: Test) {
 }*/

 test {
+    def groups = System.getProperty('groups', 'smoke')
     useTestNG() {
+        includeGroups groups
     }
 }

diff --git a/src/main/java/com/org/core/SelTestCase.java b/src/main/java/com/org/core/SelTestCase.java
index 80cad09..651529a 100644
--- a/src/main/java/com/org/core/SelTestCase.java
+++ b/src/main/java/com/org/core/SelTestCase.java
@@ -22,7 +22,7 @@ public class SelTestCase {

     private WebDriver webDriver;

-    @BeforeMethod
+    @BeforeMethod(alwaysRun = true)
     @Parameters({"browser", "url"})
     public void setUp(@Optional("firefox") String browser, @Optional("https://www.google.com/") String URL) {
         switch (browser) {
@@ -40,8 +40,9 @@ public class SelTestCase {
         webDriver.get(URL);
     }

-    @AfterMethod
+    @AfterMethod(alwaysRun = true)
     public void tearDown() {
         webDriver.quit();
     }
+
 }
-- 
2.20.1

You can save the above contents to a file say mypatch.patch and then apply the patch using the instructions detailed in this StackOverFlow post.

Answer 2

You should be able to run a specific test with the 'testInstrumentationRunnerArguments' flag:

-Pandroid.testInstrumentationRunnerArguments.class=com.abc.NameOfMyTestClass