Multi Browser or Cross Browser Testing with TestNG

It is very important to test your web application across multiple browser. Each browser as different way of processing the code and hence user experience might differ from browser to browser as a tester it is our responsibility to make sure our application works fine on most commonly used browsers such as firefox, chrome and IE. 

TestNG allows us to automate the multi-browser testing using TestNG parameters. In this post we will see in detail how we can achieve this withe simple example.

Example.

1. Create a simple test script.

2. Configure it to pass Browser Type as a parameter.

package test;


import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;
 
import org.openqa.selenium.firefox.FirefoxDriver;
 
import org.openqa.selenium.ie.InternetExplorerDriver;
 
import org.testng.annotations.AfterClass;
 
import org.testng.annotations.BeforeClass;
 
import org.testng.annotations.Parameters;
 
import org.testng.annotations.Test;
 
public class MultiBrowserTest {
 
 public WebDriver driver;
 
  @Parameters("browser")
 
  @BeforeClass
 
  // Passing Browser parameter from TestNG xml
 
  public void beforeTest(String browser) {
 
  // If the browser is Chrome, then do this
 
  if(browser.equalsIgnoreCase("chrome")) {
 
   System.setProperty("webdriver.ie.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
   
   driver = new InternetExplorerDriver();;
 
  // If browser is IE, then do this   
 
  }else if (browser.equalsIgnoreCase("ie")) { 
 
   // Here I am setting up the path for my IEDriver
 
   System.setProperty("webdriver.ie.driver", "D:\\QA\\drivers\\IEDriverServer.exe");
 
   driver = new InternetExplorerDriver();
 
  } 
 
  // Doesn't the browser type, lauch the Website
 
  driver.get("http://www.qaautomated.com"); 
 
  }
 
  // Once Before method is completed, Test method will start
 
  @Test 
  public void search() throws Exception {
    
   Thread.sleep(5000);
      driver.findElement(By.xpath("*//input[@class='search-field']")).sendKeys("appium");
      
      driver.findElement(By.xpath("*//input[@class='search-submit']")).click();
 
      driver.quit();
 
 
 }  
 
  @AfterClass public void afterTest() {
 
  driver.quit();
 
 }
 
}

3. Mention browser type details in testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test name="ChromeTest">
 
 <parameter name="browser" value="chrome" />
 
 <classes>
 
 <class name="test.MultiBrowserTest" />
 
 </classes>
 
 </test>
 
 <test name="IETest">
 
 <parameter name="browser" value="ie" />
 
 <classes>
 
 <class name="test.MultiBrowserTest" />
 
 </classes>
 
 </test> <!-- Test -->
</suite> <!-- Suite -->

4. Run it as TestNG Suit.

Parallel Execution - Consider you want to execute your tests in parallel on multiple browser then with a small change to your testng.xml you achieve this

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="tests">
  <test name="ChromeTest">
 
 <parameter name="browser" value="chrome" />
 
 <classes>
 
 <class name="test.MultiBrowserTest" />
 
 </classes>
 
 </test>
 
 <test name="IETest">
 
 <parameter name="browser" value="ie" />
 
 <classes>
 
 <class name="test.MultiBrowserTest" />
 
 </classes>
 
 </test> <!-- Test -->
</suite> <!-- Suite -->

.

TestNG Data Provider with Excel

When we want to create follow data driven approach and create a framework then best way is to use Excel to store our data and we can pass the Data as parameters using Data providers which we have seen in our previous post.

Step by step process to Implement Excel with TestNg Data Provider.

Step 1: Create a test case of Login Application with TestNG Data Provider.

Step 2:  Create a Test Data sheet.

Step 3: Create functions to Open & Read data from Excel

Step 4: Create a TestNg test case for accepting data from Excel using Data Provider.

Step 5: Run the test against the Test Case name in the Test Data file.

Read Data From Excel Sheet using Apache POI library -

Below is our code to open Excel sheet and read data from it within our Selenium test script. For this purpose, we are using the Apache POI library, which allows us to read, create and edit Microsoft Office-documents using Java. The classes and methods we are going to use to read data from Excel sheet are located in the org.apache.poi.hssf.usermodel.

1. Download the Apache POI  from the link http://poi.apache.org/ and add it in your project build path.

2. Use Below Code to read Data from Excel file.

package test;

import java.io.FileInputStream;

import java.io.FileNotFoundException;

import java.io.FileOutputStream;

import java.io.IOException;

import org.apache.poi.xssf.usermodel.XSSFCell;

import org.apache.poi.xssf.usermodel.XSSFRow;

import org.apache.poi.xssf.usermodel.XSSFSheet;

import org.apache.poi.xssf.usermodel.XSSFWorkbook;

public class ExcelUtils {

 private static XSSFSheet ExcelWSheet;

 private static XSSFWorkbook ExcelWBook;

 private static XSSFCell Cell;

 private static XSSFRow Row;

 public static void setExcelFile(String Path,String SheetName) throws Exception {
   
     try {

    // Open the Excel file

    FileInputStream ExcelFile = new FileInputStream(Path);

    // Access the required test data sheet

    ExcelWBook = new XSSFWorkbook(ExcelFile);

    ExcelWSheet = ExcelWBook.getSheet(SheetName);

    } catch (Exception e){

     throw (e);

    }

  }
 public static Object[][] getTableArray(String FilePath, String SheetName) throws Exception {

  String[][] tabArray = null;

  try {

   FileInputStream ExcelFile = new FileInputStream(FilePath);

   // Access the required test data sheet

   ExcelWBook = new XSSFWorkbook(ExcelFile);

   ExcelWSheet = ExcelWBook.getSheet(SheetName);

   int startRow = 1;

   int startCol = 1;

   int ci, cj;

   int totalRows = ExcelWSheet.getLastRowNum();

   // you can write a function as well to get Column count

   int totalCols = 2;

   tabArray = new String[totalRows][totalCols];

   ci = 0;

   for (int i = startRow; i <= totalRows; i++, ci++) {

    cj = 0;

    for (int j = startCol; j <= totalCols; j++, cj++) {

     tabArray[ci][cj] = getCellData(i, j);

     System.out.println(tabArray[ci][cj]);

    }

   }

  }

  catch (FileNotFoundException e) {

   System.out.println("Could not read the Excel sheet");

   e.printStackTrace();

  }

  catch (IOException e) {

   System.out.println("Could not read the Excel sheet");

   e.printStackTrace();

  }

  return (tabArray);

 }

 public static String getCellData(int RowNum, int ColNum) throws Exception {
 
   try{
 
    Cell = ExcelWSheet.getRow(RowNum).getCell(ColNum);
 
    int dataType = Cell.getCellType();
 
    if  (dataType == 3) {
 
     return "";
 
    }else{
 
     String CellData = Cell.getStringCellValue();
 
     return CellData;
    }
 
    }catch (Exception e){
 
    System.out.println(e.getMessage());
 
    throw (e);
 
    }
 
   
 
  }
 public static String getTestCaseName(String sTestCase)throws Exception{
   
  String value = sTestCase;

  try{

   int posi = value.indexOf("@");

   value = value.substring(0, posi);

   posi = value.lastIndexOf("."); 

   value = value.substring(posi + 1);

   return value;

    }catch (Exception e){

   throw (e);

     }

  }

 public static int getRowContains(String sTestCaseName, int colNum) throws Exception{

  int i;

  try {

   int rowCount = ExcelUtils.getRowUsed();

   for ( i=0 ; i<rowCount; i++){

    if  (ExcelUtils.getCellData(i,colNum).equalsIgnoreCase(sTestCaseName)){

     break;

    }

   }

   return i;

    }catch (Exception e){

   throw(e);

   }

  }

 public static int getRowUsed() throws Exception {

   try{

    int RowCount = ExcelWSheet.getLastRowNum();

    return RowCount;

   }catch (Exception e){

    System.out.println(e.getMessage());

    throw (e);

   }

  }
 public static Object[][] getTableArray(String FilePath, String SheetName, int iTestCaseRow)    throws Exception
  
 {   

    String[][] tabArray = null;

    try{

     FileInputStream ExcelFile = new FileInputStream(FilePath);

     // Access the required test data sheet

     ExcelWBook = new XSSFWorkbook(ExcelFile);

     ExcelWSheet = ExcelWBook.getSheet(SheetName);

     int startCol = 1;

     int ci=0,cj=0;

     int totalRows = 1;

     int totalCols = 1;

     tabArray=new String[totalRows][totalCols];

      for (int j=startCol;j<=totalCols;j++, cj++)

      {

       tabArray[ci][cj]=getCellData(iTestCaseRow,j);

       System.out.println(tabArray[ci][cj]);

      }

  }

  catch (FileNotFoundException e)

  {

   System.out.println("Could not read the Excel sheet");

   e.printStackTrace();

  }

  catch (IOException e)

  {

   System.out.println("Could not read the Excel sheet");

   e.printStackTrace();

  }

  return(tabArray);

 }
    }

 3. Create excelfile with Data which we want to pass to our test case as parameter.

4. Update the Test case for using parameters from excel.

package test;


import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
 
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
 
import org.testng.annotations.Test;
 
import org.testng.annotations.BeforeMethod;
 
import org.testng.annotations.AfterMethod;
 
import org.testng.annotations.DataProvider;
 

 
public class TestCaseExcelData {
 
 private String sTestCaseName;
 
 private int iTestCaseRow;
 
 WebDriver driver;
 
  @BeforeMethod
 
  public void beforeMethod() throws Exception {
 
   System.setProperty("webdriver.chrome.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
  
   driver=new ChromeDriver();
      
      driver.get("http://www.qaautomated.com");
 
      Thread.sleep(5000);
  } 
 
  @Test(dataProvider = "search")
  
  public void test(String searchtext) throws Exception {
 
   
      driver.findElement(By.xpath("*//input[@class='search-field']")).sendKeys(searchtext);
      
      driver.findElement(By.xpath("*//input[@class='search-submit']")).click();
 
      
  }
 
  @AfterMethod
 
  public void afterMethod() {
 
    driver.close();
 
  }
 
  @DataProvider
 
  public Object[][] search() throws Exception{
 
     // Setting up the Test Data Excel file
 
   ExcelUtils.setExcelFile("C:\\Users\\manojjai\\Documents\\dataprovider.xlsx","Sheet1");
 
   sTestCaseName = this.toString();
 
    // From above method we get long test case name including package and class name etc.
 
    // The below method will refine your test case name, exactly the name use have used
 
    sTestCaseName = ExcelUtils.getTestCaseName(this.toString());
 
     // Fetching the Test Case row number from the Test Data Sheet
 
     // Getting the Test Case name to get the TestCase row from the Test Data Excel sheet
 
   iTestCaseRow = ExcelUtils.getRowContains(sTestCaseName,0);
 
     Object[][] testObjArray = ExcelUtils.getTableArray("C:\\Users\\manojjai\\Documents\\dataprovider.xlsx","Sheet1",iTestCaseRow);
 
      return (testObjArray);
 
  }
 
}

TestNG Data Provider

When you need to pass complex parameters for example created from Java (complex objects, objects read from a property file or a database, etc…), in such cases parameters can be passed using Dataproviders. A Data Provider is a method annotated with @DataProvider. A Data Provider returns an array of objects.

Let us check out the same Search examples using Dataproviders.

Video Tutorial -

 

How to do it ?

1)  Define the method searchValue() which is defined as a Dataprovider using the annotation. This method returns array of object array.

2) Add a method test() to your DataProviderTest class. This method takes two strings as input parameters.

3) Add the annotation @Test(dataProvider = “search”) to this method. The attribute dataProvider is mapped to “search”.

4) Run the below code as testNG test case then the test will execute two times as we are passing two Data Values.

package test;

import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
 
import org.testng.annotations.DataProvider;
 
import org.testng.annotations.Test;
 
public class TestNGDataProvider {
 
 private static WebDriver driver;
 
  @DataProvider(name = "search")
 
  public static Object[][] searchValue() {
 
        return new Object[][] { { "appium" }, { "selenium" }};
 
  }
 
  // Here we are calling the Data Provider object with its Name
 
  @Test(dataProvider = "search")
 
  public void test(String searchtext) throws Exception {
 
   System.setProperty("webdriver.chrome.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
  
   driver=new ChromeDriver();
      
      driver.get("http://www.qaautomated.com");
 
      Thread.sleep(5000);
      driver.findElement(By.xpath("*//input[@class='search-field']")).sendKeys(searchtext);
      
      driver.findElement(By.xpath("*//input[@class='search-submit']")).click();
 
      driver.quit();
  }
 
}

TestNG Parameters

As we all know the value of Parameterization in automation testing. It allows us to automatically run a test case multiple times with various inputs and validation values. while creating a automation framework we need to put in some effort to support data driven testing in our automated tests. In this video we will see how we can parameterize a test case using TestNG parameters.

TestNG provides us interesting feature called TestNG Parameters. TestNG lets you pass parameters directly to your test methods with your testng.xml.

Video Tutorial -

How to do it?

Let me take a very simple example of Search on our site www.qaautomated.com, where the search text  is required to perform authentication.

1) Create a test case to perform Search which takes the one string argument as searchtext.

2) Provide searchtext as parameter using TestNG Annotation.

package test;


import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Test;
import org.testng.annotations.Parameters;
 
public class TestNGParameters{
 
 private static WebDriver driver;
 
  @Parameters("searchtext")
  @Test  
  public void test(String searchtext) throws InterruptedException {
 
   System.setProperty("webdriver.chrome.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
  
   driver=new ChromeDriver();
      
      driver.get("http://www.qaautomated.com");
 
      Thread.sleep(5000);
      driver.findElement(By.xpath("*//input[@class='search-field']")).sendKeys(searchtext);
      
      driver.findElement(By.xpath("*//input[@class='search-submit']")).click();
 
      driver.quit();
 
  }
 
}

3) Provide Search text value in testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Suite">
  <test thread-count="5" name="Test">
   <parameter name="searchtext" value="appium"/>
    <classes>
      <class name="test.TestNGParameters"/>
    </classes>
  </test> <!-- Test -->
</suite> <!-- Suite -->

4) Right click on testng.xml and run as TestNG Suit

TestNG Priority and Sequencing

Hi everyone, in this post we will see how to priorities Test cases in TestNG and how to sequence it as per our requirement. If you create a test class and add multiple test cases and run it then each time test case execution sequence is randomly selected .
With TestNG annotations @Test and parameter "priority" we can define sequence and priority for executing test cases. Let us consider the below example and if you run the test class given below multiple times you can see that each time execution sequence/flow is different hence to define a fixed execution flow we user TestNG annotation priority.

Video Tutorial -

package test;
import org.testng.annotations.Test;
 
public class TesNGSequencing {
 
 
 
  @Test
 
  public void One() {
 
      System.out.println("Test Case number One");
 
  }
 
  @Test
 
  public void Two() {
 
   System.out.println("Test Case number Two");
 
  }
 
  @Test
 
  public void Three() {
 
   System.out.println("Test Case number Three");
 
  }
 
  @Test
 
  public void Four() {
 
   System.out.println("Test Case number Four");
 
  }
 
}

Given below is the example for using priority annotations. When you run the below test class then each time execution flow is constant and starts with test case having 0 priority.

package test;
import org.testng.annotations.Test;
 
public class TesNGSequencing {
 
 
 
  @Test(priority=0)
 
  public void One() {
 
      System.out.println("Test Case number One");
 
  }
 
  @Test(priority=1)
 
  public void Two() {
 
   System.out.println("Test Case number Two");
 
  }
 
  @Test(priority=2)
 
  public void Three() {
 
   System.out.println("Test Case number Three");
 
  }
 
  @Test(priority=3)
 
  public void Four() {
 
   System.out.println("Test Case number Four");
 
  }
 
}

Skipping a Test -

If you come across a situation where there is requirement to skip some tests then in this case you can do it using @Test annotations and enabled parameter.

package test;
import org.testng.annotations.Test;
 
public class TesNGSequencing {
 
 
 
  @Test(priority=0, enabled=false)
 
  public void One() {
 
      System.out.println("Test Case number One");
 
  }
 
  @Test(priority=1)
 
  public void Two() {
 
   System.out.println("Test Case number Two");
 
  }
 
  @Test(priority=2)
 
  public void Three() {
 
   System.out.println("Test Case number Three");
 
  }
 
  @Test(priority=3)
 
  public void Four() {
 
   System.out.println("Test Case number Four");
 
  }
 
}

TestNG Assertions and Difference Between Hard Assert and Soft Assert

Hi everyone, In this post we are going to cover very important feature of TestNG which is TestNG Assertions. Why this is important? because no test case is considered complete unless we have assert actual values with expected values.Assertions helps us to verify weather the test case is passed or failed.

There are three types of Assertions in TestNG Hard Assert, Soft Assert . Let us learn all these in detail.

Video Tutorial

What is Assertion and How it Works ?

Consider below example where we are launching google.com and asserting that title should be equals to Google. this is the simplest example of assertion and this is how test case with assertion looks like.

@Test
 public void testCaseVerifyHomePage() {
  System.setProperty("webdriver.chrome.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
  WebDriver driver=new ChromeDriver();
  driver.navigate().to("http://google.com");
  Assert.assertEquals("Gooooogle", driver.getTitle());
 }

Now if you change your code to assert the title with Expected value as "Doodle" then it will throw Exception that Expected - Doodle but Found - Google.

@Test
 public void testCaseVerifyHomePage() {
  System.setProperty("webdriver.chrome.driver", "C:\\Softwares\\chromedriver_win32\\chromedriver.exe");
  WebDriver driver=new ChromeDriver();
  driver.navigate().to("http://google.com");
  Assert.assertEquals("Doodle", driver.getTitle());
 }

In this way assertions helps us to make sure test case is passed or failed and we can catch the bugs with these types of assertions.

Apart from assertEquals there are various types of Assertions as per given below.

Commonly Used Assertions -

assertEqual(String actual,String expected) :- It takes two string arguments and checks whether both are equal, if not it will fail the test.

assertEqual(String actual,String expected, String message) :- It takes three string arguments and checks whether both are equal, if not it will fail the test and throws the message which we provide.

assertEquals(boolean actual,boolean expected) :- It takes two boolean arguments and checks whether both are equal, if not it will fail the test.

assertEquals(java.util.Collection actual, java.util.Collection expected, java.lang.String message) :- Takes two collection objects and verifies both collections contain the same elements and with the same order. if not it will fail the test with the given message.

assertTrue(condition) :- It takes one boolean arguments and checks that a condition is true, If it isn't, an AssertionError is thrown.

assertTrue(condition, message) :- It takes one boolean argument and String message. It Asserts that a condition is true. If it isn't, an AssertionError, with the given message, is thrown.

assertFalse(condition) :- It takes one boolean arguments and checks that a condition is false, If it isn't, an AssertionError is thrown.

assertFalse(condition, message) :- It takes one boolean argument and String message. It Asserts that a condition is false. If it isn't, an AssertionError, with the given message, is thrown.

Hard Assert -  

Hard Assert is a type of TestNG assetion which throws Exception Immediately if the assert statement fails and move on to the next testcase and if there is any code in the current test case after assert statement it will not execute that statement. The google example which we saw earlier is also an example of hard assert. you can try adding code  after assert statement it will not get executed.
In the below example after first assert statement execution the test case execution stops and further statements in the test case are not executed.

@Test
 public void test1(){
   
   Assert.assertTrue(5<1);
   System.out.println(“Assertion Failed”);
   Assert.assertFalse(1<5);
   System.out.println(“Assertion Failed”);
   Assert.assertEquals(“Passed”, “Failed”);
   System.out.println(“Assertion Failed”);
 }

Soft Assert - 

To deal with the disadvantage of Hard Assertions, customized error handler provided by TestNG is called Soft Assertion. Soft Assertions are the type of assertions that do not throw an exception when an assertion fails and would continue with the next step after assert statement. This is usually used when our test requires multiple assertions to be executed and the user want all of the assertions/codes to be executed before failing/skipping the tests.

@Test
 public void test1(){
   SoftAssert sa= new SoftAssert();
   sa.assertTrue(5<1);
   System.out.println(“Assertion Failed”);
   sa.assertFalse(1<5);
   System.out.println(“Assertion Failed”);
   sa.assertEquals(“Passed”, “Failed”);
   System.out.println(“Assertion Failed”); 

                        sa.assertAll()

 }

TestNG Grouping and Depends On

In our previous post we have learnt about TestNG annotations . In this post we will see how we can group test cases and specify dependent test cases using TestNG annotations.

‘Groups‘ is one annotation of TestNG which can be used in the execution of multiple tests as a group. Let’s say you have hundred tests of class Devices and in it ten methods of laptops, ten method of Notebooks and so on. You probably like to run all the Notebooks tests together in a batch. And you want all to be in a single test suite. With the help of grouping you can efficiently address this situation.

How to do it…

1) Create two methods for Laptops, two methods for Notebooks and one method in conjunction with Laptops and Notebooks.

2) Group them separately with using  (groups = { ” Group Name” })

Video Tutorial -

 

import org.testng.annotations.Test;
 
public class TestGrouping {
 
  @Test (groups = { "Laptops" })
 
  public void laptop1() {
 
   System.out.println("Batch Laptops- Test Laptops1");
 
  }
 
  @Test (groups = { "Laptops" })
 
  public void laptop2() {
 
   System.out.println("Batch Laptops- Test Laptops 2");
 
  }
 
  @Test (groups = { "Notebooks" })
 
  public void notebook1() {
 
   System.out.println("Batch Notebooks- Test Notebooks 1");
 
  }
 
  @Test (groups = { "Notebooks" })
 
  public void notebook2() {
 
   System.out.println("Batch Notebooks- Test Notebooks2");
 
  }
 
  @Test (groups = { "Laptops", "Touch Screen Laptops" })
 
  public void youchScreenlaptop() {
 
   System.out.println("Batch Touch Screen Laptops- Test Touch Screen Laptops1");
 
  }
 
}

Add below code into TestNG.xml-

<suite name="Suite">
 
    <test name="Practice Grouping">
 
        <groups>
 
     <run>
 
  <include name="Laptops" />
 
     </run>
 
 </groups>
 
 <classes>
 
     <class name="PackageName.TestGrouping" />
 
 </classes>
 
    </test>
 
</suite>

Run The TestNG,xml -> right click -> Run as -> TestNG Suit.

Dependenct Test -

you may need to invoke methods in a Test case in a particular order or you want to share some data and state between methods. This kind of dependency is supported by TestNG .
TestNG allows you to specify dependencies with two options given below.

• Using attributes dependsOnMethods in @Test annotations
• Using attributes dependsOnGroups in @Test annotations.

import org.testng.annotations.Test;
 
public class Dependent {
 
  @Test (dependsOnMethods = { "OpenBrowser" })
 
  public void SignIn() {
 
   System.out.println("This will execute SignIn");
 
  }
 
  @Test
 
  public void OpenBrowser() {
 
   System.out.println("This will execute Open Browser");
 
  }
 
  @Test (dependsOnMethods = { "SignIn" })
 
  public void SignOut() {
 
   System.out.println("This will execute Log Out");
 
  }

In case of any feedback and questions post in the comment section below.