Selenium

Testing Frameworks

Created by Kavan Sheth



Best viewed in Chrome, Firefox, Opera or A browser supporting HTML5. Use arrow keys ( up ↑, down ↓, left ←, right ↑) to navigate. Uses Reveal.js by Hakim El Hattab Press ESC to enter the slide overview.

Introduction

  • Though we are using selenium for testing, how will you write your test cases?
  • Selenium WebDriver is just an API and you call it from your Java source code.
  • So problem solved :)
  • Java Community is using Unit Testing FrameWorks for long and similar can be used for Acceptance testing using Selenium
  • JUnit and TestNG are widely known Test Driven Development(TDD) frameworks. So even though we are not performing TDD, we can use these frameworks for Acceptance testing.
  • Apart from these Cucumber-Java and JBehave are also well-known Behaviour Driven Development(BDD) frameworks, which focuses more on behavioural aspects of each testcase.
  • But one thing I feel(no supporting proofs as of now) is, Framework which provides you high flexibility will most probably be beneficial in long run.
  • One more thing, this is not the END. there are number of Frameworks/tools available. Use whichever suits your needs. Prototyping will help you in understanding your needs.

JUnit



Simple JUnit Program

If you go to junit.org -> Getting Started, you will get following code:

package com.example.foo;
import org.junit.Test;
import org.junit.Ignore;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;
/**
 * Tests for {@link Foo}.
 * @author user@example.com (John Doe)
 */
public class FooTest {
    @Test
    public void thisAlwaysPasses() {
    }

    @Test
    @Ignore
    public void thisIsIgnored() {
    }
}


So as shown in example, @Test annotation denote test methods, you can have multiple test methods in a class. Apart from this JUnit supports few more annotation but it is nothing but something additional and you can use it as per your requirement.
Now question is how to execute these testcases?

Run JUnit Tests

From Console:

  • trigger Runner from main method (Runners are classes defined by Junit framework to run Tests)
  • public class MyRunner {
      public static void main(String[] args) {
        Result result = JUnitCore.runClasses(MyClassTest.class);
        for (Failure failedTests: result.getFailures()) {
          System.out.println(failedTests.toString());
        }
      }
    } 
    
  • use Runner from Console(as we saw earlier)
  • java -cp ".;libs\*" org.junit.runner.JUnitCore com.example.tests.TestData

From Eclipse:
JUnit is supported by default in Eclipse. so you can create JUnit Tests from New -> JUnit Test Case. It will ask you to add Junit jar in Build path. and You can run your tests by right clicking on your class file and Select Run As -> Junit Test. It will run all Test Methods(marked with @Test Annotation) from your class.

JUnit Annotations

Annotation About
@Test Tells JUnit that which public void Method to be considered as Test.
@Before Method marked with this annotation will run before each Test. it is used to set pre requisite before running test.
@After Method marked with this annotation will run after each Test. it is used to release resources after running test.
@BeforeClass This annotation used with methods, which are expected to run once before starting with tests run. Methods marked with this annotation need to be defined as static to work with JUnit.
@AfterClass Similar to BeforeClass, Just runs after dealing with all tests of a class
@Ignoring If you want to ignore a Test method, because of whatever reason, you can use @Ignore annotation

JUnit Assertions

  • in Test Methods you can use Assertions to assert that your test is running as expected, if Assertion fails your test will fail.
  • You can also use fail method , to explicitly fail your Test case.
  • Example:
  • import org.junit.Test;
    import static org.junit.Assert.*;
    public class TestAssertions {
       @Test
       public void testAssertions() {
          //test data
          String str1 = new String ("abc");
          String str2 = new String ("abc");
    	  
    	  //Check that two objects are equal
          assertEquals(str1, str2);
        }
    	@Test
    	public void testAssertions2() {
    	  //test Steps
          String str3 = null;
    	  
    	  //Check that an object is null
          assertNull(str3);
        }
    }
    

JUnit Assertions

source:http://junit.sourceforge.net/javadoc/org/junit/Assert.html

Hamcrest Matcher

There are two more assertions in JUnit

So Here you can use Core hamcrest Matchers or Junit Matchers for complex matching operations as following:

		@Test
  public void HamcrestCoreMatchers() {
	org.hamcrest.CoreMatchers.assertThat("AnyString", allOf(equalTo("AnyString"), startsWith("Any")));
	assertThat("Any", not(allOf(equalTo("bad"), equalTo("good"))));
	assertThat("Yes", anyOf(equalTo("Yes"), equalTo("yes")));
	assertThat(7, not(CombinableMatcher.<Integer> either(equalTo(3)).or(equalTo(4))));
	assertThat(new Object(), not(sameInstance(new Object())));
}
@Test
  public void JunitMatchers() {
	org.junit.Assert.assertThat(Arrays.asList(new String[] { "fun", "ban", "net" }), everyItem(containsString("n")));
	org.junit.Assert.assertThat(Arrays.asList("one", "two", "three"), hasItems("one", "three"));
	org.junit.Assert.assertThat("albumen", both(containsString("a")).and(containsString("b")));
} 
		

Parameter for @Test Annotation


  • @Test(timeout=1000)
  • If Method takes more than 1000 millisecond , it will fail with an exception
  • @Test(expected = Exception.class)
  • Test case will fail if method does not throw expected Exception. For More details on Exception Testing visit documentation.

Runners


  • As we discussed briefly, JUnit uses runner to run tests from your class
  • By Default JUnit uses BlockJUnit4ClassRunner, You can change it by using @RunWith Annotation
  • JUnit has Three specialized Runners for different purpose:
    • Suite
    • Parameterized
    • Categories

Suite Runner


Using this you can create Suite of Testd from multiple classes

Suite has an annotation named @SuiteClasses(TestClass1.class, ...)

Example(source):

import org.junit.runner.RunWith;
		import org.junit.runners.Suite;
		@RunWith(Suite.class)
		@Suite.SuiteClasses
		({
		  TestFeatureLogin.class,
		  TestFeatureLogout.class,
		  TestFeatureNavigate.class,
		  TestFeatureUpdate.class
		})	
		public class FeatureTestSuite {
			// the class remains empty,
			// used only as a holder for the above annotations
		}

So in above example Junit will run all test from all the classes specified under @SuiteClasses annotation.

Parameterized Runner

Using this Runner you can run same test method with different parameters


Parameterized has an annotation named @Parameters, Which specify that method annotated with it is returning object of type Collection<Object[]>


You need to make sure that your constructor is taking arguments in same order as it is defined under parameterized method.

   Example(source):

@RunWith(Parameterized.class)
public class parameterizedTest {
    @Parameters
    public static Collection<Object[]> data(){
        return Arrays.asList(new Object[][] {     
                 { 0, 0 }, { 1, 1 }, { 2, 1 }
           });
    }
    private int Input;
    private int Expected;
    public parameterizedTest(int in, int ex){
        Input= input;
        Expected= expected;
    }
    @Test
    public void test() {
        assertEquals(Expected, 
		anotherMethod(Input));
    }
}

Categories Runner


  • It is kind of a suite.
  • It has four annotations @Category, @IncludeCategory, @ExcludeCategory, @SuiteClasses
  • You can use interface or class as Category
  • You can assign one or more category to a method or class using @Category Annotation like, @Category(a.class) or @Category({a.class,b.class})
  • So when you run tests with this runner and specify @IncludeCategory(a.class), it will only run methods marked with category a.class, similarly you can exclude a category using @ExcludeCategory
  • Example

Test Execution Order


  • @FixMethodOrder(MethodSorters.JVM): Leaves the test methods in the order returned by the JVM. This order may vary from run to run.
  • @FixMethodOrder(MethodSorters.NAME_ASCENDING): Sorts the test methods by method name, in lexicographic order.
  • For More Details on Sorters

Assume


  • You can use assume methods to specify that before moving ahead you are assuming something.
  • So if that assumption is not true Junit will ignore that test
  • e.g assumeThat(File.separatorChar, is('/')), so if this assumption fails, test will be ignored.

To Be ...


  • Rules
  • Theories
  • Test fixtures

Reporting

TestNG

Testing, The Next Generation

Pronounced: 'testing'#



Simple TestNG test

import org.testng.annotations.Test;
import org.testng.TestNG;
import org.testng.annotations.Test;
/**
 * Tests for {@link Foo}.
 * @author user@example.com (John Doe)
 */
public class testngTest {
    @Test
    public void thisAlwaysPasses() {
    }
    @Test(enabled = false)
    public void thisIsIgnored() {
    }
}

when compared to JUnit program, only difference is in imports and instead of @Ignore we have @Test(enabled = false)

TestNG is similar to JUnit, but has differences in annotations and functionalities, Once we discuss it, it is upto you to decide, which you find more suitable.

One Major Difference is, unlike JUnit, TestNG is xml driven. so all configurations regarding tests will be maintained in an xml file.

Run TestNG Tests

You will have to create TestNG xml as following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Default suite">
  <test verbose="2" name="Default test">
    <classes>
      <class name="com.java.test.testngTest"/>
    </classes>
  </test> <!-- Default test -->
</suite> <!-- Default suite -->

Once you are ready with testng.xml and you can compile/run your code as following: (Assuming Structure HOME -> com -> java -> test -> testngTest.java. and your testng.xml and testng.jar (download) is in HOME folder.)

javac -cp "testng.jar" com/java/test/testngTest.java
java -cp "testng.jar;." org.testng.TestNG testng.xml

Reporting

TestNG has very good in-built Reporting Capability, which is one of the most basic need of framework.

In HOME, a Folder with name
test-output will be created.
Open index.html

Download to Explore

Console output:



Run TestNG test - again

As you noticed, to run test from console we initiated class org.testng.TestNG and passed xml as argument.

Similar thing you can do from code as well. just add following main method in your code and run your code from console as normal java program. it will dynamically create xml file and execute your tests.

public static void main(String[] args) {
	XmlClass clazz = new XmlClass(testngTest.class);
	List<XmlClass> classes = new ArrayList<XmlClass>();
	classes.add(clazz);

	List<XmlSuite> suites = new ArrayList<XmlSuite>();
	XmlSuite suite = new XmlSuite();
	List<XmlTest> tests = new ArrayList<XmlTest>();
	XmlTest test = new XmlTest();

	test.setXmlClasses(classes);
	test.setSuite(suite);
	test.setName("Default suite");
	tests.add(test);
	suite.setTests(tests);
	suite.setName("Default test");
	/* Only if need Parallel test run, we will see later
	suite.setParallel(XmlSuite.PARALLEL_METHODS);
	suite.setThreadCount(3);
	suite.setDataProviderThreadCount(3);
	*/
	suites.add(suite);
	suite.setTests(tests);
	TestNG testng = new TestNG(false);
	testng.setXmlSuites(suites);
	testng.setVerbose(10);
	testng.run();
}

Run TestNG test -once again


Running TestNG test from eclipse is also simple task. Install TestNG plugin.
If you want to run all test of a class, then just right click and select Run As -> TestNG test.
Detailed tutorial on how to create TestNG test and configuration is provided at http://testng.org/doc/eclipse.html

TestNG.xml


You can always create your xml based on examples available on internet or on documentation page. but best way to master it is by understanding its DTD(Document Type Definitions). In the xml see the top most line, you will find detail of DTD the xml is following.

A good source to understand DTD is available here and by understanding DTD you can be sure that your xml is of desired format. For basic TestNG it is not required to go through pain of learning DTD, but have a look at it, if you want to master TestNG or curious enough.

TestNG Groups

Apart from using classes, package names, methods to distinguish and identifying tests to be run, TestNG used one more parameter named Group, Which can be used with @Test annotation like this..

public class Test1 {
  @Test(groups = { "functest", "checkintest" })
  public void testMethod1() {
  }
  @Test(groups = {"functest", "checkintest"} )
  public void testMethod2() {
  }
  @Test(groups = { "functest" })
  public void testMethod3() {
  }
}

And later you can control test execution using this group names, like the group you want to test, or exclude from test.

Now we can see different annotations in TestNG.

@Test Annotation

@Before/@After Annotations

As compared to Junit, TestNG supports:

@BeforeSuite
@AfterSuite
@BeforeTest
@AfterTest
@BeforeGroups
@AfterGroups
@BeforeClass
@AfterClass
@BeforeMethod
@AfterMethod
Details can be found here.

Which also supports parameters like alwaysRun, dependsOnGroups, dependsOnMethods, enabled, groups, inheritGroups.

So TestNG tried to give a lots of power by providing variety of annotations and parameters. We will see few more important annotations with examples.

@DataProvider Annotations

@DataProvider(name = "test", parallel=true)
  public static Object[][] someData() { 
      return new Object[][] {
    	{ "Cedric", new Integer(36) },
		{ "Anne", new Integer(37)},
      };
  }  
@Test(dataProvider = "test", 
  dataProviderClass="OnlyIfDataProviderInAnother.class")
  public void testData(String name, Integer age) {
	  System.out.println("Name: " + name + ", Age: " + age);
  }  

@DataProvider Annotation indicate data provider method, and dataProvider parameter of @Test Annotation identifies data provider. So here testData method will be called twice with sets of data { "Cedric", 36} and { "Anne", 37 }. Also as parallel parameter of @DataProvider is True, it will create 5 thread by default. if you want to change it, you can either specify it in xml as
    <suite name="Suite1" data-provider-thread-count="20" >
If you are generating dynamic xml from code, use following code:
    suite.setParallel(XmlSuite.PARALLEL_METHODS);
    suite.setThreadCount(3);
    suite.setDataProviderThreadCount(3);

you can also use command line switch -dataproviderthreadcount 3
Just one point, what you are putting inside dataProvider method has no limit except return type, you can fetch data from DB, Excel file, csv file or any other way you like...

@Parameters Annotations

Example from JavaDocs:

@Parameters({ "xmlPath" })
@Test
public void verifyXmlFile(String path) { ... }

and in testng.xml:

    <parameter name="xmlPath" value="account.xml" />

@Parameters annotation is very powerful mechanism, if you want some configurable parameters for your test, you don't need to maintain separate config or property file. You just need to mention the @Parameters annotation with different parameters and these parameters should be present in your testng.xml as shown above. Here, TestNG will read xmlPath parameter from testng.xml and pass its value as argument to verifyXmlFile in path variable.

@Factory Annotations

public class WebTestFactory {      
  //createInstances method will create 10 
  //objects of WebTest class
  @Factory     
  public Object[] createInstances() {      
   Object[] result = new Object[10];       
   for (int i = 0; i < 10; i++) {      
      result[i] = new WebTest(i);      
    }      
    return result;     
} 
public class WebTest {     
  private int m_numberOfTimes;     
  public WebTest(int numberOfTimes) {      
    m_numberOfTimes = numberOfTimes;       
  }    

  @Test    
  public void testServer() {       
   //Code to test the application   
  }    
}    

@Factory can be used to create your Test class dynamically, each test class can have all annotations/parameters. It must return Object[]. Here WebTestFactory will create object array of WebTest test class, each WebTest instance is created with a different parameter. So Test may be written such way that it may behave differently for each parameter. You can use @Factory with @DataProvider and @Parameters. And in testng.xml you just need to mention
   <class name="WebTestFactory"/>

@Listeners Annotation

TestNG provides several interfaces to modify its behaviour.
Standard TestNG Documentation and javaDoc has sufficient information on these listeners, I have just mentioned purpose of each listener so you can choose listener of your need and understand power of listeners.

IAnnotationTransformer	
	- to modify the content of @Test the annotation at runtime
IAnnotationTransformer2	
	- to modify the content of @Factory or @DataProvider the annotation at runtime
IHookable				
	- to Override or skip Invocation of Test methods
IInvokedMethodListener	
	- allows you to be notified whenever TestNG is about to invoke a test. 
IMethodInterceptor		
	- Control the order in which methods will be run
IReporter				
	- to Generate Customized Report
ISuiteListener			
	- Invoked on start and finish of a Suite
ITestListener			
	- Invoked on different interfacing points like , onStart, onFinish, onTestFailure, onTestSuccess ..

@Listeners Example

Following is a good example of using this listener I found on toolsqa.com

package net.mylearnings.test;

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ISuite;
import org.testng.ISuiteListener;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestNGMethod;
import org.testng.ITestResult;
import org.testng.Reporter;
 
public class myLogger implements ITestListener, ISuiteListener, IInvokedMethodListener {
 
    //**ISuiteListener Methods
    @Override
    public void onStart(ISuite arg0) {
        Reporter.log("About to begin executing Suite " + arg0.getName(), true);
    }
 
    @Override
    public void onFinish(ISuite arg0) {
         Reporter.log("About to end executing Suite " + arg0.getName(), true);
    }
 
    //**ITestListener Methods 
    @Override
    public void onStart(ITestContext arg0) {
         Reporter.log("About to start test but has not yet invoked Configuration Annotations" + arg0.getName(), true);
    }
    
    @Override
    public void onFinish(ITestContext arg0) { 
        Reporter.log("Completed executing test " + arg0.getName(), true);
    }
 
    @Override
    public void onTestSuccess(ITestResult arg0) {
        printTestResults(arg0);
    }
 
    @Override
    public void onTestFailure(ITestResult arg0) {
        printTestResults(arg0); 
    }
    
    @Override
    public void onTestStart(ITestResult arg0) { 
        System.out.println("The execution of the main test starts now");
    }
 
    @Override 
    public void onTestSkipped(ITestResult arg0) {
         printTestResults(arg0);
    }

    private void printTestResults(ITestResult result) {
 
        Reporter.log("Test Method resides in " + result.getTestClass().getName(), true);
 
        if (result.getParameters().length != 0) {
            String params = null;
            for (Object parameter : result.getParameters()) {
                params += parameter.toString() + ","; 
            }
            Reporter.log("Test Method had the following parameters : " + params, true);
        }
 
        String status = null;
        switch (result.getStatus()) {
        case ITestResult.SUCCESS:
            status = "Pass";
            break;
 
        case ITestResult.FAILURE:
            status = "Failed";
            break;
 
        case ITestResult.SKIP:
            status = "Skipped";
        }
 
        Reporter.log("Test Status: " + status, true);
    }
 
    //**IInvokedMethodListener Methods
    @Override
    public void beforeInvocation(IInvokedMethod arg0, ITestResult arg1) { 
        String textMsg = "About to begin executing following method : " + returnMethodName(arg0.getTestMethod());
        Reporter.log(textMsg, true);
    }
 
    @Override
    public void afterInvocation(IInvokedMethod arg0, ITestResult arg1) { 
        String textMsg = "Completed executing following method : " + returnMethodName(arg0.getTestMethod());
        Reporter.log(textMsg, true);
    }
 
    private String returnMethodName(ITestNGMethod method) {
        return method.getRealClass().getSimpleName() + "." + method.getMethodName();
    }

	@Override
	public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
		// TODO Auto-generated method stub
	}
}

@Listeners Example cont.

We created a customer Listener class, But how to use it? There are multiple ways:

  • In testng.xml
  • <listeners>
        <listener class-name="net.mylearnings.test.myLogger" />
     </listeners>
     
  • Using @Listeners Annotation
  • @Listeners({ net.mylearnings.test.myLogger.class, net.mylearnings.test.mayHaveAnyOtherListner.class })
    public class MyTest {
      // ...
    }
    
    except IAnnotationTransformer and IAnnotationTransformer2, these two listeners must called from xml as they deal with annotations. Also annotation will be applicable to whole suite.
  • Using -listener net.mylearnings.test.myLogger in command line

Behaviour Driven Development Frameworks

  • BDD frameworks are developed to focus more on behaviour of system.
  • Developer of BDD, Dan North, developed it because of lack of any specification in TDD regarding what should be tested and how.*
  • Now if you are linking specification with testcase,
    • It should be tightly coupled with your implementation, other wise ensuring that implementation is as per specification would be another headache.
    • Also specification should have been maintain such that no technical skills should be required to understand/review it.
  • And both of this are supported by these BDD frameworks.
  • If I am not clear with this brief intro, I hope it will be clear once you see example in upcoming sections.
  • Also note that though these frameworks were developed for Unit testing and test driven development, it can be used for acceptance tests.

Cucumber-JVM



Below image from cukes.info explains BDD using Cucumber. In acceptance testing, step 4-5 can be replaced with bug finding and fixing.

How to Use?

Here we will be using Cucumber without Maven.

  • Download following from here
    • cucumber-core-1.2.0.jar
    • cucumber-java-1.2.0.jar
    • cucumber-junit-1.2.0.jar
    • gherkin-2.12.2.jar
    • cucumber-jvm-deps-1.0.3.jar
  • Create a new project and define directory structure as per required package.
  • Create libs folder manually under your project and copy all jars to it.
  • Add all jars to build path(from Project properties -> Java Build Path tab). For now don't worry about files shown in directory structure image.
  • Apart from these you will need selenium jars also.

How to Use?(using Maven and Eclipse)


  • Create a Maven Project in Eclipse.
    • either check "create a simple project (skip archetype selection)"
    • or select archetype maven-archetype-quickstart (you have to create resource folders)
    • provide details of Group Id, Artifact id and package
  • Once your folder structure is ready, (expecting something as shown in right side image)
    • your page objects and other test related code can be under src/test/java/<package>/
    • your .feature files can be under src/test/java/resources/<package>/

How to Use?(using Maven and Eclipse) - Cont.

  • now you need to update your POM.xml, POM should be something like following:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>CucumberTest</groupId>
  <artifactId>CucumberTest</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>CucumberTest</name>
  <dependencies>
    <dependency>
        <groupId>info.cukes</groupId>
        <artifactId>cucumber-picocontainer</artifactId>
        <version>1.2.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>info.cukes</groupId>
        <artifactId>cucumber-junit</artifactId>
        <version>1.2.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.11</version>
        <scope>test</scope>
    </dependency>
     <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>2.42.2</version>
    </dependency>
</dependencies>
 <reporting>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-report-plugin</artifactId>
        <version>2.18</version>
      </plugin>
    </plugins>
  </reporting>
</project>
  • That's it. just right click on your project folder and Run As -> Maven test.
  • here we are using cucumber-picocontainer, it is a framework for dependency injection. Though we are not using it here, just added as dependency because it implicitly adds all required dependencies like cucumber-core, cucumber-junit.. etc

Example

  • For Cucumber and Jbehave we will use same example we used for page object model. So if you have not visited design pattern section, please visit it first.
  • Here we will be having same scenario of searching flight for a destination
  • So as a first step for BDD, we need to define behaviour, for that we need to create .feature file, feature file follows gherkin syntax.
  • "Gherkin is a Business Readable, Domain Specific Language created specifically for behaviour descriptions"
  • Will add more details about syntax of gherkin as and when needed, but for now just need to create a feature file as following: FlightSearch.feature
Feature: FlightSearch
Scenario: search result for a oneway trip from NewYork to London on 25th in Business Class in Unified Airlines 
	Given you are logged-in to "http://newtours.demoaut.com/" with username "test1" and password "test1"
	When select OneWayTrip
	And select FromPort "New York"
	And select ToPort "London"
	And select FromDay "25"
	And select BusinessClass
	And select airlines "Unified Airlines"
	And click on search button  
	Then search result should be displayed

Features

  • In Cucumber, We define a feature and then write scenarios corresponding to that feature.
  • for each feature file you need to follow few rules.(Gherkin syntax)
  • Cucumber looks for few keywords in your features and identifies scenarios and steps for a particular scenario.
  • Basic keywords you need to keep in mind are
    • Feature: Brief feature detail
    • Scenario: Followed by description of that scenario
    • Each Step in Scenario should start with a keyword, It can be any of the following:
    • Given: A precondition/state for your scenario
    • When: Activity performed
    • And: Same as When, Just used to make scenario reading more logical
    • Then: Expected Result
  • Note that input data is also part of your scenario, which you need to use during execution of that scenario.

Mapping between Scenarios and Actual Automated Steps

  • Now instead of writing scenario in a simple JUnit Test as we were doing earlier
  • WebDriver driver = new ChromeDriver();	    
    driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    driver.get(baseUrl);
    
    LoginPage loginPage=PageFactory.initElements(driver, LoginPage.class);
    loginPage.setUserName("test")
    		 .setPassword("test")
    		 .clickLoginButton();
    FlightFinderPage flightFinderPage=PageFactory.initElements(driver, FlightFinderPage.class);
     
    flightFinderPage.selectOneWayTrip()
    				.setFromPort("New York")
    				.setToPort("London")
    				.selectFromDay("25")
    				.setBusinessClass()
    				.selectAirline("Unified Airlines");
    				
    FlightFinderResultPage flightFinderResultPage = flightFinderPage.clickFindFlights();			    
    
    //here you can have some verification of retrieved result
    driver.close();

Mapping between Scenarios and Actual Automated Steps - Cont.

  • You Need to divide it into multiple steps, as per your steps in your scenario.
  • To map between step in story file and step in Java code, you need to use few annotations like @Given, @And, @When, @Then
  • Your java code after segregating test into multiple steps: FlightSearchSteps.java
  • package cukesTests;
    
    import java.io.File;
    import java.util.concurrent.TimeUnit;
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.openqa.selenium.support.PageFactory;
    import cucumber.api.java.en.Given;
    import cucumber.api.java.en.Then;
    import cucumber.api.java.en.When;
    import cucumber.api.java.en.And;
    import cukesPages.*;
    
    public class FlightSearchSteps{
    	WebDriver driver;
    	LoginPage loginPage;
    	FlightFinderPage flightFinderPage;
    	FlightFinderResultPage flightFinderResultPage;
    	
    	@Given("^you are logged-in to \"(.+)\" with username \"(.+)\" and password \"(.+)\"$")
    	public void login( String url, String user,  String pass  )
    	{
    		File file = new File("C:\\chromedriver.exe");        
    	    System.setProperty("webdriver.chrome.driver", file.getAbsolutePath());
    		driver = new ChromeDriver();	    
    	    driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    	    driver.get(url);
    	    loginPage=PageFactory.initElements(driver, LoginPage.class);
    	    if (!loginPage.isLoggedin())
    	    {	
    		    loginPage.setUserName(user)
    		             .setPassword(pass)
    		             .clickLoginButton();
    	    }
    	    else
    	    {
    	    	loginPage.clickflightslink();
    	    }
    	}
    	@When("select OneWayTrip")
    	public void selectOneWayTrip()
    	{	
    		
    		flightFinderPage=PageFactory.initElements(driver, FlightFinderPage.class);
    		flightFinderPage.selectOneWayTrip();
    	}
    	
    	@And("select FromPort \"(.+)\"")
    	public void selectFromPort( String fport)
    	{
    		flightFinderPage.setFromPort(fport);
    	}
    	@And("select ToPort \"(.+)\"")
    	public void selectToPort( String tport)
    	{
    		flightFinderPage.setToPort(tport);
    	}
    	@And("select BusinessClass")
    	public void selectBusinessClass()
    	{
    		flightFinderPage.setBusinessClass();
    	}
    	@And("select FromDay \"(.+)\"")
    	public void selectFromDay( String fday)
    	{
    		flightFinderPage.selectFromDay(fday);
    	}
    	@And("select airlines \"(.+)\"")
    	public void selectAirlines( String airlines)
    	{
    		flightFinderPage.selectAirline(airlines);
    	}
    	@And("click on search button")
    	public void clickSearch()
    	{
    		 flightFinderPage.clickFindFlights();
    	}
    	@Then("search result should be displayed")
    	public void resultVerification()
    	{
    		  //here you can have some verification of retrieved result	
    		driver.close();
    	}
    }

Mapping between Stories and Actual Automated Steps - Cont.

Few Points to be considered for in FlightSearchSteps.java

  • Methods for each step can be in any order in file.
  • for each data input, we replace it using \"(.+)\", which is regular expression to read any characters between double quotes. and it will be passed as argument in sequential order, For Example we are mapping
  • Given you are logged-in to "http://newtours.demoaut.com/" with username "test1" and password "test1"
    to
    @Given("^you are logged-in to \"(.+)\" with username \"(.+)\" and password \"(.+)\"$")
    	public void login( String url, String user,  String pass  )
    So during actual execution, 'http://newtours.demoaut.com/' will be passed as first parameter url, 'test1' will be passed as second parameter user and, second 'test1' will be passed as third parameter password.

Configuration

  • Now how cucumber knows about your feature and the mapping of steps,
  • for that you need to create a dummy class and provide details about feature location, report format and junit runner using annotations.
  • most basic configuration can be as following: RunCukesTest.java
    package cukesTests;
    
    import cucumber.api.CucumberOptions;
    import cucumber.api.junit.Cucumber;
    import org.junit.runner.RunWith;
    
    @RunWith(Cucumber.class)
    @CucumberOptions(
    		features = {"src/cukesFeatures"},
    		plugin = {"pretty"})
    public class RunCukesTest {
    }
  • Now you just need to run RunCukesTest.java as JUnit Test, which will read your features from given path and execute scenarios.

Result

On successful Execution you will get something like following:

Reports

  • cucumber supports multiple report formats inherently, like formatted terminal output(formatter:pretty), html(formatter:html), JSON(formatter:json) and JUnit format(formatter:junit)
  • it can be configured as following:
  • plugin = {"pretty:target/report/report.tty", 
    	"html:target/report/",
    	"json:target/report/cucu_json_report.json",
    	"junit:target/report/cucumber_junit_report.xml"}
    					
  • Here first keyword specifies report format type and followed by name with relative path
  • Note that you will need cucumber-html-0.2.3.jar for report generation. It can be downloaded from https://search.maven.org
  • Also note that in other tutorial you may see @Cucumber.Options( format = { "pretty"} ), but as per JavaDocs format option is deprecated and suggested to use plugin

Pretty Report

  • Pretty report is for formatted terminal output
  • If you will see pretty report file on windows it will looks like file with garbage characters like [0m , but actually these are ANSI shell codes, which can be interpreted by any shell and if you see your report.tty file in cygwin,any unix machine or shell code interpreter, it will look like following:

HTML Report

  • As suggested by name, it is html based Report, with color coding and exception details on failure. as shown in image, green color is for passed steps, red is for failed and blue is for not executed steps

JSON Report

JSON report is in JSON format, as following:

[
  {
    "id": "flightsearch",
    "description": "",
    "name": "FlightSearch",
    "keyword": "Feature",
    "line": 1,
    "elements": [
      {
        "id": "flightsearch;search-result-for-a-oneway-trip-from-newyork-to-london-on-25th-in-business-class-in-unified-airlines",
        "description": "",
        "name": "search result for a oneway trip from NewYork to London on 25th in Business Class in Unified Airlines",
        "keyword": "Scenario",
        "line": 2,
        "steps": [
          {
            "result": {
              "duration": 13958352084,
              "status": "passed"
            },
            "name": "you are logged-in to \"http://newtours.demoaut.com/\" with username \"test1\" and password \"test1\"",
            "keyword": "Given ",
            "line": 3,
            "match": {
              "arguments": [
                {
                  "val": "http://newtours.demoaut.com/",
                  "offset": 22
                },
                {
                  "val": "test1",
                  "offset": 67
                },
                {
                  "val": "test1",
                  "offset": 88
                }
              ],
              "location": "FlightSearchSteps.login(String,String,String)"
            }
          },
          {
            "result": {
              "duration": 74979039,
              "status": "passed"
            },
            "name": "select OneWayTrip",
            "keyword": "When ",
            "line": 4,
            "match": {
              "location": "FlightSearchSteps.selectOneWayTrip()"
            }
          },
          {
            "result": {
              "duration": 122567015,
              "status": "passed"
            },
            "name": "select FromPort \"New York\"",
            "keyword": "And ",
            "line": 5,
            "match": {
              "arguments": [
                {
                  "val": "New York",
                  "offset": 17
                }
              ],
              "location": "FlightSearchSteps.selectFromPort(String)"
            }
          },
          {
            "result": {
              "duration": 107311863,
              "status": "passed"
            },
            "name": "select ToPort \"London\"",
            "keyword": "And ",
            "line": 6,
            "match": {
              "arguments": [
                {
                  "val": "London",
                  "offset": 15
                }
              ],
              "location": "FlightSearchSteps.selectToPort(String)"
            }
          },
          {
            "result": {
              "duration": 150365451,
              "status": "passed"
            },
            "name": "select FromDay \"25\"",
            "keyword": "And ",
            "line": 7,
            "match": {
              "arguments": [
                {
                  "val": "25",
                  "offset": 16
                }
              ],
              "location": "FlightSearchSteps.selectFromDay(String)"
            }
          },
          {
            "result": {
              "duration": 84340712,
              "status": "passed"
            },
            "name": "select BusinessClass",
            "keyword": "And ",
            "line": 8,
            "match": {
              "location": "FlightSearchSteps.selectBusinessClass()"
            }
          },
          {
            "result": {
              "duration": 5007429022,
              "status": "failed",
              "error_message": "org.openqa.selenium.NoSuchElementException: no such element\n  (Session info: chrome\u003d39.0.2171.95)\n  (Driver info: chromedriver\u003d2.10.267521,platform\u003dWindows NT 6.2 x86_64) (WARNING: The server did not provide any stacktrace information)\nCommand duration or timeout: 5.01 seconds\nFor documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html\nBuild info: version: \u00272.42.2\u0027, revision: \u00276a6995d\u0027, time: \u00272014-06-03 17:42:03\u0027\nSystem info: host: \u0027M-R9TG5Y4\u0027, ip: \u0027192.168.1.3\u0027, os.name: \u0027Windows 8\u0027, os.arch: \u0027x86\u0027, os.version: \u00276.2\u0027, java.version: \u00271.7.0_60\u0027\nSession ID: 7b15410612da4cb4483f7785ba43ba15\nDriver info: org.openqa.selenium.chrome.ChromeDriver\nCapabilities [{platform\u003dWIN8, acceptSslCerts\u003dtrue, javascriptEnabled\u003dtrue, browserName\u003dchrome, chrome\u003d{userDataDir\u003dC:\\Users\\KAVAN~1.SHE\\AppData\\Local\\Temp\\scoped_dir1132_10511}, rotatable\u003dfalse, locationContextEnabled\u003dtrue, version\u003d39.0.2171.95, takesHeapSnapshot\u003dtrue, cssSelectorsEnabled\u003dtrue, databaseEnabled\u003dfalse, handlesAlerts\u003dtrue, browserConnectionEnabled\u003dfalse, nativeEvents\u003dtrue, webStorageEnabled\u003dtrue, applicationCacheEnabled\u003dfalse, takesScreenshot\u003dtrue}]\r\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)\r\n\tat sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)\r\n\tat sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)\r\n\tat java.lang.reflect.Constructor.newInstance(Unknown Source)\r\n\tat org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:204)\r\n\tat org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:156)\r\n\tat org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:599)\r\n\tat org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:352)\r\n\tat org.openqa.selenium.remote.RemoteWebDriver.findElementByName(RemoteWebDriver.java:425)\r\n\tat org.openqa.selenium.By$ByName.findElement(By.java:299)\r\n\tat org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:344)\r\n\tat org.openqa.selenium.support.pagefactory.DefaultElementLocator.findElement(DefaultElementLocator.java:59)\r\n\tat org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler.invoke(LocatingElementHandler.java:34)\r\n\tat com.sun.proxy.$Proxy16.getTagName(Unknown Source)\r\n\tat org.openqa.selenium.support.ui.Select.\u003cinit\u003e(Select.java:43)\r\n\tat cukesPages.FlightFinderPage.selectAirline(FlightFinderPage.java:67)\r\n\tat cukesTests.FlightSearchSteps.selectAirlines(FlightSearchSteps.java:74)\r\n\tat ✽.And select airlines \"Unified Airlines\"(FlightSearch.feature:9)\r\n"
            },
            "name": "select airlines \"Unified Airlines\"",
            "keyword": "And ",
            "line": 9,
            "match": {
              "arguments": [
                {
                  "val": "Unified Airlines",
                  "offset": 17
                }
              ],
              "location": "FlightSearchSteps.selectAirlines(String)"
            }
          },
          {
            "result": {
              "status": "skipped"
            },
            "name": "click on search button",
            "keyword": "And ",
            "line": 10,
            "match": {
              "location": "FlightSearchSteps.clickSearch()"
            }
          },
          {
            "result": {
              "status": "skipped"
            },
            "name": "search result should be displayed",
            "keyword": "Then ",
            "line": 11,
            "match": {
              "location": "FlightSearchSteps.resultVerification()"
            }
          }
        ],
        "type": "scenario"
      }
    ],
    "uri": "FlightSearch.feature"
  }
]

JUnit Report

JUnit reports are in XML format and it is mostly understand by Continuous integration tools.

<?xml version="1.0" encoding="UTF-8"?><testsuite failures="1" name="cucumber.runtime.formatter.JUnitFormatter" skipped="0" tests="1" time="19.505345">
<testcase classname="FlightSearch" name="search result for a oneway trip from NewYork to London on 25th in Business Class in Unified Airlines" time="19.505345">
<failure message="org.openqa.selenium.NoSuchElementException: no such element&#10;  (Session info: chrome=39.0.2171.95)&#10;  (Driver info: chromedriver=2.10.267521,platform=Windows NT 6.2 x86_64) (WARNING: The server did not provide any stacktrace information)&#10;Command duration or timeout: 5.01 seconds&#10;For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html&#10;Build info: version: '2.42.2', revision: '6a6995d', time: '2014-06-03 17:42:03'&#10;System info: host: 'M-R9TG5Y4', ip: '192.168.1.3', os.name: 'Windows 8', os.arch: 'x86', os.version: '6.2', java.version: '1.7.0_60'&#10;Session ID: 7b15410612da4cb4483f7785ba43ba15&#10;Driver info: org.openqa.selenium.chrome.ChromeDriver&#10;Capabilities [{platform=WIN8, acceptSslCerts=true, javascriptEnabled=true, browserName=chrome, chrome={userDataDir=C:\Users\KAVAN~1.SHE\AppData\Local\Temp\scoped_dir1132_10511}, rotatable=false, locationContextEnabled=true, version=39.0.2171.95, takesHeapSnapshot=true, cssSelectorsEnabled=true, databaseEnabled=false, handlesAlerts=true, browserConnectionEnabled=false, nativeEvents=true, webStorageEnabled=true, applicationCacheEnabled=false, takesScreenshot=true}]&#13;&#10;&#9;at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)&#13;&#10;&#9;at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)&#13;&#10;&#9;at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)&#13;&#10;&#9;at java.lang.reflect.Constructor.newInstance(Unknown Source)&#13;&#10;&#9;at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:204)&#13;&#10;&#9;at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:156)&#13;&#10;&#9;at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:599)&#13;&#10;&#9;at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:352)&#13;&#10;&#9;at org.openqa.selenium.remote.RemoteWebDriver.findElementByName(RemoteWebDriver.java:425)&#13;&#10;&#9;at org.openqa.selenium.By$ByName.findElement(By.java:299)&#13;&#10;&#9;at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:344)&#13;&#10;&#9;at org.openqa.selenium.support.pagefactory.DefaultElementLocator.findElement(DefaultElementLocator.java:59)&#13;&#10;&#9;at org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler.invoke(LocatingElementHandler.java:34)&#13;&#10;&#9;at com.sun.proxy.$Proxy16.getTagName(Unknown Source)&#13;&#10;&#9;at org.openqa.selenium.support.ui.Select.&lt;init&gt;(Select.java:43)&#13;&#10;&#9;at cukesPages.FlightFinderPage.selectAirline(FlightFinderPage.java:67)&#13;&#10;&#9;at cukesTests.FlightSearchSteps.selectAirlines(FlightSearchSteps.java:74)&#13;&#10;&#9;at ✽.And select airlines &quot;Unified Airlines&quot;(FlightSearch.feature:9)&#13;&#10;"><![CDATA[Given you are logged-in to "http://newtours.demoaut.com/" with username "test1" and password "test1".passed
When select OneWayTrip......................................................passed
And select FromPort "New York"..............................................passed
And select ToPort "London"..................................................passed
And select FromDay "25".....................................................passed
And select BusinessClass....................................................passed
And select airlines "Unified Airlines"......................................failed
And click on search button..................................................skipped
Then search result should be displayed......................................skipped

StackTrace:
org.openqa.selenium.NoSuchElementException: no such element
  (Session info: chrome=39.0.2171.95)
  (Driver info: chromedriver=2.10.267521,platform=Windows NT 6.2 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 5.01 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '2.42.2', revision: '6a6995d', time: '2014-06-03 17:42:03'
System info: host: 'M-R9TG5Y4', ip: '192.168.1.3', os.name: 'Windows 8', os.arch: 'x86', os.version: '6.2', java.version: '1.7.0_60'
Session ID: 7b15410612da4cb4483f7785ba43ba15
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities [{platform=WIN8, acceptSslCerts=true, javascriptEnabled=true, browserName=chrome, chrome={userDataDir=C:\Users\KAVAN~1.SHE\AppData\Local\Temp\scoped_dir1132_10511}, rotatable=false, locationContextEnabled=true, version=39.0.2171.95, takesHeapSnapshot=true, cssSelectorsEnabled=true, databaseEnabled=false, handlesAlerts=true, browserConnectionEnabled=false, nativeEvents=true, webStorageEnabled=true, applicationCacheEnabled=false, takesScreenshot=true}]

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)

	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)

	at java.lang.reflect.Constructor.newInstance(Unknown Source)

	at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:204)

	at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:156)

	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:599)

	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:352)

	at org.openqa.selenium.remote.RemoteWebDriver.findElementByName(RemoteWebDriver.java:425)

	at org.openqa.selenium.By$ByName.findElement(By.java:299)

	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:344)

	at org.openqa.selenium.support.pagefactory.DefaultElementLocator.findElement(DefaultElementLocator.java:59)

	at org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler.invoke(LocatingElementHandler.java:34)

	at com.sun.proxy.$Proxy16.getTagName(Unknown Source)

	at org.openqa.selenium.support.ui.Select.<init>(Select.java:43)

	at cukesPages.FlightFinderPage.selectAirline(FlightFinderPage.java:67)

	at cukesTests.FlightSearchSteps.selectAirlines(FlightSearchSteps.java:74)

	at ✽.And select airlines "Unified Airlines"(FlightSearch.feature:9)

]]></failure>
</testcase>
</testsuite>

Maven Report

  • though we discussed about Maven earlier, we didn't discuss about Maven Reports. Maven generates one of the very fine report for your test run.
  • You just need to add following section in your POM under dependencies in project tag
  • <reporting>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.18</version>
          </plugin>
        </plugins>
      </reporting>
  • Once added, you need to run your maven project with goal surefire-report:report

Maven Report

  • When you right click on your project, under Run As menu, you may not find goal "surefire-report:report", in that case, click on "Run Configurations"
  • Select New configuration under "Maven Build"
  • Provide name for your configuration, select Base directory by Browsing workspace, and provide goal "surefire-report:report" in Goals field and run.
  • It will create Report under site folder. for more details on usage you can refer this

Maven Report

  • Result will be something like following:

Step Definition - Doc Strings

  • By Any chance if you want to pass a long text to your function, you can specify it as Doc Strings as following:

Step Definition - Examples

  • Cucumber provides good way to substitutes input values on scenarios
  • Consider you have a scenario and have 10 different input values, you can provide all values as a list in feature file and cucumber will take care of all inputs and replace it in your scenario, while running scenario for each input value.
  • So here you need to do two things in your feature file:
  • 1. Provide input values as tabular format under "Example" tag
  • 2. Provide tokens enclosed with "<>" in scenario outline, so cucumber can replace it with corresponding input value

Step Definition - Examples Cont.


Your feature file will look something like this:

Feature: FlightSearch
Scenario Outline: search result for a oneway trip from NewYork to London on 25th in Business Class in Unified Airlines 
    Given you are logged-in to "http://newtours.demoaut.com/" with username "test1" and password "test1"
    When select OneWayTrip
    And select FromPort "<From>"
    And select ToPort "<To>"
    And select FromDay "25"
    And select BusinessClass
    And select airlines "<Airlines>"
    And click on search button  
    Then search result should be displayed

Examples:
    |  From      | To           |  Airlines                       |
    |  New York  | London       |  Unified Airlines               |
    |  New York  | London       |  Pangea Airlines                |

Step Definition - Examples Cont.

In above case, Scenario Outline will be executed twice and tokens <From>, <To> and <Airlines> will be replaced with values from Examples table.

Step Definition - Data Tables

Step Definition - Data Tables

If you have a feature with data table as following and no corresponding step defined

Scenario: Test a data table with one column
    Given single column value
      | column1 |
      |   12    | 
      |   8     | 
    When I perform sum of them 
    Then I should get value 20

You will get something like following in output by cucumber

@Given("^single column value$")
public void single_column_value(DataTable arg1) throws Throwable {
    // Write code here that turns the phrase above into concrete actions
    // For automatic transformation, change DataTable to one of
    // List<YourType>, List<List<E>>, List<Map<K,V>> or Map<K,V>.
    // E,K,V must be a scalar (String, Integer, Date, enum etc)
    throw new PendingException();
}

which tells, what kind of datatype you can use to handle DataTable passed by cucumber.

Step Definition - Data Tables - Examples

Feature:

Scenario: The sum of a list of numbers should be calculated
    Given a list of numbers
      | 17   |
      | 42   |
      | 4711 |
    When I summarize them
    Then should I get 4770

Step Definition:

@Given("^a list of numbers$")
    public void a_list_of_numbers(List<Integer> numbers) throws Throwable {
        ...
    }

Step Definition - Data Tables - Examples

Feature:

Scenario: A price list can be represented as price per item
    Given the price list for a coffee shop
      | coffee | 1 |
      | donut  | 2 |
    When I order 1 coffee and 1 donut
    Then should I pay 3

Step Definition:

@Given("^the price list for a coffee shop$")
    public void the_price_list_for_a_coffee_shop(Map priceList) throws Throwable {
        ...
    }

Step Definition - Data Tables - Examples

Feature:

Scenario: An international coffee shop must handle currencies
    Given the price list for an international coffee shop
      | product | currency | price |
      | coffee  | EUR      | 1     |
      | donut   | SEK      | 18    |
    When I buy 1 coffee and 1 donut
    Then should I pay 1 EUR and 18 SEK

Step Definition:

@Given("^the price list for an international coffee shop$")
    public void the_price_list_for_an_international_coffee_shop(List<Price> prices) throws Throwable {
       ...
    }

We need to define class Price to hold required data

public class Price {
    private String product;
    private Integer price;
    private String currency;

    public Price(String product, Integer price, String currency) {
        this.product = product;
        this.price = price;
        this.currency = currency;
    }

    public String getProduct() {
        return product;
    }

    public Integer getPrice() {
        return price;
    }

    public String getCurrency() {
        return currency;
    }
}

JBehave



Introduction

Jbehave is best described on Jbehave.org home page with following steps:

How to Use?

Here we will be using JBehave with Eclipse.

  • Download JBehave Web Distribution from here
  • Create a new project and define directory structure as per required package.
  • Make sure that you are not using default package, not sure if made any mistake but with default package JBehave wasn't able to identify storyPath.

  • Create libs folder manually under your project and copy all jars from jbehave-web-distribution-3.5.5-bin\jbehave-web-3.5.5\lib folder.
  • Add all jars to build path(from Project properties -> Java Build Path tab). For now don't worry about files shown in directory structure image.
  • JBehave includes selenium libs as well, so if you are using specific version, make sure that you replace selenium-java.jar with appropriate jar.

How to Use?

Here we will be using JBehave with Eclipse and Maven.

  • Create a Maven project from Eclipse(new -> Other -> Maven Project). Assuming that you have already installed m2e plug-in. If not revisit this.
  • Once you create maven project, create dir structure for desired package under main/java and test/java.
  • Add the following pom.xml, I have used pom with artifactId behave-maven-plugin with some changes.
  • Now you are ready to run your JBehave tests

Example

  • For JBehave we will use same example we used for page object model. So if you have not visited design pattern section, please visit it first.
  • Here we will be having same scenario of searching flight for a destination
  • So for that our story file will be something like this :FlightSearch.story
  • Search flights of different destinations
    
    Narrative: 
    In Order to verify that search result to different destinations are retrieved as expected
    here we will search flights for different destination 
    
    Scenario: search result for a oneway trip from NewYork to London on 25th in Business Class in Unified Airlines
    
    Given you are logged-in to http://newtours.demoaut.com/ with username test1 and password test1
    When select OneWayTrip
    And select FromPort New York
    And select ToPort London
    And select FromDay 25
    And select BusinessClass
    And select airlines Unified Airlines
    And click on search button
    Then search result should be displayed
    
    Scenario: search result for a oneway trip from NewYork to Paris on 25th in Business Class in Unified Airlines
    
    Given you are logged-in to http://newtours.demoaut.com/ with username test1 and password test1
    When select OneWayTrip
    And select FromPort New York
    And select ToPort Paris
    And select FromDay 25
    And select BusinessClass
    And select airlines Unified Airlines
    And click on search button
    Then search result should be displayed

Stories

  • In JBehave, We define a story and then write scenarios corresponding to that story.
  • for Each Story file you need to follow few rules.
  • Jbehave looks for few keywords in your Story and identifies scenarios and steps for a particular scenario.
  • Basic keywords you need to keep in mind are
    • Narrative: Brief story detail
    • Scenario: Followed by description of that scenario
    • Each Step in Scenario should start with a keyword, It can be any of the following:
    • Given: A precondition/state for your scenario
    • When: Activity performed
    • And: Same as When, Just used to make scenario reading more logical
    • Then: Expected Result
  • Note that input data is also part of your scenario, which you need to use during execution of that scenario.

Mapping between Stories and Actual Automated Steps

  • Now instead of writing scenario in a simple JUnit Test as we were doing earlier
  • WebDriver driver = new ChromeDriver();	    
    driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    driver.get(baseUrl);
    
    LoginPage loginPage=PageFactory.initElements(driver, LoginPage.class);
    loginPage.setUserName("test")
    		 .setPassword("test")
    		 .clickLoginButton();
    FlightFinderPage flightFinderPage=PageFactory.initElements(driver, FlightFinderPage.class);
     
    flightFinderPage.selectOneWayTrip()
    				.setFromPort("New York")
    				.setToPort("London")
    				.selectFromDay("25")
    				.setBusinessClass()
    				.selectAirline("Unified Airlines");
    				
    FlightFinderResultPage flightFinderResultPage = flightFinderPage.clickFindFlights();			    
    
    //here you can have some verification of retrieved result
    driver.close();

Mapping between Stories and Actual Automated Steps - Cont.

  • You Need to divide it into multiple steps, as per your Jbehave Steps.
  • To map between step in story file and step in Java code, you need to use few annotations like @Given, @When, @Then
  • Your java code after segregating test into multiple steps: FlightSearchSteps.java
  • package net.mylearnings.jbehaveTests;
    
    import java.io.File;
    import java.util.concurrent.TimeUnit;
    
    import org.jbehave.core.annotations.AfterStory;
    import org.jbehave.core.annotations.BeforeStory;
    import org.jbehave.core.annotations.Given;
    import org.jbehave.core.annotations.Named;
    import org.jbehave.core.annotations.Then;
    import org.jbehave.core.annotations.When;
    import org.jbehave.core.steps.Steps;
    
    import org.openqa.selenium.WebDriver;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.openqa.selenium.support.PageFactory;
    import net.mylearnings.jbehavePages.*;
    
    public class FlightSearchSteps extends Steps{
    	WebDriver driver;
    	LoginPage loginPage;
    	FlightFinderPage flightFinderPage;
    	FlightFinderResultPage flightFinderResultPage;
    	
    	@BeforeStory
    	public void initializeDriver()
    	{
    		File file = new File("C:\\Users\\kavan.sheth\\Desktop\\chromedriver.exe");        
    	    System.setProperty("webdriver.chrome.driver", file.getAbsolutePath());
    		driver = new ChromeDriver();	    
    	    driver.manage().timeouts().implicitlyWait(5, TimeUnit.SECONDS);
    	}
    	
    	@Given("you are logged-in to $url with username $user and password $password")
    	public void login(@Named("url") String url,@Named("user") String user, @Named("password") String pass  )
    	{
    	    driver.get(url);
    	    loginPage=PageFactory.initElements(driver, LoginPage.class);
    	    if (!loginPage.isLoggedin())
    	    {	
    		    loginPage.setUserName(user)
    		             .setPassword(pass)
    		             .clickLoginButton();
    	    }
    	    else
    	    {
    	    	loginPage.clickflightslink();
    	    }
    	}
    	@When("select OneWayTrip")
    	public void selectOneWayTrip()
    	{	
    		
    		flightFinderPage=PageFactory.initElements(driver, FlightFinderPage.class);
    		flightFinderPage.selectOneWayTrip();
    	}
    	
    	@When("select FromPort $fport")
    	public void selectFromPort(@Named("fport") String fport)
    	{
    		flightFinderPage.setFromPort(fport);
    	}
    	@When("select ToPort $tport")
    	public void selectToPort(@Named("tport") String tport)
    	{
    		flightFinderPage.setToPort(tport);
    	}
    	@When("select BusinessClass")
    	public void selectBusinessClass()
    	{
    		flightFinderPage.setBusinessClass();
    	}
    	@When("select FromDay $fday")
    	public void selectFromDay(@Named("fday") String fday)
    	{
    		flightFinderPage.selectFromDay(fday);
    	}
    	@When("select airlines $airlines")
    	public void selectAirlines(@Named("airlines") String airlines)
    	{
    		flightFinderPage.selectAirline(airlines);
    	}
    	@When("click on search button")
    	public void clickSearch()
    	{
    		 flightFinderPage.clickFindFlights();
    	}
    	@Then("search result should be displayed")
    	public void resultVerification()
    	{
    		  //here you can have some verification of retrieved result	
    	}
    	@AfterStory()
    	public void afterStory()
    	{	  
    	     driver.close();
    	}
    }

Mapping between Stories and Actual Automated Steps - Cont.

Few Points to be considered for in FlightSearchSteps.java

  • Methods for each step can be in any order in file.
  • @BeforeStory and @AfterStory is used for actions before and after each story(not scenario)
  • for each data input, we replace it using $<variable name>, which is used as parameters for step. For Example we are mapping
  • Given you are logged-in to http://newtours.demoaut.com/ with username test1 and password test1
    to
    @Given("you are logged-in to $url with username $user and password $password")
    public void login(@Named("url") String url,@Named("user") String user, @Named("password") String pass  )
    So during actual execution, 'http://newtours.demoaut.com/' will be stored in url variable, 'test1' will be stored in user variable and, 'test1' will be stored in password variable. and passed as parameter to login method. to ensure proper mapping between Jbehave variable(starting with $) and java parameter we are using @Named annotation.

Configuration

  • Now as part of basic configuration you need to tell Jbehave where to look for Stories and in which class it will find steps.
  • A very good tutorial available at https://blog.codecentric.de/en/2012/06/jbehave-configuration-tutorial/
  • Which is being fulfilled with following code in JbehaveTests.java
    package net.mylearnings.jbehaveTests;
    
    import java.util.Arrays;
    import java.util.List;
    import org.jbehave.core.embedder.Embedder;
    
    public class JbehaveTests {
    	private static Embedder embedder = new Embedder();
    	//List of path for stories
    	private static List storyPaths = Arrays
    			.asList("net\\mylearnings\\jbehaveStories\\FlightSearch.story");
     
    	public static void main(String[] args) {
    		//adding object having steps defined
    		embedder.candidateSteps().add(new FlightSearchSteps());
    		embedder.runStoriesAsPaths(storyPaths);	
    	}
    }
  • Now you just need to run JbehaveTests.java as Java Application, which will read your stories and execute scenarios.

Few Points

  • Currently we are using most basic configuration, we will be replacing embedder with JUnitStories.
  • in Eclipse though stories are executed as expected, I always get
  • Reports view generated with 0 stories (of which 0 pending) containing 0 scenarios (of which 0 pending)
    Probably it is because of basic configuration, we will see reporting with JUnitStories.

Using JUnit instead of Main

To use JUnit, We need to make following changes

  • Extend JUnitStories
  • Override storyPaths abstract method, which returns list of stories
  • get the embedded Embedder(this.configuredEmbedder()) and provide objects having steps(candidateSteps().add(new FlightSearchSteps()))
  • That's it. Now you are ready to run your stories with JUnit.
  • package net.mylearnings.jbehaveTests;
    
    import java.util.Arrays;
    import java.util.List;
    import org.jbehave.core.junit.JUnitStories;
    
    public class JbehaveTestsWithJUnitStories extends JUnitStories{
    	
    	@Override
    	protected List storyPaths() {
    		return Arrays.asList("net\\mylearnings\\jbehaveStories\\FlightSearch.story");
    	}
    			
    	public JbehaveTestsWithJUnitStories() {
    		super();
    		this.configuredEmbedder().candidateSteps().add(new FlightSearchSteps());
    	}
    }

Using JUnit instead of Main

  • When you run your stories with JUnit, you will be getting result as following:
  • I am sure this is not what you want, just one run for all tests.
  • So now directly feeding Steps to your embedder, we need to use StepFactory.

Using JUnit with StepFactory

To Use StepFactory you need to do following:

  • To use StepFactory, you do not need to extend Steps Class. So remove it.
  • Need to download jbehave-junit-runner from here and add to classpath.
  • Need to ensure that, you have JUnit 4.11 + and JBehave-core 3.9 +.
  • Need to change your JbehaveTests.java class as following:
  • package net.mylearnings.jbehaveTests;
    import java.util.Arrays;
    import java.util.List;
     
    import org.jbehave.core.junit.JUnitStories;
    import org.jbehave.core.steps.InjectableStepsFactory;
    import org.jbehave.core.steps.InstanceStepsFactory;
    import org.junit.runner.RunWith;
    
    import de.codecentric.jbehave.junit.monitoring.JUnitReportingRunner;
    
    @RunWith(JUnitReportingRunner.class)
    public class JbehaveTestsWithJUnitStoriesAndStepFactory extends JUnitStories{
    	
    	@Override
    	protected List storyPaths() {
    		return Arrays.asList("net\\mylearnings\\jbehaveStories\\FlightSearch.story");
    	}
    	
    	@Override
    	public InjectableStepsFactory stepsFactory() {
    		return new InstanceStepsFactory(configuration(), new FlightSearchStepsWithoutExtendingStep());
    	}
    	
    	public JbehaveTestsWithJUnitStoriesAndStepFactory() {
    		super();
    	}
    }

Using JUnit instead of Main

  • After these changes you will be getting following as Results: