Hybrid approach to Automation

What does hybrid approach to test automation mean?

It's a strategic approach that aims to support better maintainability and reduce flakiness in end-to-end (E2E) test automation. This approach involves using a combination of API tests and UI tests.

In scenarios where certain prerequisites must be met before attempting the actual test, we can use endpoint calls/API tests to achieve these prerequisites. Then, we can use webdriver tests for UI-level functional validations/assertions.

By using this approach, we can redice the flakiness that may come from  webdriver tests due to delays, object changes, and other issues. This approach is particularly useful for sections of the test case that are outside the main test case.

Let me explain with an example:

In here we are doing a mandatry field validation check; Our featue file is as below;

Scenario: Mandatory fields validation
Given I am a registered user
And I go to the accounts page
And I leave the basic salary field blank
When I try to move to the next page
Then a validation error message is displayed for the basic salary field

My step definition will be like this; 

public class MandatoryFieldsValidationStepDefinitions {

    @Given("I am a registered user")
    public void iAmARegisteredUser() {
        // Code to implement the Creat user step using RestAssured API
        
    }

    @And("I go to the accounts page")
    public void iGoToTheAccountsPage() {
        // Code to implement the "And" step using Appium
        // Navigate to the accounts page using Appium
    }

    @And("I leave the basic salary field blank")
    public void iLeaveTheBasicSalaryFieldBlank() {
        // Code to implement the "And" step using Appium
        // Find the basic salary field element and clear its value
    }

    @When("I try to move to the next page")
    public void iTryToMoveToTheNextPage() {
        // Code to implement the "When" step using Appium
        // Find the "Next" button element and click it
    }

    @Then("a validation error message is displayed for the basic salary field")
    public void aValidationErrorMessageIsDisplayedForTheBasicSalaryField() {
        // Code to implement the "Then" step using Appium
        // Find the error message element for the basic salary field and verify it is displayed
    }
}

So how do we support this in our test framework? Since the CucumberRunner takes care of executing the features by mapping them with the correct step definition, all we have to do is ensure that our design is correct and that relevant API testing and webdriver libraries are included in the project.

A sample automation framework structure for a Maven project is shown below;

CT Server?

What is CT Server?

Recently, I have been reading a lot, or rather more than usual, about Continuous Testing. In fact, I have been trying to design such a process here at BlackArrow. However, today I came across some real details about the CT Server.

So, what is a CT Server? It refers to Continuous Testing server. The blog post that I read advised against using Jenkins for CT. Also, E2E testing is not something that Jenkins deals with in CI builds.

Instead, the author recommends using CT servers, such as BuildWise, which is an open-source software. I am yet to explore how it does things, but I'll be back with another post soon.

CucumberRunner sample

In this example file, I've added the following options while extending the AbstractTestNGCucumberTests class

import cucumber.api.CucumberOptions;
import cucumber.api.testng.AbstractTestNGCucumberTests;

@CucumberOptions(
        features = "src/test/resources/features",
        glue = {"step_definitions"},
        tags = "@JREQ-BA001-2088 or @smoke or @SE",
        plugin = {
                "pretty",
                "html:target/cucumber-html-report",
                "json:target/cucumber.json",
                "rerun:target/rerun.txt"
        },
        strict = true,
        monochrome = true,
        dryRun = false,
        snippets = cucumber.api.SnippetType.CAMELCASE
)
public class MyTestRunner extends AbstractTestNGCucumberTests {
}

• tags: Allows you to run only scenarios with specific tags. In this case, we're running scenarios with any of the three tags specified in the feature file. I have the story ticket, the test inclusion category and the feature tagged.
• plugin: This specifies the output formats for the Cucumber report. In addition to the "pretty" format, we're also generating an HTML report, a JSON report, and a rerun file (which lists any scenarios that failed on the first run and need to be rerun).
• strict: If set to true, Cucumber will fail if there are any undefined or pending steps. I prefer to have this set to false and user the power of the dryRun option.
• monochrome: This makes the output in the console easier to read by removing colour and special characters.
• dryRun: If set to true, Cucumber will check that all steps have corresponding step definitions, but won't actually run the scenarios. This can be useful for quickly checking that your step definitions match your feature file.
• snippets: This specifies the naming convention for generated step definitions. In this case, we're using CamelCase naming.

Chosen path for using feature files to call my test daa

 

This is a summary of the findings from my research past week.  

Out of these, I am choosing the method 3 considering the plus points that I could think of..

Using feature files for API test automation - 3

In here the entire user payload is read from the examples table and stored as a Map in the step definition. 

This approach is useful when you have a small number of fields in the request body, and you can easily read them from the feature file's data table.

Scenario Outline: Create User Successfully

    Given the user payload is:
      | first_name    | <first_name> |
      | last_name     | <last_name>  |
      | email         | <email>      |
      | mobile_phone  | <mobile_phone> |
      | date_of_birth | <date_of_birth> |
      | password      | <password>   |
      | username      | <username>   |

    When I send a <method> request to <path>

    Then the response status code should be <status_code>

    Examples:
    | first_name | last_name | email                 | mobile_phone | date_of_birth | password  | username | method | path           | status_code |
    | John       | Doe       | john.doe@example.com  | 1234567890  | 1990-01-01    | password | johndoe  | POST   | /users/create | 201         |

My stpe definition is as below;

public class CreateUserStepDefinitions {

    private RequestSpecification requestSpec;
    private Response response;
    private Map<String, String> userPayload = new HashMap<>();

    @Given("the user payload is:")
    public void the_user_payload_is(Map<String, String> userData) {
        this.userPayload = userData;
    }

    @When("I send a {string} request to {string}")
    public void i_send_a_request_to(String method, String path) {
        requestSpec = RestAssured.given()
                .contentType(ContentType.JSON)
                .body(userPayload);

        response = requestSpec.when()
                .request(method, path);
    }

    @Then("the response status code should be {int}")
    public void the_response_status_code_should_be(int expectedStatusCode) {
        assertEquals(expectedStatusCode, response.getStatusCode());
    }

}

Let me explain the java bits in here - 

In the method the_user_payload_is, the Map<String, String> userData parameter contains the data that was provided in the Examples table of the feature file. The data is in key-value pairs where the key represents the column header and the value represents the corresponding cell value.

this.userPayload variable is an instance variable of the class (which means it can be accessed from any method within the class). In the_user_payload_is method, we are assigning the userData map to the userPayload instance variable. This is done so that we can access the userPayload map in other methods within the class, specifically the i_send_a_request_to method.

In the i_send_a_request_to method, we are using the userPayload map as the body of the request. 

The RestAssured.given() method is used to start building the request. It is followed by the .contentType(ContentType.JSON) method, which specifies that the request body will be in JSON format. The .body(userPayload) method sets the request body as the userPayload object, which contains the data for creating a new user.

The requestSpec.when().request(method, path) method is then called to send the request to the server with the given method and path. This method returns a response object, which is stored in the response variable for later use in the_response_status_code_should_be method.

Two different flavours of feature files

Format 1 - Scenario outline with example table

Given I have a valid auth token
When I submit a POST request to create a user with the following details:
| first_name  | last_name    | email    | mobile_phone | date_of_birth | password    | username    |
| <first_name>| <last_name>  | <email>  | <mobile_phone>| <date_of_birth> | <password>  | <username>  |
Then the response status code should be <status_code>

Examples:
| first_name | last_name| email                 | mobile_phone | date_of_birth | password    | username    | status_code |
| John       | Doe      | johndoe@example.com   | 1234567890   | 1990-01-01    | password123 | johndoe123  | 201         |
| Jane       | Smith    | janesmith@example.com | 0987654321   | 1985-02-14    | password456 | janesmith12 | 201         |

This format gives a higher level of abstraction where the same scenario is run multiple times with different sets of data. Yes data including the expected result as well. 

This format is useful when you don't want to repeat the same scenario. Also when you have a simpler payload.

Format 2 - Single Scenario with Given\When\Then steps

Feature: Create User API

  Scenario: Create User Successfully
    Given the user payload is:
      | first_name    | John                 |
      | last_name     | Doe                  |
      | email         | john.doe@example.com |
      | mobile_phone  | 1234567890           |
      | date_of_birth | 1990-01-01           |
      | password      | password             |
      | username      | johndoe              |
    When I send a {method} request to {path}
    Then the response status code should be 201
    |method   | path         | status_code |
    | POST    |/users/create"|201          |

This gives more detailed description of the scenario. Each @Given, @When, @Then step is elaborated with relevant data.

In this case you may be having more specific scenarios and conditions.

Using feature files for API test autimation - 2

 Feature file

Similar to the last post, I'm usinng RestAssured for API test automation. Cucumber is used for mapping the feature scenarios. Also extending AbstractTestNGCucmberTests to run the test suite.

Difference is the way the payload is handled. Instead of reading the payload from exmaple tables, I am loading it from a json file.

This way it gives me more flexibility in managing test data/payloads.

I am still using exmaple tables to read other paramaters via <> tags in my featuer file .

Feature: Create User API

  Scenario Outline: Create User Successfully
    Given the user payload file is <data_file>
    When I send a <method> request to <path>
    Then the response status code should be <status_code>

    Examples:
    | data_file                             | method | path                  | status_code |
    |/test/java/resources/user_details.json | POST   | /user_accounts/create | 201         |

Payload file

{
  "first_name": "John",
  "last_name": "Doe",
  "email": "john.doe@example.com",
  "mobile_phone": "1234567890",
  "date_of_birth": "1990-01-01",
  "password": "p@ssword_1",
  "username": "johndoe"
}

CreateUserStep.java

public class CreateUserSteps{
    private RequestSpecification requestSpec;
    private Response response;
    private String requestBody;

    @Given("the user payload file is {string}")
    public void the_user_payload_file_is(String filePath) {
        requestBody = TestDataUtil.getJsonRequestString(filePath);
    }

    @When("I send a {string} request to {string}")
    public void i_send_a_request_to(String method, String path) {
        response = requestSpec.contentType(ContentType.JSON)
                .body(requestBody)
                .when()
                .request(method, path);
    }

    @Then("the response status code should be {int}")
    public void the_response_status_code_should_be(int statusCode) {
        response.then().statusCode(statusCode);
    }
}

Using feature files for API test autimation - 1

I am usinng RestAssured for API test automation. Cucumber is used for mapping the feature scenarios with the implmentation. In other words we have use of gherkin language in feature files and replication of same in stepdefinitions which has my restAuured/java test implementations.

Since I am a fan of TestNG's capabilities I am extending AbstractTestNGCucmberTests.

Feature file

Feature: Create User API

  Scenario: Create User Successfully
    Given the user payload is:
      | first_name    | John   |
      | last_name     | Doe    |
      | email         | john.doe@example.com |
      | mobile_phone  | 1234567890 |
      | date_of_birth | 1990-01-01 |
      | password      | password |
      | username      | johndoe |
    When I send a {method} request to {path}
    Then the response status code should be 201
    |method  | path         | status_code |
    | POST    |/user_account/create"|201         |

StepDefinitions

To handle the payload, I am  using .replace method, where values in an existing jsom template is replaced with the data in exmaple table.

The helper class TestDataUtil, uses its updateJsonRequestString method to get us the payload. This is what is does;

First it reads the content of the json file. Then it replaces all the placeholders in it with the corresponding values in the data table. Also saves it as a String. The modified String is then written back to the json file. This updated json file is assigned to requestBody variable.

public class CreateUserSteps {
        private RequestSpecification requestSpec;
        private Response response;
        private String requestBody;

    @Given("the user payload is:")
    public void the_user_payload_is(Map<String, String> dataTable) {
        String requestBody = TestDataUtil.updateJsonRequestString("create_user.json")
                .replace("{{first_name}}", dataTable.get("first_name"))
                .replace("{{last_name}}", dataTable.get("last_name"))
                .replace("{{email}}", dataTable.get("email"))
                .replace("{{mobile_phone}}", dataTable.get("mobile_phone"))
                .replace("{{date_of_birth}}", dataTable.get("date_of_birth"))
                .replace("{{password}}", dataTable.get("password"))
                .replace("{{username}}", dataTable.get("username"));
    }

    @When("I send a {string} request to {string}")
    public void i_send_a_request_to(String method, String path) {
        response = requestSpec.contentType(ContentType.JSON)
                .body(requestBody)
                .when()
                .request(method, path);
    }

    @Then("the response status code should be {int}")
    public void the_response_status_code_should_b(int expectedStatusCode) {
            response.then().statusCode(expectedStatusCode);
    }
    
}

Selenium - Configure TestNG to your Selenium framework.

To enable TestNG features in out test framework, as the first step we need to load the TestNG libraries to our test development environment(Eclipse).

• Go to mvnreposotory.com and type in "testng"in the search bar. 
• In the search results, select the latest available for dependency version. At the time of writing this blog I got below 7.6.1 version. 
• Copy maven syntax for the dependency, so that we can paste it to our pom.xml. 
• Once copied append it to the pom.xml of your maven project (which the framework was being built.
• Accordingly I added below to my pom.xml.

<!-- https://mvnrepository.com/artifact/org.testng/testng -->
<dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.6.1</version>
    <scope>test</scope>
</dependency>

Also we need to install the testNG plugin to the IDE. Go to Help >> Eclipse Market Place >> search for "TestNG"

Select to install the plugin and you are good.

TestNG - files structure

Structure of the TesNG.xml file consists elements as below.

<suite>
    <test name = "">
         <classes>
            <class name="" />
         <classes>
    </test>
    <test name = "">
        <classes>
            <class name="" />
        <classes>
     </test>
</suite>

• Each testNG xml pockets its content within the parent <suite> tags.
• test tags is the next main level and you can have many test tags. These specify the module.
• classes comes next. This helps goup the many classes you have under a certain test module
• class is the smallest element which carries the class name.

Example of a sample TestNG is as below;

 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite parallel="tests" name="ShoppingCart">

  <test thread-count ="5" name="Submit Order Tests">
    <classes>
      <class name="learningspace.tests.SubmitOrderTest"/>
    </classes>
  </test>
  
  <test thread-count ="5" name="Error Validation Tests">
    <classes>
      <class name="learningspace.tests.ErrorValidationTests"/>
    </classes>
  </test>
  
</suite>

Beginners spot - Pushing your work to GitHub

 

1. Login to your GitHub account and create the repo

2. Go to command prompt;

git config --global user.name "yumani"

git config --global user.email "yumani@gmail.com"

3. Navigate to the location of the local repo and use the below commands. 

It will initialize the repo, add the files and push to the origin with the comments.

git init

git add *

git status

git commit -m "Adding the initial file set"

git remote add origin https://github.com/yumaniranaweera/snuggery.git

git push origin master

4. You can import the files structure to your Eclipse editor from "File >> Import>>

Beginners Spot - Setup Maven in Test Automation environment

1)  Install Maven  (Windows)

  - Download from maven website and unzip the file.

  - Set MAVEN_HOME -  <base dire>\apache-maven-3.8.6

  - Add to path -  <base dire>\apache-maven-3.8.6\bin

2) Test the installation

 - Open cmd and type mvn --version

 - Should return maven home, version etc.

3) Create a maven project in Eclipse ( You can use maven-quickstart-archetype)

4) Integrate to Eclipse

 - Apply Maven Surefire plugin to Eclipse.

 -  Go to Maven Surefire plugin page >Usage (https://maven.apache.org/surefire/maven-surefire-plugin/usage.html)

 - Copy the plugin management snippet and paste in the pom.xml in your eclipse project.

1. <build>
2. <pluginManagement>
3. <plugins>
4. <plugin>
5. <groupId>org.apache.maven.plugins</groupId>
6. <artifactId>maven-surefire-plugin</artifactId>
7. <version>3.0.0-M7</version>
8. </plugin>
9. </plugins>
10. </pluginManagement>
11. </build>

Paste this above your dependencies section in the pom.xml.

 - now you can run mvn clean, mvn test commands.

5) Integrate TestNG

 - We need to add an additional configuration to the plugins section in the pom.xml

 - Go to https://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html

1. <configuration>
2. <suiteXmlFiles>
3. <suiteXmlFile>testng.xml</suiteXmlFile>
4. </suiteXmlFiles>
5. </configuration>

- Also you need to add the TestNG dependency to the pom.xml, if its not already added.