Mobile testing considerations

Challenges

Heavy environment and infrastructure setup.

Executing tests in parallel.

How to leverage common code as much as possible and drive the web/mobile workflow based on the configuration. Less code, less maintenance.

Challenges (2)

Device fragmentation should be limited, so it doesn't negatively affect our setup.

Screen size and OS fragmentation too, have to be carefully picked up.

Localisation when it comes to users being able to change the app language to their own.

Beyond page objects

BEWARE OF LIMITATIONS

Wrong by design, as the object knows how to acts upon itself.

Often leads to entire PageObject complex abstraction layer and taxonomy.

Selectors are hardcoded in your objects, so no configuration is possible.

BEWARE OF LIMITATIONS (2)

Automatically built Page Objects are hard to maintain and to use. Pretty much no grouping of elements into headers and footers, or identified widgets, there would just be a big list of stuff.

It might limit your design, e.g. you staring to ignore better abstractions.

Hinders flexibility when refactoring (both structure and function).

Other Patterns

Object Map

File that maintains a logical representation of each application object that is referenced in an automated script. The purpose of the Object Map is to allow for the separation of application object property data known as the Physical Description, from the automated script via a parameter known as the Logical Name.

Use one single point of entry to store objects of any type in memory and retrieve them using a unique identifier. Assure that only one instance of an object exists for each specific data-item.

Object Map(2)

Python example

test_data/objectmap.yaml
						
							login_username:
    locator: name
    selector: username
login_password:
    locator: name
    selector: password
login_button:
    locator: class
    selector: btn-main-green
...
                        

Python example

core/elements_loader.py
						
							from selenium.webdriver.common.by import By

# avoid ContextInjection at this layer, if using Cucumber 
def get_by_key(driver, element_name, object_map):
    # avoid complex if-else constructs
    find_by = {
      'xpath': driver.find_element_by_xpath,
      'css': driver.find_element_by_css_selector,
      ...
    }
    ui_element_locator = object_map[element_name]['locator']
    ui_element_selector = object_map[element_name]['selector']
    # optional webdriver wait for element

    return find_by[ui_element_locator](ui_element_selector)
						

Python example

dsl/issues.py
						
							from core import elements_loader

def add_new_issue(context, title):
   # other related domain specific flow and actions
    elements_loader.get_by_key(context.session.driver, 'name_input', 
                               context.object_map).send_keys(title)
    elements_loader.get_by_key(context.session.driver, 'save_button', 
                               context.object_map).click()
						

Python example

spec/edit_issue_tests.py
						
							from dsl import issues
from random import randrange

def editing_issue_name_should_save_changes(context):
   # test data could be fetched from elsewhere
   new_issue_title = 'My valid issue title'
   edited_issue_title = new_issue_title + str(randrange(10000))
   # setup
   issues.add_new_issue(context, issue_title)
   # actual test
   issues.edit_issue(context, edited_issue_title)
   issue.verify_issue_title(context, edited_issue_title)
						

Loadable Component

Aims to make writing PageObjects less painful, but there is a catch ...

it only works well in complex "deep" page hierarchy (journeys), like SPA.

SlowLoadableComponent extends the pattern, via adding explicit waits for loading.

Loadable Component (2)

Helps us with asserting that all dynamic content on our page has been fully loaded and that all elements required in our test script are present on the page.

The load() method contains the code that is executed to navigate to the page, while the isLoaded() method is used to evaluate whether we are on the correct page and whether page loading has finished successfully.

JAVA example

AddIssue class turned to LoadableComponent
						
							public class AddIssue extends LoadableComponent <AddIssue> {
  // rest of the class is the same
  @Override
  protected void load() {
    driver.get("https://yourissuetracker.com/issues/new");
  }

  @Override
  protected void isLoaded() throws Error {
    String url = driver.getCurrentUrl();
    assertTrue("Not on AddIssue page: " + url, url.endsWith("/new"));
  }
}
                        

Screenplay Pattern

An actor-centric model that provides a clear distinction between goals, tasks and actions.

Thus making it easier for teams to write layered tests more consistently.

Screenplay Pattern (2)

Uses good software engineering principles such as the

• Single Responsibility Principle,
• the Open-Closed Principle,
• favours composition over inheritance,
• employs thinking from Domain Driven Design
• supports effective use of abstraction layers.

JS example

spec/add_issues.js
						
							describe('Added issue', () => {
  it('should be saved', () => {
    ...
    const john = Actor.named('John');
    const issueTitle = 'My valid test issue'
    john.attemptsTo(
      Start.withAnEmptyIssueList(),
      AddIssue.named(issueTitle)
    )
    expect(john.toSee(IssueItems.First.Displayed)).equal([ issueTitle ]);  
  })
})
                        

Mission Pattern

"The Agent remembers. The Agent uses Tools. The Agent performs Missions."

Helps us modularise our code and create personas which describe business or functional interactions of software users. Users can also be clients of different layers of your software architecture.

Mission Pattern

JS example

spec/add_issues.js
						
							describe('Added issue', () => {
  it('state should be changed only by admins', () => {
    ...
    const agentJohn = newWebCustomerMission.performs(
                        registerAdminWith('John',adminPassword,dob));
    agentJohn.obtain(chromeDriver, databaseDriver);
    webDriverTool = agentJohn.usingThe(chromeDriver);
    createIssueMission.createIssue(webDriverTool, issueTitle)
	              .fillIn('issueStateInput', with(issueState))
                      .clickOn('saveIssueButton');
    const dbIssue = fetchIssueFromDB(issueTitle).accomplishAs(agentJohn);
    // secondary concepts are memory and screens
    agentJohn.keepsInMind('issue.state', bdIssue.State);
    verifyAs(agentJohn).that(
      issuesListPage.First.State, hasText(agent.recalls('issue.state')));
  })
})
                        

Good practices

Hermetic Tests

Each test is atomic in nature, so no dependency on other state or data, but its own.

We can execute the test in different order based on needs, or in parallel, having the same result every time.

Self-suficient tests give historically reproducible results, thus support our culprit finding (what changes broke the build).

Hermetic Tests (2)

Benefits:

• Clean start
• Resilience
• Modularity
• Random run order
• Parallel testing

Hermetic Tests (3)

Considerations:

• Upfront design
• Complexity increase, that needs proper Fixture strategies
• Resource usage increase

Fluent Invocations

Is helping the test developer to determine if he/she can use the object at hand or should switch to other one.

This pattern is important for scaling a test automation framework. For example, large amount of pages, elements and methods available for usage may create a confusion.

It allows you to DRY your code, since you avoid putting the object again and again before invoking its methods.

JS example

pretty standard way
						
							describe('Login to account', () => {
    it('should work with valid credentials', () => {
        ...
        homePage.typeUsername(user.email);
        homePage.typePassword(user.password);
        homePage.clickSignInButton();
	  
        assertThat(homePage.isSignedIn(user));  
    })
})

                        

JS example

chained invocations
						
							describe('Login to account', () => {
    it('should work with valid credentials', () => {
        ...
        homePage
          .typeUsername(user.email);
          .typePassword(user.password);
          .clickSignInButton();
	  
        assertThat(homePage.isSignedIn(user));  
    })
})

                        

Back Door Manipulation

We set up the test fixture or verify the outcome by going through a back door (such as direct database access.)

If we have access to the state of the SUT, the test can set up the pre-test state of the SUT by bypassing the normal API and interacting directly with whatever is holding that state.

Back Door Manipulation (2)

Back Door Fixture Setup helps us with Access to the state of the SUT from outside the UI. Need to be mindful that,

we are bypassing a lot of the "edit checks" (input validation) normally done by the UI.

Tests become much more closely coupled to the System.

Must leave at least one End-user journey to be covered completely using the GUI.

Back Door Manipulation (3)

Back Door Verification can help us assert that actual data is in the persistence layer.

Back Door Tear Down is especially beneficial if we can use bulk database commands to wipe clean whole DB tables.

JS example

spec/add_issues.js
						
							describe('Added issue', () => {
  it('state should be changed only by admins', () => {
    ...
    // we can leverage Hexagonal Architecture, thus work 
    // with either a UI or a database
    agentJohn.obtain(chromeDriver, apiDriver);
    agentJohn.performs(put('/register', 
                           requestWith(username,password,dob)), 
                           returnsStatusCode(201));
    verifyAs(agentJohn).that(get('/users/',
                                 requestWith(agentJohn.Id)),
                                 returnsStatusCode(200));
  })
})