Testmunk joined Snapchat. For testing services, check out TestObject.

Articles


Cross Platform Mobile Testing with the Page Object Model

Posted by on September 1st, 2015

Hybrid vs. Native Apps

Many of our clients take a hybrid approach to app development. By hybrid we mean using technologies such as Phonegap, Cordova, Appcelerator Titanium, and Sencha as part of their technology stack. Hybrid apps can benefit from cross platform mobile testing.

Companies choose the hybrid approach for several reasons. A major factor is the ability to release on multiple platforms at the same time. The decision of whether or not to make a hybrid app comes down to the skill sets of the various team members developing the app. There are also some downsides to going non-native, such as the possibility of limited features, rendering and responsiveness issues, or crashes on certain devices.

Our customers have asked us about the best testing practices for identifying and addressing these challenges. Our response is cross platform mobile testing. Since hybrid apps are developed from a common code base, it makes perfect sense to have functional UI test cases that can be executed on multiple platforms.

What are cross platform tests?

“Cross platform” refers to the idea that the same tests (scenarios) can be executed on multiple platforms and/or device types. Imagine your test framework being able to test your hybrid app on both Android and iOS while remaining robust, lean, and easy to maintain. Below we will discuss cross platform mobile testing for not only Android, iOS, and Windows Mobile, but also different devices, like tablets and phones.

In this article we will show you the benefits of cross platform tests, when you should consider them, and what the best practices are regarding them. We will use the Calabash framework as an example, which in our experience is very good when it comes to mobile test automation.

cross_platform_mobile_testing_graphic_1

Benefits of cross platform app testing

The same scenarios can be executed on different platforms

Let’s stick to the Android and iOS example here. If you find that the following scenario describes the same functionality on both versions of your app, why not have it written only once and executed on both apps?

Scenario: As an existing user I should be able to login and see the Home screen
Given I am on Welcome screen
When I login
Then I should see Home screen

Test logic is uniform and can be reused

Let’s break down the When I login step from the above scenario:

When(/^I login step$/) do
	@app.welcome_screen.email.insert_text(‘testuser@testmunk.com’)
	@app.welcome_screen.password.insert_text(‘test’)
        @app.welcome_screen.login.touch
end

In all likelihood, the same test logic will apply for both your Android and iOS apps. Consider how much time we save by writing the scenario and step definitions once, instead of once for each OS.

In case a step differs between devices or platforms, you can always keep them separate. For example, the above scenario may apply for phones, but you may have a different UI on the tablet version, such as users needing to check the terms of service before pressing the login button:

When(/^I login step$/) do
	@app.welcome_screen.email.insert_text(‘testuser@testmunk.com’)
	@app.welcome_screen.password.insert_text(‘test’)

	@app.welcome_screen.terms_of_service.check

        @app.welcome_screen.login.touch
end

Less code to maintain

Less code not only means that you can script your tests faster, but also that you will save time on maintaining and updating them.

We can break up acceptance tests into 3 layers:

  1. Business language scenarios written in a declarative way (describing what you want)
  2. QA step definitions written in an imperative way (details how to do it and what to assert for)
  3. Screen objects, the interfaces used to interact with an app

Previously you may have had to write 1 or 2 tests individually, for every platform. Now, if your scenarios are the same for different platforms, you can reuse them. The only layer that will differ is #3, your screen objects. We will explain this in greater detail in a minute.

What apps can be automated this way?

If your app looks and behaves similarly on iOS and Android, or you don’t have different user experiences between tablets and phones, there is a great chance you can benefit from cross platform mobile testing.

It’s also great for hybrid apps, which are getting more and more popular, since these apps have a consistent look across different platforms.

On the other hand, if your apps are significantly different, it’s better to have dedicated tests covering their unique functionalities. In such situations you should consider having only parts of your app tested the cross platform way. The same applies when the apps have different release cycles; it may be the case that some features are introduced on only one platform at first.

Keep in mind the 3 layers we mentioned. It’s not always a good idea to make as many of them as uniform as possible. Sometimes it’s better to have different step definitions for the same scenarios across platforms.

WordPress example

Consider the WordPress apps. Despite them both being native apps, they have very many similar features, and most of these features could be defined by the same scenarios.

The scenarios below can be used as an example of a cross platform test. This test applies for both the Android and iOS versions of the app.

WordPress for iOS (left) and WordPress for Android (right)

cross_platform_mobile_testing_image_1
Scenario: “Freshly Pressed” should contain an article
When I am on the Freshly Pressed screen
Then I should see an article

WordPress for iOS (left) and WordPress for Android (right)

cross_platform_mobile_testing_image_2

A more imperative scenario about login in feature:

Scenario: As an existing user I can log in
Given I am on the Login screen
When I enter “lukas@testmunk.com” into “Username or email”
And I enter “test1234” into “Password”
And I press “Sign in” 
Then I should be logged in

How do we do it at testmunk?

At Testmunk we use the Calabash framework, together with the screen object approach, to deliver robust tests for our client’s apps.

Screen Objects

Screen Objects serve as the interface to an app. Have a look at the following test step, which enters “lukas@testmunk.com” into the email field on the screen:

@app.welcome_screen.email.insert_text(‘lukas@testmunk.com’)
@app

Is the app being tested.

@app.welcome_screen

Here we refer to the first screen of the app, which contains the login form; we called it “Welcome screen”.

	
@app.welcome_screen.email

Email is an input field in the Welcome screen.

@app.welcome_screen.email.insert_text

insert_text is a method used to fill in a field; n this case it fills the email field.

Imagine every screen of your app described this way. This is called the Screen Object or Page Object approach. Your test code is easy to use, organized, maintainable, and hidden from the business logic layer.

Ok, back to cross platform mobile testing. Having screen objects be cross platform is easy. All you really need to do is implement custom screen objects depending on the platforms you need to support, e.g. Android phone, Android tablet, iPhone, iPad, etc.

The details of how to touch a button or swipe a screen on Android and iOS are hidden behind the screen objects.

By having the same interface (method names) like “insert_text” for the input field (@app.welcome_screen.email.inser_text), the step definitions can look exactly the same:

When(/^I login step$/) do
	@app.welcome_screen.email.insert_text(‘lukas@testmunk.com’)
	@app.welcome_screen.password.insert_text(‘test’)

        @app.welcome_screen.login.touch
end

Execution

Cucumber (Calabash) provides a parameter called “profile” (-p) that can be used to load different page objects depending on the platform / device type you test on.

Examples:

calabash-android -p android-phone 
cucumber -p iPad 

That parameter will search for the corresponding configuration in the config/cucumber.yml file in the root of your features/ folder. Here is how that file could look:

android-phone: -r features/support -r features/step_definitions -r features/android/phone
iPad: -r features/support -r features/step_definitions -r features/ios/ipad

android/phone/ and ios/ipad/ contain the screen objects for each platform.

Cloud execution

When executing on our cloud of devices (encompassing multiple platforms and devices), you don’t have to worry about your config file. We will make sure that everything works smoothly and your app is ready for all markets.

Summary

Cross platform mobile testing is not for everyone or every app. However, cross platform mobile testing can be a powerful tool that saves time and effort, especially as hybrid apps become more prevalent. Many apps have similar user interfaces across operating systems, and these similarities can be harnessed to create tests which work on both Android and iOS (and even Windows Mobile). Cross platform mobile testing is something we embrace and which is becoming ever more important in modern testing. As devices and operating systems continue to fragment, it is imperative to have test scripts which are lean and yet strong, and embracing cross platform mobile testing is one way to help ensure tests remain easy to maintain and update.

What are your thoughts on cross platform mobile testing? Let us know on Twitter or leave a comment below.

About the author:
Lukas is a QA Engineer at Testmunk helping some of the biggest iOS and Android apps to establish large scale automated test suites. Before discovering his passion for automation, he worked several years as a mobile app developer. He is passionate about agile and lean methodologies and in his free time he loves to ride his motorbike.

Testmunk automates mobile app testing

LEARN MORE

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>