Cloud-Based Automated Testing: A Tutorial With Xamarin.UITest and App Center (Part 1)

While there are many advantages to testing mobile apps on real devices, it might not always be practical. For example, you might need to test many different devices and operating system versions that you don’t have available. Cloud-based testing solves this challenge, making it possible to set up your automated test, select as many devices and operating systems as you need, execute your test, and review the results. In this automated testing tutorial series, we’ll learn how to use Microsoft’s App Center, a cloud-based platform, together with Xamarin.UITest, a C# testing framework that is very popular because it works so well with both native and cross-platform apps. If you want the advantages of cloud testing and you are using Xamarin.UITest, App Center is a good option to consider.

This first blog of the series provides a brief description of App Center and Xamarin.UITest. We’ll also create the .ipa and .apk files you’ll need to run tests, which we’ll dive into in the next blog.

What makes App Center so cool?

App Center has a well-deserved reputation as a powerful platform for building, testing, releasing, and monitoring apps. App Center has many attractive capabilities, including:

• Build: Build Android and iOS apps in the cloud.

• Analytics: Get a better understanding of the behavior of app users, including information about popular device models for Android or iOS, monitoring in real-time, and more.

• Crashes: Get detailed crash reports, data about errors, a list of most affected devices, and more.

• Test: Build and run automated tests using an extensive list of hundreds of real devices and operating systems. See information about your test runs on beautiful dashboards and get detailed logs to understand test failures.

App Center supports other platforms, such as React Native, Android and iOS, and other mobile test automation frameworks such as Appium and Espresso, but for our automated testing tutorial, we’re focusing just on App Center with Xamarin.UITest.

What is Xamarin.UITest?

Xamarin.UITest is a test framework you can use to create automated UI acceptance tests for hybrid and native mobile applications. The code is written in C# and the tests run using NUnit. The mobile interactions are executed by Xamarin.UITest.IApp. Using IApp, you can automatically tap, enter text, swipe up/down, and more (click for a deeper dive on IApp). To get the app’s elements, we use Repl. This will tell you the app locators available for the page under test (class, id, label, text, automation Ids).

Before you start our tutorial

Before you get started with this automated testing tutorial:

• Make sure you’ve installed Visual Studio 2017 or later.

• Download the Android SDK and Java Development Kit and then call their installation directory using the system variables(ANDROID_HOME , JDK_HOME).

Reminder: Follow POP to build the best tests

One more note before we start. To avoid code smell in your testing framework, I always recommend taking advantage of best practices. For Xamarin.UITest, you can do this by following Page Object Pattern (POP)  to architect the best UITest code. This approach makes it much easier to create cross-platform tests for every app’s pages. Also, tests created in this way will scale much more easily as the app grows. 

Create the .ipk and .apk files

If your app is based on Xamarin Cross Platform, be aware that you need to prepare Xamarin.iOS to run it on App Center. You can follow the steps below to do this.

1. In Visual Studio, install a new Nuget package called “Xamarin.TestCloud.Agent” in your iOS project.

2. In the AppDelegate class, at your iOS project, add the following code line: 

#if ENABLE_TEST_CLOUD 
                // requires Xamarin Test Cloud Agent
       Xamarin.Calabash.Start(); 
  #endif

3. In the iOS properties, go to the Build section. Add “ENABLE_TEST_CLOUD; ” as conditional compilation symbols. 

4. Make sure the configuration and platform are properly set, as shown in the image below.

5. Create a .ipa file signed with Development Provisioning Profiles (see how to create a free provisional for details). Make sure it is created on Debug mode; if you do it on Release mode, it will not work because the ENABLE_TEST_CLOUD flag has been set for Debug mode.  

Note that Apple will reject your app if this one has Xamarin Test Cloud Agent (more about this).

6. Create an .apk file for Android devices. You can archive the apk using the Release mode.

Now you’re ready to run automated tests!

Fantastic! You’ve learned about App Center and Xamarin.UITest and successfully built the .ipa and .apk files with all the elements you need to execute tests on App Center. Stay tuned for the next post in this series, in which I’ll show you how to run automated tests on App Center, including:

• How to create a project on App Center

• How to create a test run for Android and iOS devices

• How to set up the local env to connect to App Center

• How to run the Xamarin.UITest on App Center