How to Build a Reliable QA UI Automation Framework: Part 1
Many software development teams struggle to build a user interface (UI) automation testing platform that delivers both stability and reliability. You may even have heard some discouraging words about this from your teammates. But don’t worry, you can do it! In this blog series, I will explain how to build a basic UI automation framework combining Behavior-Driven Development (BDD) and Selenium with Python. I’ll explain why I’ve chosen these specific approaches and technologies. If you know the reasons, you’re more likely to succeed at building a framework that you can use to write and run test cases. I’ll also provide examples, to show you what it will look like. This first blog is just the beginning. In future posts, I’ll explore how you can continue building your automation framework to do even more tasks.
Using Python to Build a UI Automation Framework
Within this tutorial, we will use Python to build a basic UI automation framework. As a high-level programming language, Python lets you focus on the core functionality of the application by taking care of common programming tasks. It offers simple syntax rules that make it easier to keep the codebase readable and application maintainable. It’s easy to learn, yet robust enough for the most complex applications.
Why Use Behavior Drive Development?
Derived from Test Driven Development (TDD), Behavior Driven Development (BDD) is a software development approach common in Agile environments. In both TDD and BDD, the team writes tests before the code. However, BDD differs from TDD in important ways that make it a powerful development approach:
• BDD allows a QA tester or business analyst to create test cases in simple text language (English) that both non-technical and technical team members can understand.
• BDD focuses on user stories and the system’s behavior. How does the user do this? What is the current application workflow?
BDD’s approach comes with solid benefits. Using BDD, you can evolve your testing program from running unit tests to testing application behaviors. BDD enables better collaboration within teams, solving the communication challenges that can emerge in development teams. I like to think of BDD as solving the core problem laid out in the blind men and an elephant parable: how do all the people on a team communicate, even if they don’t have the same perspective?
To achieve the greatest benefit with BDD, make sure you:
1. Convert requirements into user stories that define concrete examples.
2. Develop a solid understanding of the given-when-then formula.
3. Write the specification of the behavior of a class rather than the unit test of a class.
What is Gherkin?
Gherkin is a domain-specific language (DSL) that helps you describe business behavior without requiring the details of implementation. You use Gherkin to write specifications in BDD.
Here’s a proposed template for writing BDD test cases for a user story:
Here’s an example test case:
Using Behave
Behave is a very good framework to implement BDD, using Python. Check the behave documentation here.
Using Selenium
Selenium WebDriver, a web automation framework (a collection of open source APIs), allows you to execute tests against different browsers and platforms.
Check the documentation here.
Creating a UI Automation Framework
Let’s start our example project by installing Python using a virtual environment and the packages to run it.
Why Use a Virtual Environment?
By default, Python can’t differentiate between library versions. Python virtual environments create a way to isolate Python projects so they can have their own unique dependencies.
Consider the following scenario of two projects: Project A and Project B, each of which has a dependency on the same library, Project C. If Project A and Project B require different versions of Project C, we have problems. Maybe Project A needs v1.0.0, while Project B requires the newer v2.0.0, for example. Virtual environments solve this challenge.
Check here for a detailed explanation of the role of virtual environments in our example project.
Complete the Base Install
1. Install Python 3.6 following this guide
2. Install Pip for Windows or Ubuntu. Check here for a detailed explanation.
a) Pip is a package manager. It allows you to install and manage Python packages.
b) You can check if it’s already installed as follows:
pip3 --version
Output:
pip 19.3.1 from /usr/local/lib/python3.6/site-packages/pip (python 3.6)
3. Install VirtualEnv
pip3 install virtualenv
4. Install VirtualEnvWrapper
a) VirtualEnvWrapper is a set of extensions for the virtualenv tool. The extensions include wrappers for creating and deleting virtual environments and otherwise managing your development workflow.
pip3 install virtualenvwrapper
5. Install the Selenium driver for each browser:
a) Install geckodriver for Firefox
b) Install chromedriver for chrome
Set Up the Python Virtual Environment
1. Create a Virtual Environment:
a) Create a directory that will be the development folder
b) Navigate to it so we will be in our project root path
mkdir test_automation cd test_automation
c) Run “mkvirtualenv test_automation” in your cmd
i. This creates a folder with python executable, pip, and setuptools so the project will have its own environment
ii. It also activates the virtual environment, which is indicated with the (test_automation) on the left side of the prompt
mkvirtualenv test_automation
Output:
✔ Using base prefix '/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6' New python executable in /Users/dsolis/.virtualenvs/test_automation/bin/python3.6 Also creating executable in /Users/dsolis/.virtualenvs/test_automation/bin/python Installing setuptools, pip, wheel... done. virtualenvwrapper.user_scripts creating /Users/dsolis/.virtualenvs/test_automation/bin/predeactivate virtualenvwrapper.user_scripts creating /Users/dsolis/.virtualenvs/test_automation/bin/postdeactivate virtualenvwrapper.user_scripts creating /Users/dsolis/.virtualenvs/test_automation/bin/preactivate virtualenvwrapper.user_scripts creating /Users/dsolis/.virtualenvs/test_automation/bin/postactivate virtualenvwrapper.user_scripts creating /Users/dsolis/.virtualenvs/test_automation/bin/get_env_details No project set in /Users/dsolis/.virtualenvs/test_automation/.project (test_automation) ~
d) Run “setvirtualenvproject’:
setvirtualenvproject
Output:
Setting project for test_automation to /Users/dsolis/Documents/test_automation
i. When we activate a virtual_env we will move into the project directory
ii. We see (your_project_name) displayed
iii. This shows we have this virtual_env activated (new packages are added to this project)
2. Run ‘deactivate’:
a) This deactivates our virtual_envs
deactivate
3. Run ‘workon’:
a) This lists the available environments and activates them
workon
b) Or:
workon test_automation
4. Create the requirement packages:
a) Save this text into a txt file
i. selenium==3.141.0
ii. behave==1.2.6
iii. PyHamcrest==1.9.0
b) Run ‘pip3 install -r requirments.tx’:
pip3 install -r requirments.txt
i. Elementium uses selenium-2.48.0, but you could work with latest selenium-3.141.0
c) Run ‘pip3 install selenium==3.141.0’:
pip3 install selenium==3.141.0
i. Or add it in the requirements.txt last line before running step c
Set Up Behave
Behave has a specific directory layout:
• Features directory contains scenarios made of steps
• All feature files are stored here
• A feature file has one or more scenarios
• Steps directory for step definitions
• Implementation for the steps used in the feature files
• environment.py handles the teardown setup
1. Create the following directories and files:
google_search.py:
from hamcrest import assert_that,equal_to
from selenium.webdriver.common.by import By
@step('I navigate to the google page')
def step_impl(context):
context.browser.get("https://google.com")
assert_that(context.browser.title, equal_to("Google"))
@step('I search for "{search_term}"')
def step_impl(context, search_term):
context.browser.xpath("//*/input[@role='combobox']").write(search_term)
context.browser.xpath("(//*/input[@name='btnK'])[2]").click()
@step('I see a search result "{search_result}"')
def step_impl(context, search_result):
element = context.browser.xpath("//*/h3[contains(.,'{}')]".format(search_result))
assert_that(element.text(), equal_to(search_result))
google_search.feature:
Feature: Search Page
Background: Opening webpage
Given I navigate to google page
Scenario: Search page contains 10 navigation result pages
When I search for "python behave"
And I see "Tutorial — behave 1.2.7.dev1 documentation - Read the Docs"
environment.py:
from selenium import webdriver
def before_all(context):
context.browser = webdriver.Chrome() #have set chromedriver in your PATH
context.browser.set_page_load_timeout(20)
context.browser.implicitly_wait(20)
context.browser.maximize_window()
def after_all(context):
context.browser.quit()
2. To run our test, navigate to the features directory and type: behave
a) This will run all the scenarios added in all the feature files. There are multiple ways to limit what we want to run, such as tags or writing the feature file name.
When an assert fails, the scenario will fail:
When something we built isn’t handled as expected, it will fail like this:
Wrapping Up and What’s Coming Next
Congratulations! We just created a UI automation framework you can use for QA testing. It has all the elements you need to write test cases and perform daily tests. And this is just the beginning.
In my next post, I will explain how to:
• Understand the differences between scripting-style automation and using the page object model
• Implement a fully working example applying POM
• Review automation patterns when creating a UI automation framework
• Avoid common mistakes in working with a UI automation framework
• Plan for scalability and maintainability
