Establish the need and construct the foundation.
In this digital age, the tolerance for software that doesn’t work lies somewhere between low and zero. There is a crowd of competing apps that allow consumers to just move on to the next app when your app isn’t performing.
In the past, companies would release software updates every 12 months. As a result, they developed large, complex, and comprehensive regression test suites to fully test the software prior to release. These regression tests were performed manually, and it was not unusual for a full test to take a week or more to complete. Eventually, companies began releasing updates every six months, then quarterly, then monthly. With the advent of Agile methodologies, it is not unusual for companies to release updates every two weeks now, and sometimes even daily.
Companies still need to fully regression test their software prior to release, but a one-week manual regression timeline is clearly out of the question. As a result, more companies are moving to automated testing, which can reduce a regression test suite from days to hours.
But how do you get started? Here are some basic insights and advice on how to lay the groundwork for introducing automated regression testing into your organization.
Understand the Goals
The first step is to clearly understand your company’s goals along with its limits. When it comes to applications, most companies want high speed to market, top notch quality, and fair cost. However, it can be difficult to balance achieving all of these goals without a bit of compromise.
First and foremost, you must keep the size and stability of the code base in mind. It is easier (and cheaper) to introduce automated testing into a new app than a large, complex app. Let’s say you are an established company with a large, complex application, and market conditions are forcing you to make more frequent releases with less time between releases. You have high quality because you have built up a massive suite of manual regression tests, but you no longer have enough time to execute them all.
To maintain both speed and capacity, most companies resort to “risk-based” testing where they execute a subset of their regression test suite to fit the timeline. This approach usually results in defects beginning to slip through to production, causing quality to suffer. In this case, automating the regression testing is a good starting point, but automating a large regression test suite will require a huge capital investment.
If you want to achieve all three of your company’s goals simultaneously, you have to build a business case.
Identify a Proof of Concept Candidate
The best way to begin building your business case is to conduct a Proof of Concept (POC) project to gather the supporting data you need. When selecting your POC candidate, you need to consider the size, complexity, and frequency of the test you aim to automate. You are looking for a regression test that is fairly large, moderately complex, and time-consuming to execute. Look for a visible test that represents a sizable portion of business transactions.
Select the Right Tools
Next, you need to select the right automation tools. There are several things to consider here such as:
The type of application (Web, Desktop, Mainframe)
Open source tools vs proprietary commercial tools
Other tools in your software development ecosystem
The technical proficiency of your team
Your company’s tolerance for licensed software tools
Your tools are the building blocks of your automated testing, so make sure your selections are in line with both the goals and the requirements of your company so they too work to build your business case.
Conduct the Proof of Concept Project Internally or Externally
Once you have selected the candidate test to automate and selected the tools, it’s time to conduct the POC project and automate the test. You must decide whether you are going to use internal resources or bring in an outside consultant to help.
Using internal resources is cheap as there is no additional labor expense, but you have to figure everything out as you go with no expert guidance. If you conduct the project alone, there’s a possibility you may not complete the POC before the company loses interest. Time is money, so you have to determine whether your company values speed or cost here. Using an outside consultant will cost more, but get done much faster. As you conduct the project, be sure to track the time and money spent as a valuable element to finalizing your business case.
Present the Business Case
Now that you have the building blocks, it’s time to put them all together. The goal is to estimate the time and/or financial investment needed to automate the remaining tests and the time and cost savings from the reduced execution time. Here’s how you should structure your findings:
Identify the chosen manual regression test suite to be automated.
Record the number of hours it takes to manually execute the test.
Determine the labor cost to manually execute the test.
Record the number of hours and resource costs to convert the manual test to an automated test. Include both internal and external resources.
Record the time required to execute the automated test.
Extrapolate the cost to convert the remaining manual tests.
Determine the total time and cost required to execute all tests manually.
Determine the total labor saving realized from eliminating the manual testing.
Establish the desired frequency for automated testing.
Compare the labor savings from eliminating the manual testing with the cost of converting the test to automated testing.
Determine how long to recover the investment required to convert to automated.
If you follow this guide and present the results of your POC accurately and in line with your company’s goals, you should be one step closer to uncovering the full benefits of automated testing. To find out how we can help you build and present your business case, contact an expert today at info@stonemillconsulting.com
