RISK ANALYSIS IN SOFTWARE TESTING: HOW TO MANAGE AND MITIGATE RISK

Preparing for all sorts of potential risks in software development is a sign of a mature business, just like using sunscreen on a beach and carrying an umbrella on a rainy day are signs of mature adult behavior. 

The traditional risk analysis in software testing evolved into its most advanced form by now: the RTB. The Risk Based Testing approach provides a safety net for companies and helps reduce downtime, minimize costs and, most importantly, prevent mistakes from happening in the first place.

NASA lost a $125 million Climate Orbiter aircraft in space after one of the engineers failed to convert imperial units to metric units.

UK taxpayers picked up a bill of $1 billion when a newly designed child support system by Electronic Data Systems Inc. [EDC] had an incompatibility clash with the just updated software at the Department for Work and Pensions [DWP], affecting 2.6 million people. Totally avoidable with the risk-based testing approach.

There are hundreds of ways that things can go wrong when developing and testing your software solutions and there are only a dozen steps to be taken to minimize the chances of those errors occurring.

What Is Risk-Based Testing [RTB]?

Risk based testing is an proactive analytical testing approach wherein tests are planned, scheduled, and performed based on the potential scale of negative impact and probability of occurrence. 

This prioritizes testing for critical and/or urgently needed software modules to minimize damage. The higher the potential impact is embedded in a software component, the more skilled personnel, time, and resources will be allocated to test it.

Based on the principle that we don’t know what we don’t know, the best practice is not only to assign the most experienced QA engineer in a company to design an RTB strategy, but also to commission an audit from a software testing company to identify blind spots.

Types of Risks in Software Testing

According to ISTQB [International Software Testing Qualifications Board], there are two main types of testing risks: product and project risks. These are the main categories worth examining during risk identification. 

Project risk is intrinsic and is born from within the project. It has to do with possible organizational or technical dangers of the project itself. This includes scope risks, variability arising from a project belonging to an industry with volatile regulatory basis, budget cuts, or undermanning.

Example: A test environment for a blockchain FinTech product has not been set correctly, leading to many defects and errors going undetected.

Product risk is related to a number of potential dangers embedded in the product that concerns its reliability, security, platforms, performance, functionality, maintenance, integration, infrastructure, usability, and platforms.

Example: The software performs a function incorrectly, coming up with the wrong result or omitting the step in the process [for example, skipping a step that lets users add a number of products in the shopping cart].

For the sake of this piece, we will focus on how to plan, roll out, and maintain a risk-based testing approach for product hazards in the software testing cycle.

Why Do You Need to Use Risk Analysis in Software Testing?

There are a few reasons that companies may want to develop a risk-based testing approach while developing their software:

The Cost of the Mistake is Too High

While developing software with mission-critical functionality, like that of a healthcare product or a trading company platform, where an error or downtime may cost human lives or millions of dollars, RTB is the only way to approach the testing process.

The Deadlines are Tight or Just Got Tighter

Another case where QA risk management is a must occurs when you work under tight deadlines and need to produce the code of the same high quality under now more stringent deadlines.

In the Agile environment, it’s recommended to run a partial regression test for critical functions, as well as a full regression suite with test cases for all elements. This setup will ensure good coverage of the strategic components and a quicker process.

When it comes to a waterfall methodology, it makes sense to begin testing the strategic components during the development process. These regression tests are only recommended for detached modules that remain unchanged when new elements are added.

The Center of Excellence of Risk Management in Software Testing

The ultimate goal is to incorporate the RBT across the entire company. This is where creating a CoE based on the most advanced QA department is the best practice. 

If your company is tech-heavy and your business case is scaling fast, you may want to establish innovative procedures as part of your SDLC. This centralized cluster of software testing expertise can gather QA best practices, minimize quality assurance department costs, reduce technical debt, and drive innovation.

How to Build Risk Management and Mitigation Process in Software Testing

The Risk Based Testing approach implies that the QA team identifies, analyzes, monitors and addresses potential risks based on their cumulative weight of severity of impact and probability. DevPro provides a full cycle of QA testing services, from the consulting stage to implementation. We also assess the risks by software complexity, business criticality, frequency of use, and possible areas with a defect. These are steps and methods popular for strategic RTB planning and implementation.

Step #1. Detect Risks

Even if you feel your QA department is strong enough to handle this task, we highly recommend involving outside vendors at this point. While the implementation stage can be performed with an internal talent pool to cut costs, it’s important to get the widest range of detection expertise that you can afford. 

The detection phase is one of the major steps for risk analysis in QA testing.Techniques used during this phase include:

• Interviewing and brainstorming
• Working with RTB checklists
• Root cause analysis
• Expert panel discussions
• Delphi method
• Analyzing and reverse-engineering comparable projects
• External QA audit with RTB recommendations

As a result of this phase, your quality assurance team will have a list of potential detrimental outcomes arising from the exploitation of specific software elements and modules — a draft register of risks.

Step #2. Analyse Risks

During the risk analysis stage, a QA team will put the potential dangers on the list through several filters, which can help identify their probability of occuring and the significance of the outcome.

For this, you can build an impact matrix, the risk analysis tool that identifies a potential danger from the point of likelihood and severity of effect. Each of the parameters being accessed ranges from 1 to 5:

This stage should result in at least three buckets of risks: High risk elements, medium risk, and low [as colors in the impact matrix illustrate]. In most cases, integration processes and security modules will be your top priorities.

Another tool is a product risk checklist. This can help further structure the RBT approach, as modules are lined up side by side with potential threats gauged against each other. In the example below, the Feature E demands the most urgent attention while the Feature C can go without testing.

Test effectiveness is another thing to consider when prioritizing risks. Testers need to consider if it is possible to conduct a test and assess if a risk can be mitigated, or how difficult and resource-draining a test will be.

To get the best out of the risk analysis exercise, it’s helpful to consider how features will combine with each other and what the possible side effects of their usage could be.

Even though Dev.Pro implements the RBT approach by default on its QA teams, we don’t make meeting attendance mandatory and encourage people from all departments to attend the risk assessment meetings. This provides a multidimensional perspective and elevates team engagement from the very beginning. 

Step #3. Mitigate, Monitor and Control Risks

This stage has to do with taking steps to counteract each of the potential threats on the list.

The risk response strategy planning worksheet may contain the following fields: 

Risks that have a positive outcome need to be reinforced, accepted, enhanced, and shared.

Risks with negative outcomes carry adverse effects that are not desirable for a business and need to be counteracted: minimized, mitigated, transferred, or avoided altogether. If there is no other way, they can be accepted with further steps taken to adapt the rest of the application to the negative outcome.

At this stage, the QA department has an elaborate RBT plan with all tests that need to have scenarios designed, and test environments created and run.

Once major risks have been tested, faults have been discovered, and bugs fixed, it’s time to establish an automated or semi-automated testing environment that can detect risk-prone capabilities and counteract them.

A Risk Management plan, contingency procedures, and project communication diagram all help create a DevOps environment that has a proactive risk based testing approach.

Risk Management Process in Software Testing: Where to Start

This article aimed to explain why risk analysis is important in software testing.

Not only does risk analysis save resources due to early bug detection and a lower probability of major errors, but it also creates a corporate culture that encourages this mindset. The result: future projects and modules are designed and executed with a risk detection filter at the very early phase of project conception.

If your company is just starting to think about launching this best practice, we suggest you start with a kick-off meeting, in which developers, testers and architects try to approach the new project with an inquisitive mind and consider all potential risks.
Dev.Pro provides software testing services to a variety of industries. If you feel like your risk-based testing strategy could do with a bit of QA expertise, reach out to our sales team for a quote.