How To Measure Your Software Maturity Performance

‍Overview

Measuring the maturity of software is an essential part of the testing process. Changes in the software need to be monitored, especially if they relate to test metrics and potential defects in software itself.

Software maintenance can be expensive and time consuming, so it is important that any defects are caught as early as possible. According to the IEEE, there are three dimensions to the changes in software, with the aim being to build a Software Maturity Index that can be applied prior to software delivery.

In this blog post, we will discuss the relationship between the test being performed (irrespective of the results) and the level of defect discovery during the period under observation.

How do we measure test coverage?

Test Metrics can be used to measure test coverage prior to software delivery. This snapshot then provides a measure of the software tested at any point during the required testing.

These metrics can be calculated as follows:

Functional Test Coverage = FE/FT

In this calculation, FE is the number of test requirements that are covered by test cases that were executed against the software, while FT is the total number of test requirements.

This calculation then forms the basis of the Software Maturity Metric.

For the purposes of this blog post, we will refer to this as the Software Maturity Index (SMI). This calculation can be used to determine the readiness for release of a software system. The SMI is useful in assessing not only the software readiness in terms of changes, additions or deletions to legacy software infrastructure, but also a historical index of the impact of those changes.

The metric is calculated as follows:

SMI = Mt – (Fa + Fc + Fd)/Mt

This calculation can be broken down into the elements below:

• SMI is the Software Maturity Index value
• Mt is the number of software functions/modules in the current release
• Fc is the number of functions/modules that contain changes from the previous release
• Fd is the number of functions/modules that are deleted from the previous release.
• Fa is the number of functions/modules that are added from the previous release.

Reliability Metrics

The graphic below gives us an idea of how the calculation can be presented in a simple format.

The execution of the tests, for example, can be compared against all software releases – regardless of the add, change or delete actions – and provide the timeline to defect discovery.

In fact, if you look at the first period of the software lifecycle, you can see that the number of tests being executed has a direct correlation to the defect discovery during the same period of time. In other words, the more advanced periods of software maturity show that the level of defect discovery diminishes, even though the testing efforts themselves have increased in scale and intensity.

Summary

Ultimately, it might be beneficial to get these metrics (or at least an estimation of these numbers) to get a sense of your software maturity index

If you are using ALM software, these details can be easily retrieved.

Example:

Mt can be the total number of requirements, fc – requirements that have changed, etc.