The Astrix Blog

Expert news and insights for scientific & technology professionals.

The Life Science Industry Blog for R&D Professionals

Using Automated Tools to Regression Test LIMS

Posted on Lab Informatics. July 30, 2018

Laboratory Information Management Systems (LIMS) are an important part of any modern laboratory. It is vitally important that any LIMS in operation function properly and reliably to ensure both data integrity and regulatory compliance. As such, a thorough computer system validation (CSV) plan should be developed and executed as part of any LIMS implementation project to confirm that the system is functioning as expected.

Changes are an unavoidable part of every software development lifecycle, and this is especially true for systems deployed in a scientific laboratory. Once a LIMS has been implemented, patches, maintenance, bug fixes, enhancements, upgrades and customizations will inevitably follow to keep the system up to date and relevant for evolving workflows. As these changes are made, validation activities must be performed on the actual changed code itself to ensure it is installed and operating properly.

As an application is updated or changed, the emergence of new bugs and/or old faults is very common. In addition to performing CSV on the changed code, testing must be done to determine whether a change in one part of the LIMS has introduced or reactivated bugs in other parts of the software. Regression testing tests the aspects of LIMS functionality that were not directly changed to make sure that they are unaffected by the changes and still functioning as intended. In this blog, we will discuss the benefits and best practices of using automated tools to accomplish regression testing.

Automated Regression Testing for LIMS

Some LIMS development teams try to avoid regression testing, opting instead to just test essential functions once on the initial implementation to make sure they work. If these functions check out on this initial test, the assumption is made that those functions will still work in the future unless they’re directly modified again. The reality is, however, that complex informatics software like a LIMS requires frequent regression testing.

Every time changes are made to a LIMS (fixing a bug, customization, patches, etc.), regression testing must be performed to verify that the new code does not adversely affect the functionality of the old code. Even minor patches in a LIMS can have unexpected consequences and cause bugs in existing functionality which was working at an earlier point in time. Performance testing is also a part of regression testing to make sure that the system performance is not affected due to the changes made in the LIMS.

Regression testing typically involves re-running previously completed tests to check whether LIMS behavior has changed and/or whether previously fixed faults have reemerged. Regression testing should be done utilizing a risk-based approach by selecting the minimum set of tests needed to adequately test the old functionality after a change has been made.

As LIMS are changed fairly often, regression testing can become a frequent activity. Regression testing can be done manually using programming techniques, but this can be time consuming. For a complex LIMS project, for example, hundreds of tests may need to be repeated every time the system is updated. Manual testing can also be inconclusive. A manual approach is not always effective in finding certain classes of defects – like bugs that show up later in LIMS modules after they already passed a manual regression test.

Automated testing tools can help improve both the speed and accuracy of regression testing. Your regression test libraries can be built from the existing test cases that were developed to test old functionality. Once the automated tests have been created, they can be run rapidly and repeatedly. This can be a cost-effective method for regression testing of LIMS that have a long maintenance life. Automated tools can even be set up to re-run all regression tests at specified intervals and report any failures.

While automated testing tools can be expensive to implement, they are ultimately cost effective in the long term, as they cut down on manual testing time, find hard to detect bugs, and ensure your LIMS keeps operating properly over its lifetime. In addition, automated tools give developers the confidence to move forward with making changes to your LIMS when necessary. Without good automated tools for regression testing, developers can be hesitant to make changes to existing codebases due to the potential for unintended side effects on the overall system. This hesitancy can result in costly and time-consuming work down the road as the system reaches a point where necessary changes can no longer be postponed.

A variety of automated testing tools now specialize in regression testing and, with just a few clicks of a mouse, allow you to establish sets of testing parameters and check new iterations of code against previous software baselines. These tools will highlight inconsistencies in testing logs and detail the exact place a particular function broke and why.

Best Practices for Automated Regression Testing LIMS

Just like any piece of software, it is important to validate an automated testing tool for its intended use – testing your LIMS. So before using the tool, you will want to validate that the automated tests actually test your software correctly. There are a couple of ways to do this:

If the automated tool uses functions and logic, first prove that the functions and logic work correctly. Then you can create an automated testing suite using these validated functions and logic.
You can create a manual regression test and run it and compare the results to those produced by the automated test.

Another useful tip is to run automated test cases every day in the evening, as this allows developers to fix any issues identified in the next day’s build. This allows you to find and fix regression defects at early stages of the project, as opposed to waiting until the end to find the issues when the fix may be much more complicated.

Finally, as with other forms of automated testing, setting a regression testing tool on autopilot is not a good idea. Some conscious oversight will be necessary to ensure that the tool is functioning optimally and finding all the bugs in your LIMS. Having the exact same tests running repeatedly may allow developers to learn what development pathways allow them to pass a fixed battery of regression tests. This may end up forcing a narrow software development trajectory and allow bugs to linger in various areas of the system. Its wise to complement your automated regression testing with some manual tests that reach into other parts of the LIMS.

Important Features for Automated Regression Testing Tools

There are a number of recommended features in an automated regression testing tool. Some of these include:

Scriptless. There are a variety of scriptless regression tools available the allow you to avoid writing code for your regression testing. These tools allow users to just drag and drop simple test commands (click, click and wait, capture, etc.) which automatically create the test cases and only require a value to be entered.

Utilizes Continuous Integration with the Server. Your automated regression testing tool should have Continuous Integration tool plugins that automate the process of storing and executing test scripts on the server. This will dramatically speed up the testing process.

Easily Add and Update Regression Test Cases. Your automated regression testing tool should allow you to easily combine the various successful reusable test cases in your testing library for use whenever there are similar kinds of regression testing requirements.

Simple Regression Test Result Tracking and Reporting. Reporting and result tracking features are essential in your automated tool. Your automated tool should include the ability to track details such as the number of tests executed, test case status, priorities, etc. Screen capturing ability is also useful to easily log the bugs located. Finally, a detailed view of the test suite’s visual logs is also useful to help testers make any changes in the test cases if necessary.

Conclusion

Regression testing needs to be part of a comprehensive testing methodology that works to prevent unexpected bugs from damaging your LIMS and your company’s bottom line as changes are made to the system. While automated regression testing should form the core of your regression testing program, manual testing allows sufficient variety to ensure that no aspects of your LIMS goes unchecked. A good automated regression testing tool will serve to reduce the bug count in your releases, reduce the costs and time associated with regression testing, and improve the quality of your codebase through more frequent refactors.

Astrix Technology Group professionals have over 20 years of validation experience and regulatory compliance expertise. We understand that computer system validation and regression testing are not “one size fits all” processes, and we work to create testing processes that are based on applicable regulations and guidance, best practices for the domain, and the characteristics of the system being tested. If you would like to discuss your LIMS validation and/or regression testing strategy with an Astrix Informatics expert, please do not hesitate to contact us.