Importance of automated regression testing in insurance industry

What is regression testing?

In a nutshell, regression testing ensures that previously validated processes continue to work as intended following updates or new implementations. By verifying that new features don’t interfere with existing functionality, these tests help to quickly identify and resolve any issues, enabling faster deployment.

Through many iterations of regression testing, we can produce a comprehensive analysis, check, compare, sum up and recommend what should be done to limit number of issues in the future.

Regression testing is a methodology encompassing various types of tests, such as UI, API, functional, performance, security, end-to-end, smoke, and integration tests. It’s particularly important when dealing with patches, code fixes, software version updates, configuration changes, new requirements, or bug fixes.

It all depends on what, why and how we want to test, and to do this, it is good to use test matrix, which is a visual representation (e.g. in Excel) that helps the user to organise and track the testing of different combinations of e.g. variables, features, or conditions.

Automated vs manual testing – two sides of the same coin?

Regression testing can be performed manually by the tester or automatically when building a new version of the software. Both have the same goal, but manual testing takes much more time and is not as easily reproducible or easy to iterative. You need to have a team of specialised testers, their cost, their time, and the hope that their focus will not drift away as they repeat the same task.

Another important difference between manual and automated testing is the amount of work required to perform them.

Manual tests need to be prepared, executed, and finally summarised by the tester, whereas automated tests only need to be written once, run in the CI/CD process, and a report with the metrics of the tests performed is generated automatically, without human intervention. An automated process is many times faster because it does not depend on the human factor, but only on hardware limitations.

It is important to remember that automated testing brings significant cost reduction and quality assurance to the project. By providing proper training in test automation to people that are less tech- savvy than testers or developers, there can be significant cost and time reductions (50% or more, depending on test cases, needs, environment and project).

All in all, manual tests can’t be completely excluded from the project lifecycle (e.g. ad hoc testing) but making sure that there is a strong backbone of automation in the process, can really make a difference to stakeholders.

Remember that in any system, it is unlikely that you will find all the bugs and all the problems. Although you need to make sure that core functionality works as intended, we cannot turn a blind eye to overall quality of the product, and do as much as we can within the estimated time and budget for the best user experience.

How do insurance companies invest in regression testing?

Any software developed today should be delivered in the right quality/time ratio, and with functionalities that cannot be found in competitor’s tools (or are better than theirs).

A crucial but often overlooked aspect (and one that can make or break customers) is ensuring that our system does not stop working or lose functionality during the next update (as a large company painfully tested on its customers during the 2024 security update).

This is particularly important in sectors such as banking and insurance, where system stability and continuity are paramount.

Being able to respond and adapt quickly to customer needs is critical for success, which means ensuring that, for example, polices can be created quickly (and at the right terms), adjusted, renewed, claims can be handled, and billing and payments can be made without unnecessary delays.

Solution providers such as Sollers have to take into account whether each layer of the system is working correctly, whether it is responsive, whether UI elements are quick to find and respond, whether tools are legible, intuitive, stable, and whether the operation of the system is consistent.

All this and more is required to ensure end-user satisfaction. Quality Assurance (QA) comes to answer many of these questions, and with QA comes regression testing.

How can regression testing be used in insurance?

Let’s take a look at this scenario: a user of an insurance system (that they know and have been using for several years) wants to file a claim for the car that has just been involved in an collision, hoping for quick response from the insurer and quick information on what to do next with this particular case, not to mention, a quick payout.

After filling in all the necessary details, they notice that they are unable to create a claim in the system, even though they have filled in the form correctly. After reporting the system problem (and becoming even more emotional about this chain of events, especially as they have been paying premiums to his insurer without delay for so many years) IT discovered that after recent changes to the system, there was a bug in the claim process that prevented users from completing it. It took a whole day to fix the bug and the changes were not implemented until the next day. Needless to say, the company could lose customers, profits, and everyone’s time for something that could have been caught much earlier.

Such a situation can be avoided by using automated regression tests. Manual tests are more time- consuming and offer less time and scope to check and some things may be missed. CI/CD methodology ensures that regression tests are consistently run after every code change, catching problems early and reducing the risk of bugs reaching production. Through continues feedback, unforeseen problems would be reflected in the report, allowing developers to fix the bug before it is deployed in production.

What challenges do insurers face for regression tests automation?

There are several major challenges in automated regression testing. One of these is integration with external systems/platforms, as the test results do not depend solely on the test tool provider (such as Sollers), and an interruption in the operation of such a system can distort the results.

Another problem is compatibility with old systems, which can be crucial when choosing an automated testing tool, as not all old software supports the latest solutions (e.g. new browsers), which can lead to compatibility conflicts.

Apart from the above, one of the biggest challenges in introducing automatic testing into a project is the lack of understanding of how automated testing works, and lack of more IT knowledge that is required to run such tests. If you want to save time and money by introducing e.g. business people to automated tools, there may be maintenance problems due to lack of understanding of how tests are structured, how code works, how to deal with bugs, or simply over-reliance on technical experts to come in and “save” users from failure.

This, together with the lack of appropriate tools on the market, may be enough to discourage stakeholders from changing their mindset between manual and automated testing, and missing the opportunity that the latter offers.

How to prepare for regression testing automation and develop an effective strategy?

When planning regression tests, you should consider how often you should run such tests. There is no definitive answer to this question, as it depends on the size of the test suite, the size of the project, and hardware limitations. You should therefore define the main scenarios in which such tests are useful, e.g.:

• Before releases: before implementing changes in production, we should run regression tests to ensure that there are no unwanted changes in operation after implementation.
• After integration with external systems: the operation of external systems can disrupt the operation of our software, so it is a good idea to run regression tests after such integration.
• After major code changes: major code changes can unexpectedly change the behaviour of some features, especially on large systems. Therefore, after major changes to our repository, we should make sure that everything works as it should.
• After bug fixes: bug fixes are caused by a scenario that the developer did not foresee. Therefore, after fixing a bug, you should check that you have not introduced another bug into the system.
• During the CI/CD process: this is the best time to run regression tests, because it doesn’t disrupt the team’s work, and the results can help identify problems on the fly.

Of course, the above are not the only scenarios in which we should run automated regression tests. Regression testing should be done on a “the more, the better” basis, because it allows us to be sure that what we build works the way we want it to.

How BATS can help you?

BATS (Business Automation Test Suite) is a Sollers tool which can give you an edge with competitive features and pricing in the rapidly changing insurance market. BATS is a collection of regression test cases with predefined structure, e.g. for the Guidewire platform. It streamlines testing and helps you prioritise, analyse and deliver faster than you might expect. With its new features, constant alignment with business needs, and easy-to-use syntax (based on Gherkin; natural language) you don’t need to devote too many resources to testing, and can still deliver on time, with good quality and with less budget.

When we compare BATS to other regression testing tools, the natural language in which BATS tests are written comes to the fore, thanks to which the person writing the test does not need to have any technical knowledge of automated testing.

BATS is an easy-to-use tool, and the entry threshold is low. A few keywords are enough to write a test, execute it and use many of BATS features. 

BATS as a tool is compatible with many systems that provide solutions for insurers, such as Guidewire (including GWCP), INSIS and Rife. There are over 300 test steps available to users, and over 150 pre-built tests designed for Guidewire.

Tomasz Ostrowski – Developer at Sollers Consulting

Tomasz is involved in developing tools used in the insurance sector, such as BATS, where he applies his programming and QA knowledge. He does not limit himself to backend technologies; he is also interested in front-end and DevOps technologies. He focuses on delivering high-quality code that supports clients using technologies like Guidewire and INSIS.

Artur Kisiel – Senior Analyst 

Artur is working on BATS and GoQu projects, supporting QA capabilities. With extensive experience in engineering, design, and analysis, he actively enjoys teamwork and takes responsibility for overseeing projects. Keen on developing skills in Business Intelligence, data analysis, and data visualization, ensuring a comprehensive approach to delivering high-quality solutions across multiple domains.