Driving patient health forward with automated software testing
Our innovative use of test automation with EXACT
EXACT is a market leading Dental Practice Management system. It’s a complex and mature product with hundreds of years of code and thousands of features.
In pursuit of driving up patient oral health outcomes and helping our practices run a great business, we need to deliver a high quality product ever faster. To ensure a new release is ready to ship, our teams conduct extensive ‘regression testing’.
Regression testing is defined as a type of software testing to confirm that a recent program or code change has not affected existing feature
At Henry Schein One, we have had a rhythm of shipping 6 releases each year.
Each 8-weekly EXACT release cycle comprised 3 x 2 week development ‘Sprints’, followed by 2 weeks of regression testing. During this period, the team stops feature development and focus completely on regression testing.
The development team fixes showstopper bugs – bugs that can’t be released with the software.
The problem with manual regression testing
Because EXACT is a big product, we can never afford to test the entire product. It would take far too long.
Instead, we focus our testing efforts on the areas that we know we changed – the risky parts.
There are however, downsides to this approach:
- Repetitive and tedious work. Regression testing is not enjoyable work for test analysts.
- It’s costly. 25% of our development team’s time is spent performing regression testing.
- Slow. The cost of a bug goes up exponentially the later it is discovered – could we find them sooner?
- Windows 10 updates. Microsoft increasingly rolls out updates that can break EXACT, but since the change is not done by us, we have no way to predict where it will happen.
- Slow release cycles. Shipping a new feature to a customer takes a minimum of 10 weeks.
We needed a way to make our work both more efficient and enjoyable.
Extending our Software Testing
Test automation mimics human behaviour. Essentially it clicks buttons, creates appointments and moves scheduled appointments, to list a few of its activities. While we already had a number of automated tests, they were not comprehensive, and were time consuming to run.
Our theory was that test automation could solve a number of our problems:
- Eliminate some of the repetition of manual testing
- Reduce costs
- Discover bugs earlier
- Protect against Windows 10 updates affecting parts of our product
Ambitiously, we thought that maybe, just maybe – we might be able to complete regression testing every day. We could do this by getting virtual machines to run thousands of tests against several versions of Windows and across all the countries we run EXACT.
Our development teams have built a framework and test scripts that simulate the EXACT installation and upgrade process. They execute hundreds of our core set of regression tests every few hours.
This has already helped reduce our human regression testing effort by 60%, and stabilise the product much faster. We aim to finish well before the end of 2019, and have already seen outstanding results:
- Faster shipping. We have now shipped a release of EXACT every two weeks for the last 6 months. Already, we are able to get important updates out to customers faster.
- Windows 10 updates. We are much more effectively detecting Windows 10 updates that cause breakages – often, our automated tests tell us within hours.
- Shift left on quality. Most importantly, it has driven a ‘shift-left’ of quality. Developers are now notified of problems they introduce much sooner, and are provided with feedback and learning that was not previously possible.
With our test automation we can now try small features, get feedback, refine them, and iterate much faster.
We no longer have discussions asking ‘Should we delay the release?’
Back when we shipped a version of EXACT every 8 weeks, it was a big deal if a feature was a few days late. We’d often have the unpleasant choice between delaying the entire release by a week, or delivering the feature 8 weeks late.
Now, we’re able to be unwavering with our quality standard and timetable. If a feature isn’t ready on time, it will ship in the next release – only 2 weeks later.
“When I first joined SOE, I didn’t think this was possible. EXACT is a massive and mature product – could we really achieve sufficient test automation to speed up our delivery and improve quality? Thanks to some hard work by a team of smart people, the answer is ‘yes’.”
Niels Grootscholten, Henry Schein One
We haven’t yet been able to ship every day, but our team at Henry Schein One are confident to get there by the end of the year. It’s exciting to see our teams innovate at this pace in pursuit of delivering more customer value than ever, with EXACT.