Testing of Artificial Intelligence (AI) and Machine Learning (ML)

Specialized qa services and concepts

Artificial intelligence is is on everyone’s mouth and it’s spreading into various business sectors. The testing world, where QualityMinds feel at home, is rarely taken into account while discussing the impact of the AI but it is also changing and facing completely new challenges caused by machine learning.

Hero of Artifical Intelligence (AI) Testing Ardee

Testing will never be the same

The quality of AI systems is of crucial importance, especially when their failure causes far-reaching consequences, as in the case of autonomous driving. Since AI learns on the basis of data and there are no clearly prescribed algorithms developed by humans, testing requires entirely new approaches. Based on our project experience, but also generally applicable in the context of “AI testing”, we have identified a large number of questions for the conception of the test strategy and test design that should be asked in order to set up a needs-oriented test strategy. The complexity of this is particularly evident in our largest AI project, funded by the German Federal Government, which aims to develop a generally applicable validation methodology for AI in autonomous vehicles and – which is our task – to systematically consider critical exceptional situations, both within the road traffic as well as the AI system and sensor technology.

AI Expertise

Test strategy

The test strategy is the main building block to test in an effective and efficient way. While testing “traditional systems” (web, mobile, API, etc.) can be based on several existing test strategies, ML-based systems do not. For example, the focus must be placed much more on data, its structure, semantics, etc., as well as on data generation techniques. We advise product teams on how this type of test strategy can be designed. We do this in an agile way with methods such as OnePager or 10 minute test strategy as well as risk storming.

Corner cases

When testing ML-based systems, the systematic identification of potential corner cases, i.e. exceptional situations, is extremely relevant. We advise product teams on the identification, design and testing of corner cases and are part of the training and testing process as well as on the application level.

Test data quality

Testing ML-based systems is much more data-focused than “traditional” testing approaches. In particular, the emphasis on high quality test data and data-driven approaches requires a solid knowledge of both test data management and data science. We offer our customers a combination of both worlds by providing methods and tools to design and build a pipeline for test data in projects.

Hands-on testing

Any successful test strategy is a mix of carefully crafted test cases, test automation, and exploratory testing. The latter, in particular, requires a very good understanding of the domain and excellent testing skills. We offer our customers test experts for exploratory practical testing to identify the most important cornerstones for ML-based systems.

ML-based testing tools

The various techniques known from data science and machine learning can also be used to improve the testing process. The market for ML-based test tools is slowly developing. We provide our customers with an overview and tool evaluation based on their requirements and our knowledge of this market.

References

There are many publications and presentations on AI testing. Our recent insights into this fascinating topics can be found and downloaded in this section.