Testing of Artificial Intelligence (AI) and Machine Learning (ML)
Specialized qa services and concepts
Testing will never be the same
AI Expertise
Test strategy
The test strategy is the main building block to test in an effective and efficient way. While testing “traditional systems” (web, mobile, API, etc.) can be based on several existing test strategies, ML-based systems do not. For example, the focus must be placed much more on data, its structure, semantics, etc., as well as on data generation techniques. We advise product teams on how this type of test strategy can be designed. We do this in an agile way with methods such as OnePager or 10 minute test strategy as well as risk storming.
Corner cases
When testing ML-based systems, the systematic identification of potential corner cases, i.e. exceptional situations, is extremely relevant. We advise product teams on the identification, design and testing of corner cases and are part of the training and testing process as well as on the application level.
Test data quality
Testing ML-based systems is much more data-focused than “traditional” testing approaches. In particular, the emphasis on high quality test data and data-driven approaches requires a solid knowledge of both test data management and data science. We offer our customers a combination of both worlds by providing methods and tools to design and build a pipeline for test data in projects.
Hands-on testing
Any successful test strategy is a mix of carefully crafted test cases, test automation, and exploratory testing. The latter, in particular, requires a very good understanding of the domain and excellent testing skills. We offer our customers test experts for exploratory practical testing to identify the most important cornerstones for ML-based systems.
ML-based testing tools
The various techniques known from data science and machine learning can also be used to improve the testing process. The market for ML-based test tools is slowly developing. We provide our customers with an overview and tool evaluation based on their requirements and our knowledge of this market.
References
There are many publications and presentations on AI testing. Our recent insights into this fascinating topics can be found and downloaded in this section.
Research Survey "Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety
Quality assurance of AI – the dangers lurk above all behind the corners
Highly Automated Corner Cases Extraction: Using Gradient Boost Quantile Regression for AI Quality Assurance
WSAM: Visual Explanations from Style Augmentation as Adversarial Attacker and Their Influence in Image Classification
Context-Based Interpretable Spatio-Temporal Graph Convolutional Network for Human Motion Forecasting
QualityMinds AI Testing Experts
Our team combines knowledge of QA and deep AI / ML expertise in a field which is being researched at this very moment.
Tobias Varlemann
Lead R&D
Dr Namrata Gurung
Data Scientist
Bettina Stühle-Stein
Senior Test Expert
Dr Michael Mlynarski
CEO