Software and systems engineering – Software testing – Part 11: Guidelines on the testing of AI-based systems
Last updated: 18 Jul 2024
Development Stage
Pre-draft
Draft
Published
Scope
What is ISO/IEC TR 29119-11 about?
ISO/IEC TR 29119-11 is an International Standard on software and systems engineering which is part 11 of a multi-series that provides an introduction to AI-based systems. These systems are typically complex (e.g., deep neural nets), are sometimes based on big data, can be poorly specified, and can be non-deterministic, which creates new challenges and opportunities for testing them.
ISO/IEC TR 29119-11 explains those characteristics which are specific to AI-based systems and explains the corresponding difficulties of specifying the acceptance criteria for such systems.
Who is ISO/IEC TR 29119-11?
ISO/IEC TR 29119-11 on testing of AI-based systems is relevant to:
- IT industry
- Computer and data scientists
- Machine learning engineers
- Software engineers
Why should you use ISO/IEC TR 29119-11?
The testing of traditional systems is well-understood, but AI-based systems, which are becoming more prevalent and critical to our daily lives, introduce new challenges. ISO/IEC TR 29119-11 has been created to introduce AI-based systems and provide guidelines on how they might be tested.
ISO/IEC TR 29119-11 presents the challenges of testing AI-based systems, the main challenge being the test oracle problem, whereby testers find it difficult to determine expected results for testing and therefore whether tests have passed or failed. It covers testing of these systems across the life cycle and gives guidelines on how AI-based systems in general can be tested using black-box approaches and introduce white-box testing specifically for neural networks. ISO/IEC TR 29119-11 describes options for the test environments and test scenarios used for testing AI-based systems.
Adopting the testing of AI-based systems in compliance with ISO/IEC TR 29119-11 can help overcome challenges related to systems specifications, test input, self-learning systems, and flexibility and adaptability, amongst others.
©ISO/IEC 2022. All rights reserved.
