Recommended Practice for The Evaluation of Artificial Intelligence (AI) Dialogue System Capabilities

Scope

This recommended practice establishes an evaluation framework for the capabilities of artificial intelligence dialogue systems such as chatbots, consulting terminals, or operation interfaces. The recommended practice defines and classifies the types and levels of the intelligence capabilities according to a checklist of criteria. The checklist tables describe the criteria used to determine the level that a dialogue system achieves based on the analysis of behavior and performance. ©IEEE 2022. All rights reserved.

Purpose

This recommended practice provides a framework for the evaluation of AI applications in dialogue systems, where both industry and academia can achieve consensus on professional terms and definitions, and share a common understanding on test methods as well. This recommended practice can also be used as the test guide for AI dialogue systems in smart manufacturing and smart city construction. ©IEEE 2022. All rights reserved.

External Links