NMIs like the UK’s National Physical Laboratory (NPL) develop and use measurement and metrics to support traceable confidence in information supply chains. They are also one of the most important national bodies that many people have never heard of; they provide and maintain tools, frameworks, and methodologies developed through rigorous science and international agreements to build confidence in supply chains, technologies and markets. Metrology—or the science of measurement—provides a transparent, well-defined approach needed to validate AI systems and algorithms, ensuring they perform as intended and do not perpetuate bias or inaccuracies. This transparency helps build confidence and assurance in AI solutions across various stakeholders, and ensures AI is developed and deployed in a responsible, trustworthy manner.

Confidence in AI is crucial for its widespread adoption and ensuring trustworthiness. Ultimately, this supports realisation of the opportunities AI provides, including economic prosperity, improved public services and increased personal opportunities. This is one reason why NPL has set out A Life Cycle for Trustworthy and Safe Artificial Intelligence Systems. The US NMI, the National Institute of Standards and Technology (NIST), defines the following as characteristics of trustworthy AI systems:

• Valid and reliable
• Safe, secure and resilient
• Accountable and transparent
• Explainable and interpretable
• Privacy-enhanced
• Fair with harmful bias managed

NMIs such as NPL, NIST, and Germany’s Physikalisch-Technische Bundesanstalt (PTB), are national bodies which are part of an international system for ensuring the comparability of measurement standards and measurement capabilities. These bodies provide independent, government-backed capabilities to maintain their respective country’s national standards and provide traceability to the International System of Units (the SI) at stated levels of confidence—known as measurement uncertainty. However, in an era of digitalisation and multi-modal data, the remit of NMIs is not limited to the measurement process itself and complex digital systems are being developed which have their confidence underpinned by metrological robustness.

A major open question with AI systems and their component parts, such as data and algorithms, surrounds the issue of how to trust or have confidence in the results produced by these systems. A common theme which surrounds this issue is the lack of transparency of the decision-making process behind AI and an inability to demonstrate provenance across the data and processing lifecycle.

Since NMIs specialise in traceability of measurement information from a primary measurement standard to the point of use, this can be applied to the AI system lifecycle to allow users and developers to quantify the confidence in the data and algorithms present in AI.

The independent nature of NMIs means that they can work noncompetitively across different companies and industries without self-interest impacting output, a significant advantage in an ever-competitive landscape of AI system development. As arm’s length or government bodies, and a key part of National Quality Infrastructures, NMIs can help build public trust in AI system development and deployment by facilitating a central independent resource for AI system testing and evaluation. In addition, the international network of NMIs through international metrology groups or organisations such as EURAMET (The European Association of National Metrology Institutes) and BIPM (Bureau International des Poids et Mesures) can aid standardisation across borders. The development of global standards for AI is essential to ensure consistency of vocabulary and procedures across systems and application areas. Metrology plays a key role in establishing these standards, which can facilitate global collaboration and innovation while ensuring the reliability, safety and effectiveness of AI technologies.

Current Challenges which Metrology can address.

NMIs generally have a track record for improving data quality through metrological robustness and traceability, and AI and the machine learning (ML) models and algorithms that enable it to present several areas where this expertise is of paramount importance:

• Measurement uncertainty and its propagation: This is a growing area of interest in which NMIs are perfectly positioned to take a leadership role and develop examples and techniques to demonstrate the value of understanding the uncertainty associated with AI/ML system outputs. It is fundamental to the traceability and trustworthiness of AI/ML predictions and decisions that they are accompanied by reliable quantitative assessment of uncertainty.
• Data quality and integrity: AI systems depend on large volumes of data to learn patterns and make predictions. The quality, accuracy and precision of this data directly impact the performance and reliability of AI models. Applying metrological robustness to AI/ML algorithms can enhance the system’s overall reliability. Ensuring that all AI systems are trained on high-quality, unbiased data is challenging, but understanding the quality of the data used to train AI can help understand their limitations. NMIs can help address this by providing standards and techniques to assess and quantify data quality and integrity throughout the AI system lifecycle.
• Bias in AI systems: Bias in AI models can lead to inequitable or inaccurate outcomes. Metrological techniques can help identify and mitigate biases in data and measurement processes (e.g. calibration, instruments) through rigorous testing and validation. Development of quality-assured benchmark datasets from calibrated measurements can provide a valuable service for companies wishing to test their AI processes.
• Driving innovation: Collaboration between NMIs, tech companies, and academic institutions can drive innovation in AI and metrology, leading to the development of new measurement techniques and standards.

Different sectors and industries are developing AI capabilities at different rates but the most likely areas to be interested in the metrological robustness that NMIs provide are those which are more regulated due to their safety-critical nature and therefore under more stringent requirements for data and decision making. Example industries where quality and traceability are of paramount importance and necessary to achieve operation are healthcare, security, pharmaceutical manufacturing, aerospace, finance, and evolving areas such as autonomous vehicles, biologic (pharmaceuticals) and renewable energy. SMEs and manufacturers new to adopting AI and other digital technologies will also need techniques and guidance to trust autonomous decision making. Given the traditional role of NMIs in providing measurement assistance to SMEs, this is an excellent area for NMIs to aid the adoption and upskilling of AI techniques.

NMIs have a critical role in developing and maintaining methods and standards to ensure AI technologies are robust, reliable and trustworthy. NMIs should take a proactive leadership role in the integration of AI within metrology, focusing on developing tools, establishing standards, ensuring transparency, and fostering innovation. By doing so, NMIs can ensure that AI technologies are harnessed effectively and ethically, benefiting a wide range of sectors.

NPL will be publishing an extended version of this blog as a position paper in the near future and we will also be discussing the role of NMIs in AI standards at the AI Standards Hub Global Summit on 17 and 18 March 2025. Including presentations by NMIs from the UK (NPL), Germany (PTB) and the Republic of Korea (KRISS).