The innovation of autonomous laboratories in the fields of chemistry and materials science, integrating artificial intelligence (AI) and automation, holds the potential to transform research processes by expediting the exploration of novel molecules and materials. Milad Abolhasani emphasizes the necessity for standardized definitions and metrics to facilitate the comparison and enhancement of these technologies efficiently.
Enhancing Discovery in Chemistry and Materials Science Through Standardized Metrics for Autonomous Laboratories
Chemistry and materials science are witnessing a surge of interest in autonomous laboratories, leveraging AI and automated systems to streamline research and discovery processes. A set of standardized definitions and performance metrics is being proposed to enable researchers, non-specialists, and prospective users to gain a clearer insight into the functionalities and comparative performance of these cutting-edge technologies.
The advent of autonomous laboratories offers significant potential for expediting the discovery of new molecules, materials, and manufacturing techniques, with broad applications spanning from electronic devices to pharmaceuticals. While these technologies are relatively nascent, some have demonstrated the capability to reduce the timeline for identifying new materials from months or years to mere days.
Milad Abolhasani, the corresponding author of a paper outlining the new metrics and an associate professor of chemical and biomolecular engineering at North Carolina State University, acknowledges the current spotlight on autonomous labs. However, he underscores the existing uncertainties surrounding these technologies, particularly in the varied interpretations of terms like ‘autonomous’ across research teams. This diversity in reporting styles poses challenges in comparing and evaluating different technologies, hindering collaborative learning and progress within the field.
The proposal aims to address fundamental questions such as the distinctive strengths of each autonomous lab and how insights from one lab can enhance the performance of another. By introducing a set of shared definitions and performance metrics, the objective is to foster a culture of mutual learning and advancement in these transformative research acceleration platforms.
One of the key elements of the proposal is a precise definition of autonomous labs along with seven core performance metrics that researchers should incorporate in their publications related to autonomous laboratories:
- Degree of autonomy: the level of user guidance required by the system.
- Operational lifetime: the duration for which the system can function without user intervention.
- Throughput: the time taken by the system to conduct a single experiment.
- Experimental precision: the reproducibility of the system’s outcomes.
- Material usage: the total quantity of materials utilized by the system per experiment.
- Accessible parameter space: the system’s capacity to encompass all variables in each experiment.
- Optimization efficiency.
“Optimization efficiency stands out as a crucial metric, albeit a complex one that defies a concise definition,” Abolhasani explains. Researchers are encouraged to quantitatively evaluate their autonomous lab’s performance and experiment-selection algorithm by comparing it against a baseline, such as random sampling.
The standardization of reporting practices for autonomous labs is envisioned to ensure the generation of reliable, reproducible results that harness the potential of AI algorithms leveraging the extensive, high-quality datasets produced by these labs.
The research paper titled “Performance Metrics to Unleash the Power of Self-Driving Labs in Chemistry and Materials Science” is featured in the open-access journal Nature Communications.
Nature Communications, a peer-reviewed, open-access scientific journal under the Nature Portfolio, covers a wide spectrum of natural sciences, including physics, biology, chemistry, medicine, and earth sciences. Established in 2010, the journal operates editorial offices in London, Berlin, New York City, and Shanghai.
Reference:
“Performance metrics to unleash the power of self-driving labs in chemistry and materials science” by Amanda A. Volk and Milad Abolhasani, published on 14 February 2024 in Nature Communications.
DOI: 10.1038/s41467-024-45569-5
Amanda Volk, a recent Ph.D. graduate from NC State, is the first author of the paper.
This research received support from the Dreyfus Program for Machine Learning in the Chemical Sciences and Engineering (award number ML-21-064), the University of North Carolina Research Opportunities Initiative program, and the National Science Foundation through grants 1940959 and 2208406.
