MLCommons Association
MLCommons is the world’s defacto benchmarking organization. It is a non-profit industry consortium supported by 125+ members and affiliates, including startups, leading companies, and academics developing and maintaining benchmark standards in a precompetitive space. Our members and partners include many leading technology companies, including Cisco, Dell, Google, Hewlett Packard, Intel, Meta, Microsoft, NVIDIA, and Qualcomm, in addition to other leading commercial and research organizations from around the world.
Role within AIRIS
Within AIRIS, MLCommons serves as the lead partner for evaluation and benchmarking. It is responsible for designing the project's comprehensive evaluation framework and integrated benchmark suite, assessing accuracy, robustness, fairness, interpretability, and usability across all use cases. MLCommons will further operationalise this framework through four internal evaluation rounds and open benchmarking involving external researchers, with the objective of establishing AIRIS as the reference benchmark for mechanism-informed biomedical generative AI. In addition, MLCommons contributes its expertise and tools to dataset annotation and data access frameworks.