Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge.
Publication/Presentation Date
7-8-2025
Abstract
Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.
Volume
16
Issue
1
First Page
6274
Last Page
6274
ISSN
2041-1723
Published In/Presented At
Zenk, M., Baid, U., Pati, S., Linardos, A., Edwards, B., Sheller, M., Foley, P., Aristizabal, A., Zimmerer, D., Gruzdev, A., Martin, J., Shinohara, R. T., Reinke, A., Isensee, F., Parampottupadam, S., Parekh, K., Floca, R., Kassem, H., Baheti, B., Thakur, S., … Bakas, S. (2025). Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge. Nature communications, 16(1), 6274. https://doi.org/10.1038/s41467-025-60466-1
Disciplines
Business Administration, Management, and Operations | Health and Medical Administration | Management Sciences and Quantitative Methods
PubMedID
40628696
Department(s)
Administration and Leadership
Document Type
Article