Risk-Aware Human AI Decision Deferral: A Structured Public Source Review
DOI:
https://doi.org/10.46610/JoIDTA.2026.v03i03.005Keywords:
Expert allocation, Human–AI collaboration, Learning to defer, Structured review, UncertaintyAbstract
Human AI decision deferral allocates cases between an automated predictor and available experts according to expected error, consultation cost, and operational constraints. This structured public-source review uses a documented verification search conducted in September 2026; the historical search records were unavailable. A total of 43 bibliography records and 39 records surfaced by eight web queries yielded 59 unique records after 23 duplicate or companion citations were removed. Six nonreports were excluded; 53 report families were retrieved and assessed, with three background reviews and three out-of-scope reports excluded from the primary synthesis. A total of 43 primary research report families were included. Evidence includes theory, synthetic competence, trained model proxies, retrospective human annotations, and controlled human interaction. Design-specific appraisal used MMAT 2018 for two human-interaction reports and QUADAS-2 for one selected diagnostic experiment; remaining reports received descriptive applicability profiles. Five numerical examples were checked against source tables, with no meta-analysis or model reproduction. Findings support evaluating the joint routing system at declared operating points. Confidence in clinical and service benefits remains limited by bounded web retrieval, heterogeneous reused benchmarks, model-based expertise, and retrospective evaluation.
References
D. Madras, T. Pitassi and R. Zemel, “Predict responsibly: Improving fairness and accuracy by learning to defer,” Advances in Neural Information Processing Systems 31, 2018.
H. Mozannar and D. Sontag, “Consistent estimators for learning to defer to an expert,” Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 7076–7087.
R. Verma and E. Nalisnick, “Calibrated learning to defer with one-vs-all classifiers,” Proceedings of the 39th International Conference on Machine Learning, 2022, pp. 22184–22202.
G. Bansal, B. Nushi, E. Kamar, E. Horvitz, and D. S. Weld, “Is the most accurate AI the best teammate? Optimizing AI for teamwork,” The Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021, pp. 11405–11414.
B. Wilder, E. Horvitz, and E. Kamar, “Learning to complement humans”, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, 2020, pp. 1526–1533.
D. Leitão, P. Saleiro, M. A. T. Figueiredo and P. Bizarro, “Human-AI collaboration in decision-making: Beyond learning to defer,” arXiv, Jun. 2022.
Y. Montreuil, L. Yu, A. Carlier, L. X. Ng, and W. Ooi, “Adversarial robustness in one-stage learning-to-defer,” AISTATS, 2026.
J. Strong, E. Sun, H. Rogers, H. Higham and A. Noble, “Learning to defer: A survey,” Zenodo preprint, Dec. 2025.
J. V. Alves et al., “FiFAR: A fraud detection dataset for learning to defer,” arXiv, Dec. 2023.
J. V. Alves et al., “A benchmarking framework and dataset for learning to defer in human-AI decision-making,” Scientific Data, vol. 12, Apr. 2025.
J. V. Alves et al., “Cost-sensitive learning to defer to multiple experts with workload constraints,” Transactions on Machine Learning Research, 2024.
V. Pearson, J. V. Alves, J. Bono, M. A. T. Figueiredo, and P. Bizarro, “Uncertainty-aware systems for Human-AI collaboration,” Transactions on Machine Learning Research, 2026.
P. Hemmer, L. Thede, M. Vössing, J. Jakubik and N. Kühl, “Learning to defer with limited expert predictions,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 5, Jun. 2023.
P. Hemmer, S. Schellhammer, M. Vossing, J. Jakubik, and G. Satzger, “Forming effective Human-AI Teams: Building machine learning models that complement the capabilities of multiple experts,” Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 2022, pp. 2478–2484.
Q. N. Hong et al., Mixed Methods Appraisal Tool (MMAT), Version 2018: User Guide. Montreal, QC, Canada: McGill University, 2018.
G. Bansal, B. Nushi, E. Kamar, D. S. Weld, W. S. Lasecki, and E. Horvitz, “Updates in Human-AI teams: Understanding and addressing the performance/compatibility tradeoff,” The Thirty-Third AAAI Conference on Artificial Intelligence, 2019, pp. 2429–2437.
H. Mozannar, A. Satyanarayan, and D. Sontag, “Teaching humans when to defer to a classifier via exemplars,” The Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022, pp. 5323–5331.
P. F. Whiting et al., “QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies,” Annals of Internal Medicine, vol. 155, no. 8, pp. 529–536, Oct. 2011.
A. Goldberg, I. Ullah, T. G. Khuong, B. K. Rachmat, Z. Xu, I. Guyon, and N. B. Shah, “Usefulness of LLMs as an author checklist assistant for scientific papers: NeurIPS’24 experiment,” arXiv preprint arXiv:2411.03417, Nov. 5, 2024.
M. J. Page et al., “The PRISMA 2020 statement: an updated guideline for reporting systematic reviews,” BMJ, vol. 372, 2021.
S. Liu, Y. Cao, L. Feng, B. An, and L. Ong, “When more experts hurt: Underfitting in multi-expert learning to defer,” ICML, 2026.
Z. Wei, Y. Cao, and L. Feng “Exploiting Human-AI dependence for learning to defer,” Proceedings of the 41st International Conference on Machine Learning, 2024, pp. 52484–52499.
A. Mao, M. Mohri, and Y. Zhong, “Mastering multiple-expert routing: Realizable H-Consistency and strong guarantees for learning to defer,” Proceedings of the 42nd International Conference on Machine Learning, 2025, pp. 43035–43066.
K. Dvijotham et al., “Enhancing the reliability and accuracy of AI-enabled diagnosis via complementarity-driven deferral to clinicians,” Nature Medicine, vol. 29, pp. 1814–1820, Jul. 2023.
S. Joshi, S. Parbhoo, and F. Doshi-Velez, “Learning-to-defer for sequential medical decision-making under uncertainty,” Transactions on Machine Learning Research, 2023.
T. Lykouris and W. Weng, “Learning to defer in congested systems: The AI-Human interplay,” arXiv, Feb. 2024.
A. Ponomarev, “A simple heuristic for controlling human workload in learning to defer,” Pattern Recognition, 2024, pp. 120–130.
R. Verma, D. Barrejon and E. Nalisnick, “Learning to defer to multiple experts: Consistent surrogate losses, confidence calibration, and conformal ensembles,” Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, 2023, pp. 11415–11434.
Y. Cao, H. Mozannar, L. Feng, H. Wei, and B. An, “In defense of softmax parametrization for calibrated and consistent learning to defer,” 37th Conference on Neural Information Processing Systems, 2023.
H. Narasimhan, W. Jitkrittum, A. K. Menon, A. Rawat, and S. Kumar, “Post-hoc estimators for learning to defer to an expert,” Advances in Neural Information Processing Systems 35, 2022.
A. Mao, M. Mohri, and Y. Zhong, “Regression with multi-expert deferral,” Proceedings of the 41st International Conference on Machine Learning, 2024.
A. De, P. Koley, N. Ganguly, M. Gomez-Rodriguez, “Regression under human assistance,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 3, Apr. 2020.
A. De, N. Okati, A. Zarezade, and M. Gomez-Rodriguez, “Classification under human assistance,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 7, May 2021.
A. Mao, C. Mohri, M. Mohri, and Y. Zhong, “Two-stage learning to defer with multiple experts,” Advances in Neural Information Processing Systems 36, 2023.
Y. Montreuil, Y. S. Heng, A. Carlier, L. X. Ng, and W. T. Ooi, “A two-stage learning-to-defer approach for multi-task learning,” Proceedings of the 42nd International Conference on Machine Learning, 2025, pp. 44726–44749.
Y. Montreuil, A. Carlier, L. X. Ng, and W. T. Ooi, “Why ask one when you can ask k? Learning-to-defer to the top-k experts,” International Conference on Learning Representations, 2026.
S. Liu, Y. Cao, Q. Zhang, L. Feng, and B. An, “Mitigating underfitting in learning to defer with consistent losses,” Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, 2024, pp. 4816–4824.
A. Mao, M. Mohri, and Y. Zhong, “Realizable H-Consistent and Bayes-Consistent loss functions for learning to defer,” Advances in Neural Information Processing Systems, 2024.
D. H. Duy, Y. Montreuil, M. Meyer, A. Carlier, L. X. Ng, and W. T. Ooi, “Online learning-to-defer with varying experts,” arXiv, May 2026.
M.-A. Charusaie and S. Samadi, “A unifying post-processing framework for multi-objective learn-to-defer problems,” Advances in Neural Information Processing Systems 37, 2024.
V. Keswani, M. Lease, and K. Kenthapadi, “Towards unbiased and accurate deferral to multiple experts,” Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, Jul. 2021, pp. 154–165.
D. Tailor, A. Patra, R. Verma, P. Manggala and E. Nalisnick, “Learning to defer to a population: A meta-learning approach,” Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, pp. 3475–3483, 2024.
J. Strong, P. Saha, Y. Ibrahim, C. Ouyang, and J. A. Noble, “Identity-free deferral for unseen experts,” International Conference on Learning Representations, 2026.
M.-A. Charusaie, H. Mozannar, D. Sontag, and S. Samadi, “Sample efficient learning of predictors that complement humans,” Proceedings of the 39th International Conference on Machine Learning, 2022, pp. 2972–3005.
H. Mozannar, H. Lang, D. Wei, P. Sattigeri, S. Das, D. Sontag, “Who should predict? Exact algorithms for learning to defer to humans,” Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, 2023, pp. 10520–10545.
C. Nguyen, T.-T. Do, and G. Carneiro, “Probabilistic learning to defer: Handling missing expert annotations and controlling workload distribution,” International Conference on Learning Representations, 2025.
Z. Zhang, W. Ai, K. Wells, D. Rosewarne, T.-T. Do, and G. Carneiro, “Learning to complement and to defer to multiple users,” arXiv, Jul. 2024.
S. Wu, S. Kher, and P. Smyth, “Learning to assign prediction tasks to agents with capacity constraints,” arXiv, May 2026.
Z. Zhang, C. C. Nguyen, D. Rosewarne, K. Wells, and G. Carneiro, “Fatigue-aware learning to defer via constrained optimization,” arXiv, Apr. 2026.
Y. Montreuil, A. Carlier, L. X. Ng, and W. T. Ooi, “Adversarial robustness in two-stage learning-to-defer: Algorithms and guarantees,” arXiv, 2025, Feb. 2025.
M. Ghoummaid and U. Shalit, “When to act and when to ask: Policy learning with deferral under hidden confounding,” Advances in Neural Information Processing Systems 37, 2024.
F. Palomba, A. Pugnana, J. M. Alvarez, and S. Ruggieri, “A causal framework for evaluating deferring systems,” Proceedings of the 28th International Conference on Artificial Intelligence and Statistics, 2025, pp. 2143–2151.
Y. Li, L. Li, X. Wang, Z. Mao, J. Wu, and W. Bao, “A causal target for learning to defer under hidden confounding,” Proceedings of the AAAI conference on Artificial Intelligence, vol. 40, no. 28, pp. 23248–23255, Mar. 2026.
