A Multimodal Artificial Intelligence Framework for Intelligent Mortgage Risk Assessment Using Underwriting and Loan-Performance Information
Abstract
Mortgage risk models often rely solely on underwriting variables, with unclear targets and evaluation protocols. This study proposes a multimodal artificial intelligence framework using separate encoders for financial and loan-performance features from the same mortgage dataset, adaptive fusion and SHAP explanations. Experiments use Fannie Mae Single-Family Loan Performance Data (2007Q1; 206,914 loans; 4,695 defaults, 2.27%). Default is defined as 90-day delinquency or credit-event exit within 12 months after loan age 12 months, excluding already delinquent loans and non-adverse exits. Evaluation includes stratified 70/15/15 splitting, train-only preprocessing, 10 repeated splits with overlap-corrected inference, loan-disjoint later-age testing and external validation on 377,697 loans from 2018Q1 (0.41% defaults), approximately eleven years later. Compared with Logistic Regression, Random Forest, XGBoost and MLP, the framework achieves ROC-AUC 0.840 ± 0.008 and PR-AUC 0.147 ± 0.011 (random baseline 0.023), with corrected 95% confidence intervals of 0.831–0.850 and 0.133–0.160, respectively. ROC-AUC reaches 0.791 on later-age testing and 0.824 externally, below Random Forest (0.839). Multimodal learning outperforms financial-only (0.797) and loan-performance-only (0.747) models. However, corrected comparisons show adaptive fusion is statistically comparable to concatenation, fixed-weight fusion, Logistic Regression, XGBoost and MLP, outperforming only Random Forest in ROC-AUC on the random split. Class balancing and prevalence shifts require probability recalibration. SHAP highlights FICO score, state, loan purpose and combined LTV. Ethical handling, transparency and human oversight remain essential.
Keywords
Funding
This research received no external funding.
References
- Chan, L., Hogaboam, L., Cao, R. (2022). Artificial Intelligence in Credit, Lending, and Mortgage. In: Applied Artificial Intelligence in Business. Applied Innovation and Technology Management. Springer, Cham. https://doi.org/10.1007/978-3-031-05740-3_13
- Garcia-Teruel, Rosa M., (2022). Integrating Artificial Intelligence into the Mortgage Credit Market (December 16, 2022). Available at SSRN: https://doi.org/10.2139/ssrn.4304716
- Bi, S., & Bao, W. (2024). Innovative application of artificial intelligence technology in bank credit risk management. arXiv preprint arXiv:2404.18183. https://doi.org/10.48550/arXiv.2404.18183
- Anderson, J. T., Freybote, J., Lucus, D., Seiler, M. J., & Simon, L. (2022). Using Artificial Intelligence to Identify Strategic Mortgage Default Attitudes. Journal of Real Estate Research, 44(3), 429–445. https://doi.org/10.1080/08965803.2021.2009621
- Zhou, M. (2022). Credit risk assessment modeling method based on fuzzy integral and SVM. Mobile Information Systems, 2022(1), 3950210. https://doi.org/10.1155/2022/3950210
- Rasel, I. H., Ibrahim, M., Pritty, A. A., Fahim, A. S. M., & Jahan, N. (2023). Beyond FICO: Enhancing mortgage default forecasting and inclusive lending via multimodal graph neural networks and urban mobility analytics. Frontiers in Computer Science and Artificial Intelligence, 2(2), 62-81. https://doi.org/10.32996/fcsai.2023.2.2.5
- Yohannes, K. T. F. (2022). Next-Generation Mortgage Platforms: Merging Explainable AI, Sign Language Intelligence, and Sustainable IT for Risk-Aware Software Solutions. International Journal of Computer Technology and Electronics Communication, 5(3), 5117-5121. https://doi.org/10.15680/vhjvc153
- Sriram, H. K. (2022). AI Neural Networks In Credit Risk Assessment: Redefining Consumer Credit Monitoring And Fraud Protection Through Generative AI Techniques. Migration Letters, 19(6), 1017-1032. https://migrationletters.com/index.php/ml/article/view/11619
- Hung, M. T., & Lo, H. C. (2024). Risk analysis of mortgage loan default for bank customers and AI machine learning. Journal of Applied Finance & Banking, 14(6), 33-50. https://doi.org/10.47260/jafb/1463
- Konovalova, N., Saksonova, S., Velinov, E. (2026). Improvement of Credit Risk Assessment Using AI. In: Kabashkin, I., Yatskiv, I., Prentkovskis, O. (eds) Reliability and Statistics in Transportation and Communication: Ecosystems for Smart Connectivity and Intelligent Mobility. RelStat 2025. Lecture Notes in Networks and Systems, vol 1822. Springer, Cham. https://doi.org/10.1007/978-3-032-19245-5_48
- Ramakrishnan, R., Rohella, P., Mimani, S., Jiwani, N., & Logeshwaran, J. (2024, March). Employing AI and ML in risk assessment for lending for assessing credit worthiness. In 2024 2nd International Conference on Disruptive Technologies (ICDT) (pp. 561-566). IEEE. https://doi.org/10.1109/ICDT61202.2024.10489313
- El Sayad, I., Chong, F. H. L., & Gosal, B. (2026). Risk assessment in mortgages: a comparative study of AI models. PeerJ Computer Science, 12, e3494. https://doi.org/10.7717/peerj-cs.3494
- Hasan, M. M. (2026). AI-Enabled Financial Information Systems for Credit Risk Forecasting to Support Small Business Growth. American Journal of Interdisciplinary Studies, 7(01), 270-314. https://doi.org/10.63125/fhzazv80
- Raji, A. A. H., Alabdoon, A. H. F., & Almagtome, A. (2024, April). AI in credit scoring and risk assessment: enhancing lending practices and financial inclusion. In 2024 International conference on knowledge engineering and communication systems (ICKECS) (Vol. 1, pp. 1-7). IEEE. https://doi.org/10.1109/ICKECS61492.2024.10616493
- Chitturi, N. (2025). AI-driven credit scoring and alternative data: Expanding financial inclusion and access to credit for underserved populations. IPHO-Journal of Advance Research in Social Science and Humanities, 3(09), 06-17. https://doi.org/10.5281/zenodo.17231820
- Zhang, X. (2025, February). The application of alternative data in credit scoring. In International Workshop on Navigating the Digital Business Frontier for Sustainable Financial Innovation (ICDEBA 2024) (pp. 789-794). Atlantis Press. https://doi.org/10.2991/978-94-6463-652-9_84
- Di Maggio, M., & Ratnadiwakara, D. (2026). Invisible primes: Fintech lending with alternative data. Management Science. https://doi.org/10.1287/mnsc.2024.07854
- Barykin, S., Shulga, A., Dinets, D., & Mikhaylov, A. (2026). Digital twin-enhanced credit risk prioritization in mortgage portfolios: a hybrid model approach. Quality & Quantity, 60(2), 7701-7723. https://doi.org/10.1007/s11135-026-02579-3
- Xu, T., Zhu, W., & Zhang, J. (2026, May). A Study on the Application of Alternative Data in Credit Assessment for the Unbanked Population. In Proceedings of the 2026 2nd International Conference on Sustainable Development and Digital Transformation (pp. 379-385) https://doi.org/10.2139/ssrn.6973241
- Wang, T., Liu, R., & Qi, G. (2022). Multi-classification assessment of bank personal credit risk based on multi-source information fusion. Expert systems with applications, 191, 116236. https://doi.org/10.1016/j.eswa.2021.116236
- Gao, T., & Lau, R. Y. (2024, August). Leveraging Deep Learning and Multimodal Signals from Social Media to Enhance Credit Risk Prediction. In 2024 IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) (pp. 1-6). IEEE. https://doi.org/10.1109/ICSPCC62635.2024.10770419
- Lu, S., Zhang, X., Su, Y., Liu, X., & Yu, L. (2025). Efficient multimodal learning for corporate credit risk prediction with an extended deep belief network. Annals of Operations Research, 361, 309–346. https://doi.org/10.1007/s10479-025-06612-w
- Fang, Y. (2025, September). Design of a Multimodal Credit Risk Forecasting System for Financial Text Analysis. In Proceedings of the 2nd International Symposium on Integrated Circuit Design and Integrated Systems (pp. 191-196). https://doi.org/10.1145/3772326.3774731
- Shen, C., Pan, W., & Shen, X. (2026). Credit risk prediction and heterogeneity analysis for SMEs based on large language models and multimodal data fusion. Complex & Intelligent Systems, 12, 82. https://doi.org/10.1007/s40747-025-02192-1
- Cao, Y., Chen, Z., Kumar, P., Pei, Q., Yu, Y., Li, H., ... & Ndiaye, P. M. (2025, November). RiskLabs: Predicting financial risk using large language model based on multimodal and multi-sources data. In 2025 IEEE International Conference on Data Mining Workshops (ICDMW) (pp. 901-909). IEEE. https://doi.org/10.48550/arXiv.2404.07452
- Hu, X., Xu, T., Chen, Y., & Wang, S. (2026). Predicting mortgage default with machine learning: AutoML, class imbalance, and leakage control. arXiv preprint arXiv:2602.00120. https://doi.org/10.48550/arXiv.2602.00120
- Fannie Mae. (2026). Single-Family Loan Performance Data [Data set]. Fannie Mae Capital Markets. https://capitalmarkets.fanniemae.com/credit-risk-transfer/single-family-credit-risk-transfer/fannie-mae-single-family-loan-performance-data (accessed October 2026).
- Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). https://doi.org/10.1145/2939672.2939785
- Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR 2019). https://doi.org/10.48550/arXiv.1711.05101
- Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432
- Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21, 6. https://doi.org/10.1186/s12864-019-6413-7
- Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. J. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans (Eds.), Advances in Large Margin Classifiers (pp. 61-74). MIT Press. https://home.cs.colorado.edu/~mozer/Teaching/syllabi/6622/papers/Platt1999.pdf
- Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (PMLR 70, pp. 1321-1330). https://doi.org/10.48550/arXiv.1706.04599
- Elkan, C. (2001). The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI 2001) (pp. 973-978). https://www.researchgate.net/publication/2365611_The_Foundations_of_Cost-Sensitive_Learning
- Nadeau, C., & Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52(3), 239-281. https://doi.org/10.1023/A:1024068626366
- Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6(2), 65-70.
- Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (pp. 4765-4774). https://doi.org/10.48550/arXiv.1705.07874
- Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021). Improving performance of deep learning models with axiomatic attribution priors and expected gradients. Nature Machine Intelligence, 3, 620-631. https://doi.org/10.1038/s42256-021-00343-w