A transformer-based model for default prediction in mid-cap corporate markets
From MaRDI portal
Publication:6167415
Abstract: In this paper, we study mid-cap companies, i.e. publicly traded companies with less than US $10 billion in market capitalisation. Using a large dataset of US mid-cap companies observed over 30 years, we look to predict the default probability term structure over the medium term and understand which data sources (i.e. fundamental, market or pricing data) contribute most to the default risk. Whereas existing methods typically require that data from different time periods are first aggregated and turned into cross-sectional features, we frame the problem as a multi-label time-series classification problem. We adapt transformer models, a state-of-the-art deep learning model emanating from the natural language processing domain, to the credit risk modelling setting. We also interpret the predictions of these models using attention heat maps. To optimise the model further, we present a custom loss function for multi-label classification and a novel multi-channel architecture with differential training that gives the model the ability to use all input data efficiently. Our results show the proposed deep learning architecture's superior performance, resulting in a 13% improvement in AUC (Area Under the receiver operating characteristic Curve) over traditional models. We also demonstrate how to produce an importance ranking for the different data sources and the temporal relationships using a Shapley approach specific to these models.
Cites work
- A new mixture model for the estimation of credit card exposure at default
- Artificial neural networks in bankruptcy prediction: General framework and cross-validation analysis
- Bankruptcy prediction using terminal failure processes
- Deep learning
- Forecasting financial failure using a Kohonen map: a comparative study to improve model stability over time
- scientific article; zbMATH DE number 3078997 (Why is no real title available?)
- Management estimation in banking
- Managerial Applications of Neural Networks: The Case of Bank Failure Predictions
- Support vector machines for default prediction of SMEs based on technology credit
- The elements of statistical learning. Data mining, inference, and prediction
- The value of text for small business default prediction: a deep learning approach
Cited in
(9)- Machine learning in bank merger prediction: a text-based approach
- Review-based recommendation under preference uncertainty: an asymmetric deep learning framework
- Mitigating adversarial attacks on transformer models in credit scoring
- Fifty years at the interface between financial modeling and operations research
- Enhancing forex market trend analysis with bidirectional long short-term memory and transformer attention network
- Profit-based uncertainty estimation with application to credit scoring
- Fighting sampling bias: a framework for training and evaluating credit scoring models
- Multi-label feature selection considering label importance-weighted relevance and label-dependency redundancy
- A hybrid multi-layered ensemble model based on heterogeneous information network for small and medium-sized enterprise default prediction
This page was built for publication: A transformer-based model for default prediction in mid-cap corporate markets
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6167415)
