[article]
Titre : |
A new goodness-of-fit test for event forecasting and its application to credit defaults |
Type de document : |
texte imprimé |
Auteurs : |
Andreas Blöchlinger, Auteur ; Markus Leippold, Auteur |
Année de publication : |
2011 |
Article en page(s) : |
pp. 487-505 |
Note générale : |
Management |
Langues : |
Anglais (eng) |
Mots-clés : |
Out-of-sample validation Probability calibration Hosmer–Lemeshow statistic Bernoulli mixture models Credit risk |
Index. décimale : |
658 Organisation des entreprises. Techniques du commerce |
Résumé : |
We develop a new goodness-of-fit test for validating the performance of probability forecasts. Our test statistic is particularly powerful under sparseness and dependence in the observed data. To build our test statistic, we start from a formal definition of calibrated forecasts, which we operationalize by introducing two components. The first component tests the level of the estimated probabilities; the second validates the shape, measuring the differentiation between high and low probability events. After constructing test statistics for both level and shape, we provide a global goodness-of-fit statistic, which is asymptotically {chi}2 distributed. In a simulation exercise, we find that our approach is correctly sized and more powerful than alternative statistics. In particular, our shape statistic is significantly more powerful than the Kolmogorov–Smirnov test. Under independence, our global test has significantly greater power than the popular Hosmer–Lemeshow's {chi}2 test. Moreover, even under dependence, our global test remains correctly sized and consistent. As a timely and important empirical application of our method, we study the validation of a forecasting model for credit default events. |
DEWEY : |
658 |
ISSN : |
0025-1909 |
En ligne : |
http://mansci.journal.informs.org/cgi/content/abstract/57/3/487 |
in Management science > Vol. 57 N° 3 (Mars 2011) . - pp. 487-505
[article] A new goodness-of-fit test for event forecasting and its application to credit defaults [texte imprimé] / Andreas Blöchlinger, Auteur ; Markus Leippold, Auteur . - 2011 . - pp. 487-505. Management Langues : Anglais ( eng) in Management science > Vol. 57 N° 3 (Mars 2011) . - pp. 487-505
Mots-clés : |
Out-of-sample validation Probability calibration Hosmer–Lemeshow statistic Bernoulli mixture models Credit risk |
Index. décimale : |
658 Organisation des entreprises. Techniques du commerce |
Résumé : |
We develop a new goodness-of-fit test for validating the performance of probability forecasts. Our test statistic is particularly powerful under sparseness and dependence in the observed data. To build our test statistic, we start from a formal definition of calibrated forecasts, which we operationalize by introducing two components. The first component tests the level of the estimated probabilities; the second validates the shape, measuring the differentiation between high and low probability events. After constructing test statistics for both level and shape, we provide a global goodness-of-fit statistic, which is asymptotically {chi}2 distributed. In a simulation exercise, we find that our approach is correctly sized and more powerful than alternative statistics. In particular, our shape statistic is significantly more powerful than the Kolmogorov–Smirnov test. Under independence, our global test has significantly greater power than the popular Hosmer–Lemeshow's {chi}2 test. Moreover, even under dependence, our global test remains correctly sized and consistent. As a timely and important empirical application of our method, we study the validation of a forecasting model for credit default events. |
DEWEY : |
658 |
ISSN : |
0025-1909 |
En ligne : |
http://mansci.journal.informs.org/cgi/content/abstract/57/3/487 |
|