5.3 Calibrating the Rating Model

Calibration assigns a default probability estimate to each possible overall score. According to European Directive 2006/48/EC, the rating scale used by the credit institution to group debtors in classes with a reasonably small band of PD values. The directive allows the use of "direct estimates of risk parameters [which] may be seen as the outputs of grades on a continuous rating scale". (Directive 2006/48/EC, Annex VII, Part 4, 4)

There two distinct calibration procedures to be used, depending on the methodology used to build the scoring function.

• Logistic regression: already yields sample-dependent PD estimates, which need to be rescaled to each segment's average PD.
• Statistical and heuristic models (e.g. MDA): calibration assigns PD values to scores; rescaling may be necessary.

 Unless a full data survey is used to generate the data set, external data is necessary to calibrate the rating system. Specifically, for all segments it is necessary to know the a priori PD (the average default rate).

5.2 Developing the scoring function

The statistical analysis leading to the development of the scoring function occurs in two phases:

• Univariate analysis
• Multivariate analysis

Univariate analysis

Building a catalog of indicators

1. In the first step, the quantitative data items are combined to form indicator components which are meaningful in business and enable economic analysis. E.g. gross profit, EBITDA, EBIT, working capital...
   
2. Definition of relative indicators: constructional figures, relative figures, and index figures. E.g. gross margin, EBITDA margin, EBIT margin, acid test, current ratio...
   
3. Working hypothesis: Good > Bad or Bad > Good. The hypothesis has to be monotonic in order to apply either MDA or logistic regression. Non monotonic indicators need to be transformed in PDs to be used monotonically in the analysis.

Analyzing indicators for hypothesis violations
Two possible alternatives.

• Measure of discriminatory power: if positive, the hypothesis holds; if negative, the hypothesis must be rejected and the indicator cannot be used in the subsequent phases.
• Compare medians: reviewing of whether the indicator's median values differ significantly for the good and bad groups of cases and correspond to the working
  hypothesis.
  

5.1 Data Set Quality

The data collection process is central in establishing the quality of the data set and, hence, of the statistical rating model.

Data requirements and sources

The data collection process must include all data categories relevant to creditworthiness in the debtor segment to be studied (see 2. Best-Practice Data Requirements for Credit Assessment). It is necessary to

• specify the data to be collected more precisely on the basis of the defined data categories.
• quality assurance requirements for quantitative, qualitative and external data.

Quantitative data

Annual financial statements are usually standardized by commercial law. This makes them reliable indicators of a company's financial success. However, care must be taken when comparing data from companies abiding by different accounting standards (IAS/IFRS vs. domestic GAAP vs. foreign GAAP).

Other relevant quantitative data might not be available in standardized form (e.g. income and expense accounts, information from borrowers on assets/liabilities). Collecting such information meaningfully can be challenging.