A Markov regenerative process with recurrence time and its application

Abstract

This study proposes a non-homogeneous continuous-time Markov regenerative process with recurrence times, in particular, forward and backward recurrence processes. We obtain the transient solution of the process in the form of a generalized Markov renewal equation. A distinguishing feature is that Markov and semi-Markov processes result as special cases of the proposed model. To model the credit rating dynamics to demonstrate its applicability, we apply the proposed stochastic process to Standard and Poor’s rating agency’s data. Further, statistical tests confirm that the proposed model captures the rating dynamics better than the existing models, and the inclusion of recurrence times significantly impacts the transition probabilities.

Introduction

One of the most significant risks faced by any financial institution is the credit risk, that is, the lender’s risk that the borrower may not meet its debt obligations. Consequently, an essential pillar of credit risk modeling is the accurate estimation of default probability over different time horizons. The default probability can be obtained using the market data of credit derivatives, for instance, credit default swap prices, or using the historically observed credit rating data.

Credit ratings are among the various parameters that quantify the credit risk associated with the borrower since they quantify a firm’s ability to pay back the loan and its likelihood of default. These ratings are either issued by international organizations such as Standard & Poor’s or obtained by financial institutes using the internal rating-based methods within Basel regulatory framework guidelines. In either case, one expects a higher interest rate from a firm whose rating is lower. Any firm’s rating level is dynamic and changes over time because of the random credit risk, either due to idiosyncratic or systematic changes. Therefore, the dynamics of ratings need to be modeled by an appropriate stochastic process. Jarrow et al. (1997) applied the Markov process to capture the time evolution of credit ratings. A drawback of their model is that it gives a zero-default probability in a small time interval to the bonds with high credit ratings leading to numerical issues. Kijima and Komoribayashi (1998) proposed new risk adjustments to overcome this drawback and demonstrated that their model is robust with respect to the recovery rate, especially for higher rated bonds. For more details, interested readers can refer to an excellent study by Duffie and Singleton (2003). Note that other than the various stochastic modeling approaches, many researchers are adopting machine learning techniques to model the credit risk (Kou et al. 2014, 2021; Wang et al. 2020; Shen et al. 2020), but we do not focus on this direction herein.

Several studies (Nickell et al. 2000; Kavvathas 2001; Lando and Skødeberg 2002; Altman and Kao 1992a, b) highlighted the non-Markovian behavior of transition probabilities of credit ratings. The main patterns include time dependence (Nickell et al. 2000), downward rating momentum (Carty and Fons 1994) and (Nickell et al. 2000), the duration effect (Carty and Fons 1994) and ageing effect (Fons and Kimball 1991; Jonsson and Fridson 1996). Empirical studies by Altman and Kao (1992a, 1992b) stressed the impact of length of time, that is, the duration effect, since issue on default risk. They found that default risk increases the first three-four years of rating issuance. Altman and Kishore (1996) observed similar results in that low-rated bonds are less likely to default in the first year. They argued that the plausible reasons for this lag in default are (a) loans are not granted to the companies who are in immediate danger of bankruptcy, and (b) the companies who raised money recently are likely to have monetary capital to repay the creditors. Over the longer time horizons, these effects become more pronounced in the sense that they have a larger impact on rating migration probabilities. It is therefore of significant importance to develop models that can capture these non-Markovian patterns.

Several authors have developed models in this direction to incorporate non-Markov behavior. For a detailed review on such models, interested readers can refer to D’Amico et al. (2019). Christensen et al. (2004) attempted to address non-Markovian behavior by extending the state space. More specifically, the authors distinguish the downgrades and the upgrades using an extended state space, that is, if a firm receives a lower rating, say “X,” it is given “X−.” In contrast, if a firm upgrades to a rating, say “Y,” it is given “Y+.” The states with a −ve sign have a higher probability of downgrade than those with a +ve sign. D’Amico (2011); D’Amico et al. (2012, 2010, 2011) proposed a semi-Markov credit rating model to address the non-suitability of Markov process. D’Amico (2011); D’Amico et al. (2010) applied semi-Markov processes with recurrence times to address the issue of the ageing effect. Recently, D’Amico et al. (2016) applied semi-Markov processes (SMP) with an extended state space to account for downward rating momentum. Pasricha et al. (2017) applied the semi-Markov model for portfolio optimization of credit risky bonds.

Pasricha et al. (2017) proposed a more general model based on Markov regenerative process (MRGP) to examine the downward rating momentum and duration effect. They argued that since rating momentum exists only in a downward direction, the Markov property is satisfied only when there is a migration from a given rating to a better rating (i.e., an upward movement). In addition, the time spent in a state is a general distribution (non-exponential distribution) in MRGP. Hence, MRGP models both the randomness of time in the transition between two states and their non-Markovian behavior. Through experiments on the data from S&P, the authors demonstrated that an MRGP based model is a better model for the credit rating dynamics than a semi-Markov and a Markov model. However, the two issues, that is, non-homogeneity and ageing effect, are not addressed in their model. It is of great interest to quantify these effects since they improve understanding of the rating process and the forecasts one may wish to associate with ratings.

Motivated by the importance of duration effect in rating migration, the superior performance of MRGP over SMP, and the limitation of Pasricha et al. (2017), we propose a generalization of the MRGP model (Pasricha et al. 2017) following D’Amico et al. (2010) to incorporate the duration effect in MRGP model. We develop a non-homogeneous MRGP model with recurrence time processes (i.e., the backward and the forward recurrence processes). We further derive a generalized Markov renewal equation governing the transition probabilities of the proposed process. As discussed in D’Amico et al. (2010), the underlying idea is that employing the recurrence time processes can explicitly model the impact of the duration inside a state on the transition probabilities. These processes provide complete information of the duration inside a state and more accurate estimates of transition probabilities and other performance measures.

The contribution of this article is twofold. First, we generalize the results for non-homogeneous SMP in D’Amico et al. (2010) to non-homogeneous MRGPs. Second, we apply the proposed generalized process to the dynamics of credit ratings to capture the duration effect. The numerical experiments reveal that the transition probabilities of an MRGP (Pasricha et al. 2017) change significantly on the inclusion of the recurrence time processes, and the resulting non-homogeneous MRGP model gives a better fit to the actual data. Thus, the proposed model provides a more accurate model to capture non-Markov patterns in credit ratings.

The remainder of this paper is organized as follows. “Non-homogeneous continuous-time Markov regenerative process” section introduces the time non-homogeneous MRGP followed by the introduction of recurrence time processes and derives the main results. This section also derives the global and local kernel for our application on credit rating dynamics. The estimation methodology and computational results on real data from the S&P rating agency are discussed in “Numerical illustration” section. Finally, concluding remarks are presented in “Conclusion” section.

Non-homogeneous continuous-time Markov regenerative process

This section presents the non-homogeneous MRGP model with recurrence times followed by an application to model the credit rating migration.

Consider a non-homogeneous MRGP¹$\{Z(t),~t\ge 0\}$ with state space $\Omega$ and the corresponding non-homogeneous continuous-time Markov renewal sequence $\{(X_n,T_n),~n=0,1,\ldots \}$. The transition probabilities for {$Z(t),t\ge 0$} defined by

$$\begin{aligned} V_{ij}(s,t)=P\{Z(t)=j\mid Z(s)=i\},i\in \Omega ^{'} ,j\in \Omega ,~s\le t, \end{aligned}$$

can be obtained by solving the following generalized Markov renewal equation defined by the kernels

$$\begin{aligned} V(s,t)=E(s,t)+K(s,t)*V(s,t), \end{aligned}$$

or

$$\begin{aligned} V_{ij}(s,t)=E_{ij}(s,t)+\sum _{\gamma \in \Omega '} \int _s^t V_{\gamma j}(y,t)dK_{i\gamma }(s,y),~i\in \Omega ^{'} ,j\in \Omega ,~s\le t. \end{aligned}$$

(1)

The first part², $E_{i,j}(s,t)$ of the above equation represents the transition from state i at time s to state j at time t without regeneration. The second part represents the probability of transition from state i to some state $\gamma$ at time y by regeneration and then from state $\gamma$ to state j in time $(t-y)$ following some trajectory. While Eq. (1) can be solved numerically by discretization, obtaining an explicit expression for kernels is a challenging task. Furthermore, the form of the kernels is application-dependent.

From Equation (1), we observe that the time s is a regeneration time, and the system enters the state i at time s itself. However, this may not be the case in practice. It is possible that the system entered state i at a time that is before time s but stayed in the state i until time s. Similar behavior can happen at the final time t. It is well known that this information plays an important role in determining the transition probabilities. Therefore, to incorporate this information, we consider the forward and backward recurrence processes (Kulkarni 1996) and obtain the expressions for perturbed transition probabilities. We know that for a renewal process $\{N(t),~t\ge 0\}$ with renewal epochs as $\{T_1,T_2,\ldots \}$, the backward and forward recurrence processes are respectively defined as

$$\begin{aligned} B(t)= & {} t-T_{N(t)}, \end{aligned}$$

(2)

$$\begin{aligned} F(t)= & {} T_{N(t)+1}-t. \end{aligned}$$

(3)

Here, B(t) represents the time elapsed since the most recent renewal at or before time t and F(t) represents time from t until first renewal after t. It is referred to as the residual process or forward recurrence process. We can incorporate these two recurrence processes at the initial and final time for non-homogeneous MRGP. Figure 1 represents the sample path of a time non-homogeneous MRGP with recurrence times at the initial and final time. One can observe that at initial time s, system is in state i but it entered state i at time $T_1$, that is, the backward process is $s-T_1$ and the forward process is $T_2-s$. One can define similarly at the final time t. Our investigation aims to understand how the transition probabilities of the MRGP are perturbed by imposing constraints on the recurrence processes both at initial and final times. The results presented here are natural generalizations of the results obtained in D’Amico (2011) for SMP.

Fig. 1

Sample path of non-homogeneous MRGP with recurrence times. At initial time s, the system is in state i but it entered state i at time $T_1$, that is, the backward process is $s-T_1$ and the forward process is $T_2-s$. Similarly, at the final time t, the backward process is $t-T_n$ and the forward process is $T_{n+1}-t$. On the contrary, in case of homogeneous MRGP, backward and forward process values are assumed to be equal to 0

Definition 2.1

For $i\in \Omega ',~j\in \Omega$ and for $s<{\tilde{a}}<t<{\tilde{b}}$, the transition probabilities with age and residual life only at the final time, are defined as

$$\begin{aligned} V_{ij}^{BF}(s;{\tilde{a}},t,{\tilde{b}})=P(Z(t)=j,B(t)\le t-{\tilde{a}},F(t)\le {\tilde{b}}-t\mid Z(s)=i, T_{N(s)}=s). \end{aligned}$$

(4)

We can obtain the transition probabilities $V_{ij}^{BF}(s;{\tilde{a}},t,{\tilde{b}})$ as follows:

Theorem 2.1

For $i\in \Omega ',~j\in \Omega$ and for $s<{\tilde{a}}<t<{\tilde{b}}$, we have

$$\begin{aligned} V_{ij}^{BF}(s;{\tilde{a}},t,{\tilde{b}})= & {} I_{\{{\tilde{a}}=s\}}(E_{ij}(s,t)-E_{ij}(s,{\tilde{b}})) \nonumber \\&+\sum _{m\in \Omega '}\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})dK_{im}(s,\theta ), \end{aligned}$$

(5)

where $I_A$ is the indicator function, that is, $I_A=1~\text{ if }~A~\text{ occurs } \text{ and }~0~\text{ otherwise }$.

Proof

See “Appendix” for proof. $\square$

Next, we define the transition probabilities of an MRGP with recurrence times at both the initial and final time points.

Definition 2.2

For $i\in \Omega ',~j\in \Omega$ and for $a<s<b<{\tilde{a}}<t<{\tilde{b}}$, the transition probabilities with backward and forward time at the initial and final time, are defined as

$$\begin{aligned}&^{bf}V_{ij}^{BF}(a,s,b;{\tilde{a}},t,{\tilde{b}})=\\&P(Z(t)=j,B(t)\le t-{\tilde{a}},F(t)\le {\tilde{b}}-t\mid Z(s)=i,B(s)=s-a,F(s)>b-s), \end{aligned}$$

provided $1-H_i(a,b)>0$ where $H_i(a,b)=P(T_{N(s)+1}>b \mid Z(s)=i,T_{N(s)}=a)$ and is given by

$$\begin{aligned} H_{i}(a,b)=\displaystyle \sum _{j\in \Omega }E_{ij}(a,b). \end{aligned}$$

We obtain the transition probabilities $^{bf}V_{ij}^{BF}(a,s,b;{\tilde{a}},t,{\tilde{b}})$ in the following theorem:

Theorem 2.2

For $i\in \Omega ',~j\in \Omega$ and for $a<s<b<{\tilde{a}}<t<{\tilde{b}}$ such that $1-H_i(a,b)>0$, we have

$$\begin{aligned} ^{bf}V_{ij}^{BF}(a,s,b;{\tilde{a}},t,{\tilde{b}})= & {} I_{\{{\tilde{a}}=a\}}\frac{(E_{ij}(a,t)-E_{ij}(a,{\tilde{b}}))}{1-H_i(a,b)} \nonumber \\&+\sum _{m\in \Omega '}\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\frac{dK_{im}(a,\theta )}{1-H_i(a,b)}, \end{aligned}$$

(6)

where $V_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})$ are given by Eq. (5).

Proof

See the “Appendix” for the proof. $\square$

Special Cases: The following are the particular cases of Eqs. (5) and (6), which can be of interest.

• 1 The probability with backward at initial time, denoted by $^{b}V_{ij}(a,s;t)$, is

  $$\begin{aligned} ^{b}V_{ij}(a,s;t)=P(Z(t)j\mid Z(s)=i, B(s)=a-s). \end{aligned}$$

  After setting $s=b$ and adjusting ${\tilde{a}}$ and ${\tilde{b}}$ terms in Eq. (6), we obtain

  $$\begin{aligned} ^{b}V_{ij}(a,s;t)=\frac{E_{ij}(a,t)}{1-H_i(a,s)}+\sum _{m\in \Omega '}\int _s^tV_{mj}(\theta ;t)\frac{dK_{im}(a,\theta )}{1-H_i(a,s)}. \end{aligned}$$

  (7)

• 2 If there is no backward, that is, $a=s$ in Eq. (7), we derive Eq. (1),

  $$\begin{aligned} V_{ij}(s,t)=E_{ij}(s,t)+\sum _{\gamma \in \Omega '} \int _s^t V_{\gamma j}(y,t)dK_{i\gamma }(s,y),~i\in \Omega ^{'} ,j\in \Omega ,~s\le t. \end{aligned}$$

  (8)

• 3 If $\{Z(t),t\ge 0\}$ is an SMP, we have every transition time epoch as a regeneration epoch and $\Omega '=\Omega$. In this case, the local kernel of MRGP becomes,

  $$\begin{aligned} E_{ij}(s,t)=P(Z(t)=j,T_{N(t)+1}>t\mid Z(s)=i)=\delta _{ij}(1-H_{i}(s,t)). \end{aligned}$$

  After substituting the value of $E_{ij}(s,t)$ and replacing $\Omega '$ by $\Omega$ in Eq. (6), we have the following result: Result: For $i,j\in \Omega$ and for $a<s<b<{\tilde{a}}<t<{\tilde{b}}$ such that $1-H_i(a,b)>0$, we obtain

  $$\begin{aligned} ^{bf}V_{ij}^{BF}(a,s,b;{\tilde{a}},t,{\tilde{b}})= & {} \delta _{ij}I_{\{{\tilde{a}}=a\}}\frac{(H_{i}(a,{\tilde{b}})-H_{i}(a,t))}{1-H_i(a,b)} \nonumber \\&+\sum _{m\in \Omega }\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\frac{dK_{im}(a,\theta )}{1-H_i(a,b)} \end{aligned}$$

  (9)

  where $V_{mj}^{BF}(u;{\tilde{a}},t,{\tilde{b}})$ is given by

  $$\begin{aligned} V_{ij}^{BF}(u;{\tilde{a}},t,{\tilde{b}})= & {} \delta _{ij}I_{\{{\tilde{a}}=u\}}(H_{i}(u,{\tilde{b}})-H_{i}(u,t)) \\&+\sum _{m\in \Omega }\int _u^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})dK_{im}(u,\theta ) \end{aligned}$$

  which is same as the result in D’Amico (2011).

Remark 2.1

Note that the results derived in Theorems 2.1 and 2.2 are general because they do not pertain to a particular application. More specifically, these results generalize the theory of time-homogeneous MRGP. Before applying these formulae to any application, one needs to determine the global and local kernels corresponding to that particular application. We investigate its application to model the dynamics of credit ratings.

Application to credit rating model and analysis

Following Pasricha et al. (2017), Fig. 2 illustrates the state transition diagram of the credit rating model. The determination of kernels for credit rating application in a non-homogeneous framework is a direct extension of the results from Pasricha et al. (2017).

Fig. 2

State transition diagram for a MRGP model with catastrophe for credit rating dynamics. Here, the state space is $\{AAA,AA,A,BBB,BB,B,C,D\}$, where D denotes the default, an absorbing state

Determining the global kernel

Let $p_{ij}(s)$ be the one-step transition probabilities for the embedded time non-homogeneous Markov chain at time s. Then, for $i,j\in \Omega ^{'}$,

$$\begin{aligned} p_{ij}(s)=\lim _{t\rightarrow \infty }K_{ij}(s,t). \end{aligned}$$

Let $F_{ij}(s,t); i \in \{1,2,\ldots ,7\},~j \in \Omega$ represents the distribution function of the waiting time in each state i, given that next state j is known. The global kernel $K(s,t)=[K_{ij}(s,t)]_{i,j \in \Omega ^{'}},~s\le t$ is given by

$$\begin{aligned} K_{ij}(s,t)=F_{ij}(s,t)p_{ij}(s). \end{aligned}$$

Note that $K_{ij}(s,t)==0$ when $p_{ij}(s)=0$.

Determining the local kernel

Now, we determine $E_{ij}(s,t),~i\in \Omega ^{'}$, $j\in \Omega$ and $s\le t$, the probability of migrating from state i at time s to state j at time t without any regeneration (upward movement). Since a transition to any good state from a bad state is a regeneration epoch, if a firm’s present rating is i and $j<i$, we obtain

$$\begin{aligned} E_{ij}(s,t)=0 ~~ if ~~ i>j,~s\le t,~~i\in \Omega ',j\in \Omega . \end{aligned}$$

For $i\le j$, let us define $\Omega ^{(i)}$ as the set of all states reachable from state i by downward movement that is $\Omega ^{(i)}=\{i,i+1,...,8\}$ for each $i\in \Omega ^{'}$. Let $\{M^{(i)}(t);~t\ge s\}, i\in \Omega ^{'}$ be the subordinated non-homogeneous semi-Markov process (since sojourn time in each state has general distribution in which there are only downward movements from state i at time s but no upward movement). Here, we can address the problem of rating momentum by selecting the parameters so that the probability of downward movement is high. The transition probabilities of this time non-homogeneous semi-Markov process for each $i\in \Omega '$ will give us $E_{ij}(s,t)~\forall ~j$, the local kernel of non-homogeneous MRGP.

The transition probabilities of subordinated non-homogeneous SMP’s can be obtained by solving it for a non-homogeneous SMP, say $\{M^{(i)}(t),~ t\ge s\}$ with state space $\Omega ^{(i)}$ but with only downward movement possible that is from state i at time s to state $j~(j>i)$, $i,j \in \Omega ^{(i)}$. Let $(Y_n^{(i)},S_n^{(i)})$ be the non-homogeneous Markov renewal sequence where $Y_n^{(i)}$ represents the state at nth transition and $S_n^{(i)}, n\ge 1$ with the state space equal to ${\mathbb {R}}^+$ represents the time of nth transition.

The kernel $Q^{(i)}(s,t)=[Q^{(i)}_{jk}(s,t)],~j,k \in \Omega ^{(i)},~s\le t$ of non-homogeneous SMP is defined by

$$\begin{aligned} Q^{(i)}_{jk}(s,t)=P(Y^{(i)}_{n+1}=k;S^{(i)}_{n+1}-S^{(i)}_n\le t\mid Y^{(i)}_n=j,S^{(i)}_n=s),~j,k \in \Omega ^{(i)},~t\ge s, \end{aligned}$$

and it follows that

$$\begin{aligned} q^{(i)}_{jk}(s)=\lim _{t\rightarrow \infty } Q^{(i)}_{jk}(s,t),~j,k \in \Omega ^{(i)}, \end{aligned}$$

is the transition probabilities of the embedded time non-homogeneous DTMC in the process. Therefore, for $j,k \in \Omega ^{(i)}, ~t\ge s$,

$$\begin{aligned} G^{(i)}_{jk}(s,t)= & {} P(S^{(i)}_{n+1}-S^{(i)}_n\le t\mid Y^{(i)}_n=j,Y^{(i)}_{n+1}=k,S^{(i)}_n=s)\\= & {} \left\{ \begin{array}{ccc} \frac{Q^{(i)}_{jk}(s,t)}{q^{(i)}_{jk}(s)} &{} \text{ if } &{} q^{(i)}_{jk}(s)\ne 0 \\ 1 &{} \text{ if } &{} q^{(i)}_{jk}(s)=0\\ \end{array} \right. . \end{aligned}$$

Clearly, $G^{(i)}_{jk}(s,t)$ is same as $F^{(i)}_{jk}(s,t)$ introduced in “Data and methodology” section. Furthermore, probability that the process will be in state j at time t is given by

$$\begin{aligned} H^{(i)}_j(s,t)=P(S^{(i)}_{n+1}-S^{(i)}_n \le t\mid Y^{(i)}_n=j,S^{(i)}_n=s),~j\in \Omega ^{(i)},~s\le t. \end{aligned}$$

It can be observed that

$$\begin{aligned} H^{(i)}_j(s,t)=\sum _{k=j}^{8}Q^{(i)}_{jk}(s,t). \end{aligned}$$

Now, the transition probabilities of the time non-homogeneous SMP $\{M^{(i)}(t),~t\ge s\}$ are defined by

$$\begin{aligned} \phi ^{(i)}_{jk}(s,t)=P(M^{(i)}(t)=k\mid M^{(i)}(s)=j),~j,k \in \Omega ^{(i)},~s\le t \end{aligned}$$

and are obtained by solving the following renewal equation

$$\begin{aligned} \phi ^{(i)}_{jk}(s,t)=\delta _{jk}(1-H^{(i)}_j(s,t))+\sum _{\beta =j}^k \int _s^t \phi ^{(i)}_{\beta k}(y,t)dQ^{(i)}_{j\beta }(s,y). \end{aligned}$$

In addition, observe that

$$\begin{aligned} \phi ^{(i)}_{jk}(s,t)=0 ~~\text{ if }~ j>k. \end{aligned}$$

Finally, the transition probabilities in a matrix form are given by

$$\begin{aligned} {\varvec{\Phi }}^{(i)}(s,t)=\mathbf{H }^{(i)}(s,t)+\mathbf{Q }^{(i)}(s,t)*{\varvec{\Phi }}^{(i)}(s,t), \end{aligned}$$

where $\mathbf{H }^{(i)}(s,t)$ is the diagonal matrix with jth diagonal entry $1-\mathbf{H }^{(i)}_j(s,t).$ This ${\varvec{\Phi }}^{(i)}(s,t)$ will be upper triangular matrix.

Therefore, for given initial state $i\in \Omega '$, the local kernel $E_{ij}(s,t)~\forall ~j\ge i,~ t\ge s$ is obtained as follows:

$$\begin{aligned} E_{ij}(s,t)= & {} P\{Z(t)=j,T_{n+1}> t |Z(T_n)=i,T_n=s\}\\= & {} P\{Z(t)=j|T_{n+1}> t,Z(T_n)=i,T_n=s\}P\{T_{n+1}> t |Z(T_n)=i,T_n=s\}\\= & {} \phi ^{(i)}_{ij}(s,t)\left( 1-P(T_{n+1}\le t|Z(T_n)=i,T_n=s)\right) \\= & {} \phi ^{(i)}_{ij}(s,t)\left( 1-\sum _{k\in \Omega '}K_{ik}(s,t)\right) . \end{aligned}$$

For each $i\in \Omega ^{'}$, this $E_{ij}(s,t),~j\in \Omega ,~s\le t$ of the MRGP describes the behavior of ratings evolution between two regeneration epochs, which is how the rating moves to a lower rating before moving to an upper rating and is given by

$$\begin{aligned} E_{ij}(s,t)= \left\{ \begin{array}{ccc} 0 &{} \text{ if } &{} i>j \\ \phi ^{(i)}_{ij}(s,t)\left( 1-\sum _{k\in \Omega '}K_{ik}(s,t)\right) &{} \text{ if } &{} i\le j\\ \end{array} \right. . \end{aligned}$$

Hence, by taking $F_{ij}(t)$ with different intensities, increasing or decreasing, we take care of downward momentum using local kernel.

Time-dependent solution of non-homogeneous MRGP

The time-dependent solution for $t\ge s$ of the time non-homogeneous MRGP with recurrence times is given by

$$\begin{aligned} ^{bf}V_{ij}^{BF}(a,s,b;{\tilde{a}},t,{\tilde{b}})= & {} I_{\{{\tilde{a}}=a\}}\frac{(E_{ij}(a,t)-E_{ij}(a,{\tilde{b}}))}{1-H_i(a,b)}\\&+\sum _{m\in \Omega '}\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\frac{dK_{im}(a,\theta )}{1-H_i(a,b)}, \end{aligned}$$

where $V_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})$ is obtained from Eq. (5) and E(s, t) and K(s, t) are as in “Determining the global kernel” and “Determining the local kernel” sections.

This Markov renewal equation, which is a special case of indexed Markov renewal equations, is a Volterra integral equation of Type II and can be solved through discretization (D’Amico 2011).

Numerical illustration

This section presents the empirical analysis performed on S&P rating agency’s credit rating data. The objective of these experiments is to (a) observe how the inclusion of recurrence times (forward and backward) impacts the transition probabilities of the MRGP and (b) examine the performance of non-homogeneous MRGP on real-life data in comparison to the homogeneous MRGP and SMP models. All the experiments are performed using R software. We start with a description of the data followed by the methodology adopted to estimate the unknown parameters of the model.

Data and methodology

We use the same data as in Pasricha et al. (2017), that is, the quarterly data of long-term issuer ratings for companies that S&P has rated in the US from 1985 to 2015. In contrast to Pasricha et al. (2017), the model is time non-homogeneous; we therefore need to obtain the parameters at different initial times s to address the time non-homogeneity. For this purpose, we discretize the time interval and consider the time corresponding to the year 1985 as $s=0$ and that corresponding to the year 2015 as $s=30$. On the other hand, Pasricha et al. (2017) considered just one time and used the complete data set to estimate the parameters only once.

Estimation of the parameters

The parameters required for the model implementation are as follows:

• (a) the matrix P(s) of the embedded Markov chain at each $s=0,1,2,\ldots ,30$: Following Pasricha et al. (2017), for a fixed s, using the historical data until time s, a frequency matrix is obtained with (i, j)th element representing the number of transitions from i to j in one unit time. The transition probabilities $p_{ij}(s),i,j\in \Omega$ are then deduced by normalizing the obtained frequency matrix. The procedure is repeated for each $s\in \{0,1,\ldots ,30\}$. In contrast to Pasricha et al. (2017), who only estimate the P matrix at $s=30$, we estimate the P(s) every year to incorporate the time homogeneity.

• (b) the waiting time distribution $F_{ij}(s,t),i,j\in \Omega$ in initial state i at time s and final state j at time t. For fixed s and t, the following steps are followed:

  • (1) First, we identify the transitions with initial rating i at time s and the final rating j before time t, that is, in a time duration of $t-s$.

  • (2) For all the identified transitions, we obtain the best fit distribution and estimate its parameters using maximum likelihood estimation.

  • (3) Further, the Kolmogorov–Smirnov (KS) test is applied to statistically test the accuracy of the best fit distribution. Based on the hypothesis testing, we found that Weibull³ distribution is the best fit. This is in accordance with the results in Carty and Fons (1994), who observed that the Weibull distribution closely models the life span characteristics of bond ratings.

Results

This subsection presents the numerical results obtained through the experiments on the real-life data. In “Impact of time non-homogeneity and ageing effect” section, we provide the results of the experiments performed to investigate whether the proposed model can address the time non-homogeneity and ageing effect present in the rating dynamics. Here, we do not examine whether the proposed model is more efficient than those defined in the literature. We present the comparison with those models in “Comparison” section, where we demonstrate that the proposed model significantly outperforms the other models and provides a more accurate fit to the real-life data.

First, we follow the abovementioned procedure to estimate the transition probability matrices P(s) of the embedded Markov chain and the waiting distributions $F_{i,j}(s,t)$. We then substitute these estimates in the global and local kernels obtained in “Determining the global kernel” and “Determining the local kernel” sections, which yields transition probabilities of the time non-homogeneous MRGP upon substitution in Equation (5).

Impact of time non-homogeneity and ageing effect

For illustration purposes, we present the one-step transition probability matrix P(s) of the embedded Markov chain at time $s=0,4,7$ in Matrix 1, Matrix 2, and Matrix 3, respectively⁴. Further, we wish to highlight that if we estimate the matrix P(s) using a time-homogeneous MRGP model in Pasricha et al. (2017), the estimated matrix is independent of the initial time s. More specifically, if the user wants to estimate the one-step transition probability matrix at two different times, say $t_1$ and $t_2$, the resulting matrix will be the same if estimated through the model of Pasricha et al. (2017) (e.g., Matrix 1 in Pasricha et al. 2017), but will be significantly different if calculated using the proposed model.

Matrices 1 to 3 illustrate the variation in the P(s) at different s for the embedded Markov chain. However, the process that governs the credit rating dynamics is the process $\{Z(t),~t\ge 0\}$, a time non-homogeneous MRGP and not the embedded Markov chain. Therefore, it is essential to determine whether this time dependence carries over to the process $\{Z(t),~t\ge 0\}$. For this purpose, we utilize Equation (5) to estimate the transition probabilities of the credit rating process over a 5-year time horizon starting at time $s=0$ and $s=3$. The results are presented in Matrix 4 and Matrix 5, from which we can deduce that the time dependence indeed carries over to the process $\{Z(t),~t\ge 0\}$. More specifically, the variation among the three matrices implies that the proposed model can capture the time non-homogeneity present in the credit rating dynamics, an empirical phenomenon documented in the literature (Nickell et al. 2000). At this stage, we can only say that the proposed model can capture time non-homogeneity, but we cannot claim that it will outperform the other models. Again, if $V(s,s+5)$ is estimated using the time-homogeneous model, the same matrix would be obtained for any choice of s.

Matrix 1

This matrix represents the one-year transition probability matrix P(s) of the embedded Markov chain, estimated at time $s=0$. This matrix is an important ingredient in evaluating the transition probabilities of the non-homogeneous MRGP process $\{Z(t),~t\ge 0\}$.

Matrix 2

This matrix represents the one-year transition probability matrix P(s) of the embedded Markov chain, estimated at time $s=4$.

Matrix 3

This matrix represents the one-year transition probability matrix P(s) of the embedded Markov chain, estimated at time $s=7$.

Matrix 4

This matrix is the 5-year transition probability matrix $V(s,s+5)$ of the time non-homogeneous MRGP process estimated at time $s=0$. This is estimated using Equation (5) after the global and local kernels are estimated as described in “Determining the global kernel” and “Determining the local kernel” sections.

Matrix 5

This matrix is the 5-year transition probability matrix $V(s,s+5)$ of the time non-homogeneous MRGP process estimated at time $s=3$. This is estimated using Equation (5) after the global and local kernels are estimated as described in “Determining the global kernel” and “Determining the local kernel” sections.

Fig. 3

This figure presents the cumulative probability of default over a period of 10 years for ratings A, BBB, BB, B estimated at different choice of initial times $s=0,1,2,3$. Note that the figure is on the log scale. Further, the x-axis depicts the time as $t=s+h$. These probabilities are estimated under the assumption that there is no age at initial time, which means the firm entered the initial state at time s itself. This figure confirms that the inclusion of time non-homogeneity in the model significantly impacts the default probabilities

Further to confirm whether the variation visible in Matrices 1, 2, and 3 and Matrices 4 and 5 are statistically significant, we first plot the cumulative probability of default over ten years for ratings A, BBB, BB, B at different initial times $s=0,1,2,3$ on a log scale in Fig. 3. On the x-axis, time is given as $t=s+h$. These probabilities are estimated by the proposed model considering no age at the initial time that is, the firm entered the initial state at time s. The variation in the matrices is now translated into the cumulative default probabilities. For example, in the cumulative probability of default for rating BB in Fig. (3c), note the difference in probability of default curves for times $s=0$, $s=1$ and times $s=1$,$s=2$. On the other hand, a time-homogeneous model (Pasricha et al. 2017) would not capture this behavior of rating dynamics, that is, the model will generate the same values for any choice of s. To check the statistical significance of the differences in the curves for two different values of s for any particular rating, we perform a one-sided Student’s t-test as follows. For a fixed rating, say BB, we apply two-sided Student’s $t-$test on the difference between the cumulative default probabilities for two different values of s, say $s_1$ and $s_2$. We then repeat this test for all the possible pairs $(s_1,s_2)$ and all the rating categories. More concretely, we test the following

$$\begin{aligned} H_0: e_1-e_2= 0,~~~H_1: e_1-e_2\ne 0 , \end{aligned}$$

where $e_1$ and $e_2$ are the cumulative default probabilities for time $s_1$ and $s_2$, respectively. For 21 out of the 24 tests performed, the p-values obtained are less than 0.05, thus rejecting the null hypothesis. Therefore, the variations observed for different values of initial times are statistically significant.

Fig. 4

This figure presents the cumulative default probability over a horizon of 10 years for the credit ratings A, BBB, BB, B when estimated at time $s=5$ on a log scale. Further, these probabilities are estimated under the assumption that there is some age a at the initial time $a=1,2,3,4$. More specifically, the firm entered the initial state at time $s-a$ and continued in the same rating until time s. On the x-axis we measure the time h from which we get $t=s+h$. This figure confirms that the inclusion of the age process in the model significantly impacts the default probabilities

The previous experiments examined how the time non-homogeneity impacts the transition probabilities of the MRGP. However, to determine the impact of ageing, Fig. 4 presents the cumulative probability of default over ten years for ratings A, BBB, BB, B estimated at time $s=5$ on a log scale. These probabilities are estimated considering the age at initial time $a=1,2,3,4$, that is, the firm entered the initial state at time $s-a$ and continued in the same rating until time s. We can deduce that the default probabilities vary with time of rating evaluation and duration inside the rating, which is further confirmed by the hypothesis test performed similarly to as outlined in the previous paragraph. In conclusion, the inclusion of time non-homogeneity and ageing processes in a time-homogeneous MRGP model significantly impacts the transition probabilities.

Comparison

Fig. 5

This figure presents the average default probabilities over a horizon of 10 years for different rating categories. For each rating category, we have the real probabilities obtained through S&P and the estimated probabilities from four models namely a time-homogeneous SMP, b time non-homogeneous semi-Markov process (NHSMP), c time-homogeneous MRGP, and d time non-homogeneous Markov regenerative process (NHMRGP). It reveals that the average probabilities obtained from NHMRGP are close to the real probabilities, which is further confirmed by a hypothesis test in “Comparison” section

The results presented in “Impact of time non-homogeneity and ageing effect” section advocate for the time non-homogeneous MRGP model to address non-Markov patterns in the rating dynamics; nevertheless, we still need to test whether the proposed model provides better estimates in comparison to those obtained from the other models in the literature. For this purpose, we compare the proposed model with the existing models, non-homogeneous semi-Markov, MRGP, and semi-Markov models with the default probabilities given by S&P for 1981-2014. We consider the average S&P default rates provided in Richhariya et al. (2015).

For comparison, we average out the default distributions obtained by the proposed model over the period for different initial times s and different possible backward recurrence time a to obtain the average default probabilities over the period. A similar procedure is followed for other models in the literature by fixing the $F_{ij}(s,t)$ and $p_{ij}(s)$ for non-homogeneous versions. Figure 5 presents the default distribution on a log scale. We observe that the default probabilities obtained for a horizon of 10 years by non-homogeneous MRGP model are closer to the actual default probabilities obtained by S&P in contrast to the MRGP (Pasricha et al. 2017), SMP, and its non-homogeneous counterpart. To justify that these observations are statistically significant and are not just by chance, we perform a hypothesis test similar to that in Pasricha et al. (2017). We perform a one-sided Student’s t-test on the difference between the errors obtained from different models, that is,

$$\begin{aligned} H_0: E_1\ge E_2,~~~H_1: E_1< E_2. \end{aligned}$$

By fixing $E_1$ to be the error between the default rates obtained from the proposed model and S&P, we perform this test three times, one for each of the other three models. Here, $E_2$ is selected to be the errors between the other models and S&P. For each of the four tests, the p-values obtained are less than 0.05, thus rejecting the null hypothesis. Therefore, we conclude that the proposed model provides a better fit to the rating dynamics compared to the existing models. As a result, it can provide better forecasts for risk management purposes.

Conclusion

We introduced a time non-homogeneous MRGP with recurrence times to address the time dependence and ageing effect on the transition probabilities. The applicability of the proposed model in developing a credit rating model is illustrated using real data of credit rating dynamics obtained from Bloomberg. The proposed model can address the issues of the suitability of Markov processes in the credit risk environment, such as dependence on the time of rating evaluation, dependence on the business cycle, duration effects, and momentum effects. Comparison between the average default rates obtained by existing models, S&P report, and the proposed model reveals that the proposed model captures the rating dynamics better than other models in the literature. The proposed model can provide a more general and better approach to model credit ratings. Hence, it is of significant interest to banks and financial institutions concerned about the default risk.

Appendix

Definition: Non-Homogeneous MRGP:

A sequence of bi-variate random variables $\{(X_n,T_n),~n=0,1,\ldots \}$ is known as a time non-homogeneous Markov renewal sequence (Cinlar 1975; Kulkarni 1996) if

• 1. ${T_0=0, ~T_{n+1}\ge T_n;~ X_n \in \Omega = \{0,1,2,\ldots \}}$

• 2. $\forall ~ n \ge 0,$ $P\{X_{n+1}=j,T_{n+1}-T_n\le t \mid X_n=i,T_n=s,X_{n-1},T_{n-1},\ldots ,X_0,T_0\}$ $=P\{X_{n+1}=j,T_{n+1}-T_n \le t \mid X_n=i,T_n=s\}$ (Markov property)

A stochastic process $\{Z(t),~t\ge 0\}$ with state space $\Omega$ is referred to as a time non-homogeneous MRGP if a non-homogeneous continuous-time Markov renewal sequence {$(X_n,T_n),~n=0,1,\ldots$} exists such that all conditional finite dimensional distributions of {$Z(T_{n}+t),~t\ge 0$} given {$Z(u),0\le u\le T_n, X_n=i,T_n=s\}$ are same as those of {$Z(T_n+t),~t\ge 0$} given $\{Z(T_n),T_n=s,X_n=i\}$, $i\in \Omega ^{'} \subset \Omega$. This implies that, in this case {$Z(T_n^+)=X_n,~n=0,1,\ldots$} or {$Z(T_n^-)=X_n,~n=0,1,\ldots$} is an embedded discrete time non-homogeneous Markov chain and also that $T_n$’s are regeneration points of {$Z(t),~t\ge 0$}. Here, $T_n^+$ and $T^-_n$ represent the time immediately after and immediately before the regeneration, respectively. Here, the non-homogeneity is considered with respect to the time. In MRGP, the time spent inside a state can be any distribution unlike the Markov environment, where it has to be an exponential distribution. The sample path of time non-homogeneous MRGP is presented in Fig. 6 where $\{T_1,T_2,\ldots \}$ is a sequence of regeneration time points. One can observe that the state of the process can change between two regeneration points $T_n$ and $T_{n+1}$ unlike the semi-Markov environment where each change of state is a regeneration point. In other words, each change of state does not renew the system and hence local behaviors exist between two consecutive Markov regenerative points.

Fig. 6

Sample path of a time non-homogeneous MRG

To investigate the dynamics of the process, we need to define the behavior at regeneration time points (global kernel) and in between the regeneration time points (local kernel). First, we define the global kernel $K(s,t)=[K_{ij}(s,t)]_{i,j \in \Omega ^{'}}$ associated with the process as

$$\begin{aligned} K_{ij}(s,t)=P\{Z(T_{n+1})=j,T_{n+1}-T_n \le t |Z(T_n)=i,T_n=s\}, i,j\in \Omega ', s\le t. \end{aligned}$$

It is the probability that starting from regeneration state i at time s, the next regeneration is at state j within time t. It follows that $P(s)=[p_{ij}(s)]_{i,j\in \Omega ^{'}}$, the one-step transition probability matrix at time s of the embedded time non-homogeneous DTMC with state space $\Omega ^{'}$ is given by

$$\begin{aligned} p_{ij}(s)=P\{Z(T_{n+1})=j\mid Z(T_n)=i,T_n=s\}=\lim _{t\rightarrow \infty } K_{ij}(s,t),~~ i,j\in \Omega ^{'}. \end{aligned}$$

Next, we define the local kernel $E(s,t)=[E_{ij}(s,t)]_{i \in \Omega ^{'},~j\in \Omega }$ as

$$\begin{aligned} E_{ij}(s,t)=P\{Z(t)=j,T_{n+1}> t |Z(T_n)=i,T_n=s\},~~i\in \Omega ^{'},j\in \Omega ,~s\le t. \end{aligned}$$

It is the probability that, starting from regeneration state i at time s, no regeneration is reached within time t and the state at time t is j. It describes the local behavior, that is, the behavior of the process between two regeneration epochs of the embedded time non-homogeneous DTMC.

Proof of Theorem 2.1

Proof

Using the standard first renewal argument, we have

$$\begin{aligned}&P(Z(t)=j,B(t)\le t-{\tilde{a}},F(t)\le {\tilde{b}}-t\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}>t\mid Z(s)=i,T_{N(s)}=s)+ \nonumber \\&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}\le t\mid Z(s)=i,T_{N(s)}=s). \end{aligned}$$

(10)

In the first term, the conditions $\{T_{N(s)+1} > t\}$, $\{T_{N(t)+1}\in [t,{\tilde{b}}]\}$ and $\{T_{N(t)}\in [{\tilde{a}},t]\}$ imply that the Markov renewal process at t and s is equal, that is, $N(t) = N(s)$ and there is no regeneration until time t. Hence, we have

$$\begin{aligned}&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}], T_{N(s)+1}>t\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} P(Z(t)=j,T_{N(s)}={\tilde{a}},T_{N(s)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} I_{\{{\tilde{a}}=s\}}P(Z(t)=j,T_{N(s)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} I_{\{{\tilde{a}}=s\}}(E_{ij}(s,t)-E_{ij}(s,{\tilde{b}})), \end{aligned}$$

(11)

where the second equality follows from the observation that last regeneration occurred at s, which is the same as ${\tilde{a}}$ and the last equality follows from the definition of the local kernel that there is no regeneration until time t and state at time t is j.

We can write the second term in Eq. (10) using the regenerative property at time s as follows

$$\begin{aligned}&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}], T_{N(s)+1}\le t\mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} \sum _{m\in \Omega '}\int _s^t P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}] \mid T_{N(s)+1}=\theta ,Z(\theta )=m) \nonumber \\&\times P(Z(\theta )=m,T_{N(s)+1}=\theta \mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} \sum _{m\in \Omega '}\int _s^t V_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\times P(Z(\theta )=m,T_{N(s)+1}=\theta \mid Z(s)=i,T_{N(s)}=s) \nonumber \\= & {} \sum _{m\in \Omega '}\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})dK_{im}(s,\theta ). \end{aligned}$$

(12)

From Eqs. (10) to (12), we get the desired result (5). $\square$

Proof of Theorem 2.2

Proof

Similar to Theorem 2.1, following the standard first renewal argument, we can write

$$\begin{aligned}&P(Z(t)=j,B(t)\le t-{\tilde{a}},F(t)\le {\tilde{b}}-t\mid Z(s)=i,B(s)=s-a,F(s)>b-s) \nonumber \\= & {} P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b) \nonumber \\= & {} P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}>t\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b)+ \nonumber \\&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}\le t\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b). \end{aligned}$$

(13)

In the first term in Eq. (13), given $t>u$ the conditions $\{T_{N(s)+1} > u\}$ and $\{T_{N(s)+1} > t\}$ implies $N(t) = N(s)$ and there is no regeneration until time t. Consequently, we have

$$\begin{aligned}&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}>t\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b)\nonumber \\= & {} P(Z(t)=j,T_{N(s)}={\tilde{a}},T_{N(s)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b)\nonumber \\= & {} I_{\{{\tilde{a}}=a\}}P(Z(t)=j,T_{N(s)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b)\nonumber \\= & {} I_{\{{\tilde{a}}=a\}}\frac{P(Z(t)=j,T_{N(s)+1}\in (t,{\tilde{b}}]\mid Z(s)=i,T_{N(s)}=a)}{P(T_{N(s)+1}>b \mid Z(s)=i,T_{N(s)}=a)}\nonumber \\= & {} I_{\{{\tilde{a}}=a\}}\frac{E_{ij}(a,t)-E_{ij}(a,{\tilde{b}})}{1-H_i(a,b)}. \end{aligned}$$

(14)

We can write the second term in Eq. (13) using the regenerative property at time b as follows

$$\begin{aligned}&P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}],T_{N(s)+1}\le t\mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b)\nonumber \\= & {} \sum _{m\in \Omega '}\int _s^t P(Z(t)=j,T_{N(t)}\in [{\tilde{a}},t],T_{N(t)+1}\in (t,{\tilde{b}}] \mid T_{N(s)+1}=\theta ,Z(\theta )=m)\nonumber \\&\times P(Z(\theta )=m,T_{N(s)+1}=\theta \mid Z(s)=i,T_{N(s)}=a,T_{N(s)+1}>b) \nonumber \\= & {} \sum _{m\in \Omega '}\int _s^t V_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\times \frac{P(Z(\theta )=m,T_{N(s)+1}=\theta \mid Z(s)=i,T_{N(s)}=a)}{P(T_{N(s)+1}>b\mid Z(s)=i,T_{N(s)}=a)} \nonumber \\= & {} \sum _{m\in \Omega '}\int _s^tV_{mj}^{BF}(\theta ;{\tilde{a}},t,{\tilde{b}})\frac{dK_{im}(a,\theta )}{1-H_i(a,b)}. \end{aligned}$$

(15)

From Eqs. (13) to (15), we obtain the desired result (6). $\square$

Weibull distribution and MLE

The cumulative distribution function for the Weibull distribution is given by

$$\begin{aligned} F(x)=1-e^{(-\frac{x}{\lambda })^k}I_{\{x>0\}}, \end{aligned}$$

where $\lambda ,k>0$ are scale and shape parameters, respectively, and $I_A$ is an indicator function. The implicit equations to obtain the maximum likelihood estimator for the $\lambda$ and k are given by

$$\begin{aligned} {{\hat{\lambda }} }=\left( {\frac{1}{n}}\sum _{{i=1}}^{n}x_{i}^{k}\right) ^{1/{\hat{k}}}. \end{aligned}$$

The maximum likelihood estimator for k is

$$\begin{aligned} {{\hat{k}}}^{{-1}}={\frac{\sum _{{i=1}}^{n}x_{i}^{{\hat{k}}}\ln x_{i}}{\sum _{{i=1}}^{n}x_{i}^{{\hat{k}}}}}-{\frac{1}{n}}\sum _{{i=1}}^{n}\ln x_{i}. \end{aligned}$$