Estimation of Parameters of the Two-Parameter Rayleigh Distribution Based on Progressive Type-II Censoring Using Maximum Likelihood Method via the NR and the EM Algorithms

Murithi Daniel Fundi^*, Edward Gachangi Njenga, Kemboi George Keitany

Department of Statistics and Actuarial Science, Kenyatta University (KU), Nairobi, Kenya

Email address:

^*Corresponding author

To cite this article:

Murithi Daniel Fundi, Edward Gachangi Njenga, Kemboi George Keitany. Estimation of Parameters of the Two-Parameter Rayleigh Distribution Based on Progressive Type-II Censoring Using Maximum Likelihood Method via the NR and the EM Algorithms. American Journal of Theoretical and Applied Statistics. Vol. 6, No. 1, 2017, pp. 1-9. doi: 10.11648/j.ajtas.20170601.11

Received: November 15, 2016; Accepted: November 30, 2016; Published: December 20, 2016

Abstract: In this article, Maximum likelihood estimates for the shape and scale parameters of two-parameter Rayleigh distribution are obtained based on progressive type-II censored samples using the Newton-Raphson (NR) method and the Expectation-Maximization (EM) algorithm. A simple algorithm discussed in [2-3] is used for generating progressive type-II censored samples. Based on this censoring scheme, approximate asymptotic variances are derived and used to construct approximate confidence intervals of the parameters. The performance of these two maximum likelihood estimation algorithms is compared in terms of simulation results of root mean squared error (RMSE) and the coverage rates. Simulation results showed that in nearly all the combination of simulation conditions the estimators based on the EM algorithm have less root mean squared error (RMSE) and narrower widths of confidence intervals compared to those obtained using the NR algorithm. Finally, an illustrative example with real-life data sets is provided to illustrate how maximum likelihood estimation using the two algorithms works in practice.

Keywords: Two-Parameter Rayleigh Distribution, Maximum Likelihood Estimation, EM Algorithm, NR Method, Progressive Type-II Censoring

1. Introduction

The two-parameter Rayleigh distribution is a particular case of a Weibull distribution widely used in reliability theory and life testing. Rayleigh [25] introduced this distribution in connection with a problem in acoustics. Rayleigh distribution has a nice relation to other distributions including Chi-Square and most extreme value distributions. In addition, the hazard function of this distribution increases with an increase in time. As a result, the distribution has attracted several researchers as it occurs in different forms including one-parameter Rayleigh distribution, and two-parameter Burr type X distribution also known as the Generalized Rayleigh distribution. According to Surles and Padgett, the two-parameter Rayleigh distribution is an extreme value distribution that is effective in modeling general life data [26].

In literature, several distinguished authors have extensively studied estimation, inferential, and predictions issues for one-parameter Rayleigh distribution although not much has been done on two-parameter Rayleigh distribution. Interested readers are referred to [9, 10, 15, and 16] for exposure to the Rayleigh distribution.

Recently, Khan, Provost, and Singh [17] considered the predictive inference based on doubly censored samples for the two-parameter Rayleigh distribution. Very recently, Dey, Dey, and Kundu [12] derived interval and point estimates of the scale and location parameters of a two-parameter Rayleigh distribution using progressive Type-II censored samples.

A continuous random variable X is said to have a two-parameter Rayleigh distribution with a scale parameter  and location parameter µ, if its density function is given by:

(1)

The corresponding distribution function for x µ is given by:

(2)

The presence of the location parameter makes the two-parameter more effective in analyzing real life data sets compared to one-parameter Rayleigh distribution.

In reliability testing, an experimenter may cease testing before all the experimental units fail due to time constraint or lack of funds. Samples that results from such situations are known as censored samples. There are numerous censoring methods available to an experimenter with type-II and type-I censoring schemes being the commonly used schemes in life testing. A mixture of these two schemes results to a hybrid censoring scheme. However, type-II, type I, and hybrid censoring schemes do not give room for removal of experimental units before the terminal point of the experiment. Progressive type-II censoring scheme allows such removal hence it gained popularity in life-testing and reliability experiments. In this paper, we consider progressive Type-II censoring scheme.

In the recent statistical literature, progressive censoring scheme has attracted many reliability practitioners and theoreticians. Interested readers are referred to [2-4]. For more recent references, refer to [24,27], as well as references, cited therein.

Recently, Lio, Chen, & Tsai [19] investigated inference of the estimated parameters of the generalized Rayleigh distribution based on progressive type-I interval censoring scheme. The study reviewed that use of progressive type I interval censored samples to estimates the MLEs using Expectation Maximization algorithm yields more accurate and precise parametric estimates. Very recent, Dey et al. [12] derived interval and point estimates of the scale and location parameters of a two-parameter Rayleigh distribution using progressive Type-II censored samples.

The purpose of this article is to develop an estimation procedure for the scale and shape parameters of the two-parameter Rayleigh distribution based on progressive type-II censoring scheme. We first derive the maximum likelihood estimators of the unknown parameters. Since the MLEs of the shape and scale parameters of the two-parameter Rayleigh distribution cannot be obtained in the explicit form, we propose the use of the NR and the EM algorithms to compute the MLEs. Progressive type-II right censored samples are considered as incomplete data hence both the EM and the NR algorithms are suitable numerical iterative procedures for finding the MLEs. For more information regarding the EM algorithm including its application and advantages compared to those of NR method readers are referred to [1, 18, and 29]. For derivation and application of the Newton-type method refer to [20, 21, and 23].

The rest of the article is organized as follows. In section 2, progressive type-II censoring scheme is briefly discussed, the MLEs of the scale and location parameters are derived based on progressive type-II censoring using the EM and NR algorithms. Based on this censoring scheme, approximate asymptotic variances are derived and used to construct approximate confidence intervals of the parameters. In Section 3, simulation results and discussions are provided. In section 4, an illustrative example is provided using real-life data sets. In the final section, a conclusion is provided.

2. Parameter Estimation

2.1. Progressive Type-II Censoring Scheme

Let n identical items be put on a life-testing experiment at time 0 with the corresponding lifetimes X₁, X₂, X₃,…, X_n being independent and identically distributed with the density function given in equation (1). Further, suppose that integer m <n is fixed at the beginning of the experiment (where m <n is the number of units to be observed completely until failure) with  and specified.

This implies that progressive censoring will occur in m failure stages as follows. At the time of the first failure, a random sample of size R_1, (X₁: m: n) surviving items are randomly drawn from n-1 remaining surviving units in the experiment leaving n-1-R₁ survival units. At the time of the second failure, a random sample of size R₂, (X₂: m: n) is randomly drawn from n-2 surviving units leaving n-2-R₁ surviving items in the experiment. The process is continued until the m^th failure time X_m:n:m is evident (the m^th stage) when all R_m = n - m - R₁ - R₂ - … -R_m-1 surviving items are removed from the life-testing experiment. The set of an observed lifetime  is a progressively type II censored sample. According to Balakrishnan and Aggarwala, a progressively type –II censoring scheme consist of m failure stages and R₁, R₂,…, R_m random samples, such that  with fixed before the study. Where  denotes the j^thcensored random sample [2].

It is imperative to note that, if m=0 there is no censoring, if , then n=m (complete sample situation), and if , then, which is the conventional type-II right censoring scheme. In this article, we will use  in place of for j=1, 2, 3, …, m to make the notation simple.

2.2. Maximum Likelihood Estimation Based on Progressive Type-II Censoring

MLE is one of the standard techniques for estimating unknown parameters of distribution or a model. The principle concept behind this method is to select the value of the parameter under which the underlying data is most likely to be observed.

Suppose n identical units are placed at the same time on a life-testing experiment. Let x_{1: n}, x_{2: n}…x_m:n be a progressive type-II censored random sample from density function in equation (1). According to Balakrishnan and Aggarwala, m ordered failures out of the sample of size n are observed under this scheme and random samples R₁, R₂ … R_m of survival units drawn and removed from the experiment at each of m^th failure stage [2]. The likelihood function based on progressive type-II censored random samples as in [2] is given by:

(3)

Substituting the value of f (.) and F (.) in equation (3), the log likelihood function of  and  constructed on progressive type-II censored sample ignoring the constant term can be written as follows:

(4)

The log-likelihood function of (4) is written as:

(5)

2.3. Expectation-Maximization (EM) Algorithm

Let with  denotes the progressive type-II right-censored data from a population with density function and distribution given in equations (1) and (2), respectively.

We propose the use of EM algorithm discussed in [7] as follows.

Let some of the complete data vectors W be observed such that W = (Y; Z), where  and, for denotes the censored data (missing data) and denotes the observed data.

The log-likelihood function of the complete data set can be written as:

(6)

The MLEs of the parameters  and µ based on W are obtained as:

(7)

(8)

The E-Step of the EM algorithm requires substituting any function of Z_jk (say h (Z_ik)) by E (h (Z_jk) /Z_jk>y_j). Hence, equations (7) and (8) becomes

(9)

(10)

We make use of theorem by Ng et al. [22] that states that given ; the conditional distribution of Zik follows a truncated two-parameter Rayleigh distribution with left truncation at y_j. Hence,

The conditional expectations in equations (9) and (10) are obtained as:

(11)

(12)

(13)

The M-step of the (h+1)^th iteration of the EM algorithm is completed by substituting the above conditional expectations on to equations (9) and (10) as follows:

Hence,

(14)

 is the estimate of  at the (h+1)^th iteration of the EM algorithm.

Once  is obtained, µ^(h+1) is obtained as follows;

(15)

The value  is then used as a new value of  in the succeeding iteration. The MLEs of  can be obtained by repeating the E-step and M-step until convergence.

2.4. The Newton-Raphson Algorithm

We will directly extend the argument for deriving the Newton-Raphson algorithm for optimization in one dimension to two-dimensional problems as discussed by Devore and Berk [8] giving the two-parameter Newton-Raphson method as:

(16)

Where  is the Hessian matrix (a matrix with (i, j) entry equal to the second derivatives with respect to θ^j and θⁱ⁾ and  is the score function (a vector of derivatives).

From equation (5),  and  are obtained as:

(17)

(18)

Hence, equation (16) becomes

(19)

The procedure is reiterated until there is no significant difference between

2.5. Approximate Interval Estimation

The approximate asymptotic variances of the shape and the scale parameters and the confidence intervals are obtained as follows:

Let with  denoting a progressive type-II right-censored sample from a population with density and distribution functions given in equations (1) and (2), respectively.

The Fisher information matrix  is then obtained by taking the expectation of minus the second derivatives of equation (3.6) with respect to θ^j and θⁱ. Cox and Hinkley [6], established that if  belongs to an open interval of a real line, then some of Cramer-Rao regularity conditions are satisfied, and as the sample size increases, the distribution of the MLE tends to be approximately bivariate normally distributed with mean  and covariance matrix. In practice,  is estimated by . The distribution of the MLEs is denoted by:

, where  is the observed information matrix given by

(20)

Using equation (3.6), the elements of the observed information matrix can be obtained as follows:

                   (21)

(22)

Since,

The exact asymptotic variance of  cannot be obtained in explicit form. We rely on the results of Dey, Dey, and Kundu [11] who applied Corollary of Theorem 3 of Smith [28], to approximate the asymptotic variance of  by  using the inverse of the observed information as:

(23)

Hence,

(24)

Using equation (20), a  approximate confidence intervals for  and  are obtained as

(25)

Respectively, where,  is the -th percentile point of the standard normal distribution.

3. Results and Discussions

In this section, a simulation study is performed to compare the performance of MLEs of the two-parameter Rayleigh distribution obtained using the NR method and the EM algorithm based on progressive type-II censored samples. Progressive type-II is right censored samples from two-parameter Rayleigh distribution were generated using the algorithms discussed in [2-3].

In comparing the performance of the MLEs, four measures considered were the root mean squared error (RMSE) and the 95% approximate confidence width of MLEs. Suppose  is the MLE of  for the i^th replication of the  algorithm method simulated, then the Bias and RMSE of  are computed as follows:

i.    , where and or

ii.  

In this paper, samples of sizes 20, 30, 40, 50, and 70 were used and the censoring schemes considered are given in Table 1 and 2 below.

Table 1. Censoring Schemes for , and.

Table 2. Censoring Schemes for with fixed number of failures.

It is imperative to note that, in all the above censoring schemes no restriction has been imposed on the maximum number of iterations and convergence is assumed to occur when the absolute differences between successive estimates are less than 0.0001.

Table 3. The RMSE and the width of 95% approximate confidence intervals of the MLEs for the parameters of two-parameter Rayleigh distribution under progressive type-II censoring for the EM and NR algorithms using scheme 1.

Table 4. The RMSE and the width of 95% approximate confidence intervals of the MLEs for the parameters of two-parameter Rayleigh distribution under progressive type-II censoring by the EM and NR algorithms using scheme 2.

Table 5. The RMSE and the width of 95% approximate confidence intervals of the MLEs for the parameters of two-parameter Rayleigh distribution under progressive type-II censoring by the EM and NR algorithms using scheme 3.

Table 6. The RMSE and the width of 95% approximate confidence intervals of the MLEs for the parameters of two-parameter Rayleigh distribution under progressive type-II censoring by the EM and NR algorithms using scheme 4.

Table 7. The RMSE and the width of 95% approximate confidence intervals of the MLEs for the parameters of two-parameter Rayleigh distribution under progressive type II censoring by the EM and NR algorithms with fixed number of failures when .

 

A Summary of Results from Tables 3-7 is Provided Below

i.   The MLEs realized using the EM algorithm have lower levels of RMSE compared to those obtained by the NR method in nearly all combinations of simulation conditions.

ii.   The widths of 95% approximated confidence intervals of parameters and  obtained using the EM algorithm tends to be lesser compared to those obtained by NR method in nearly all combinations of simulation conditions. According to Gulhar et al., a smaller width is better because it captures the true parameter value (CV) within a small span and the results are more accurate and precise [13].

iii.   For a fixed sample size n (e.g. n=30), we noted that as the number of failures (m) increases (i.e. from 20 to 25), the RMSE and widths of confidence intervals of MLEs obtained using both the EM and NR algorithms decreases. For RMSE and widths of the confidence intervals (compare Table 3 and 4, and Table 5 and 6). This implies that the performance of MLEs becomes better.

iv.   When the number of failures m is fixed, we observed that as the sample size n increases the RMSE, and the widths of 95% approximate confidence intervals of MLEs obtained using both the EM and NR algorithms decreases (see Table 7). This indicates that the MLEs are consistent in nature.

v.   When the value of  is fixed, we noted that as the value of  increases, the RMSE for all the estimates increases, which indicates the consistency of the estimators.

vi.   Additionally, if m=0 there is no censoring, under this condition zero samples are generated hence it is not possible to obtain the corresponding MLEs. On the other hand, if , then n=m (complete sample situation), under this condition estimates are extremely biased.

4. Example Using Real-Life Data

Now consider a real-life data set to illustrate how maximum likelihood estimation using the NR method and the EM algorithm for the two-parameter Rayleigh distribution works in practice. We have utilized progressive type-II censoring to analyze a real data representing the survival times (in years) of 46 patients given chemotherapy treatment as discussed in [5]. The discussion indicated that the Rayleigh Distribution is acceptance for this data set (provides a good fit). The data set is given as follows:

0.047, 0.115, 0.121, 0.132, 0.164, 0.197, 0.203, 0.260, 0.282, 0.296, 0.334, 0.395, 0.458, 0.466, 0.501,

0.507, 0.529, 0.534, 0.540, 0.570, 0.641, 0.644, 0.696, 0.841, 0.863, 1.099, 1.219, 1.271, 1.326, 1.447, 1.485, 1.553, 1.581, 1.589, 2.178, 2.343, 2.416, 2.444, 2.825, 2.830, 3.578, 3.658, 3.743, 3.978, 4.003, 4.033

From the above data, progressive type-II censored samples were generated with m=20, 30, and 40 as follows:

Table 8. Censoring scheme  and progressive type-II censored samples for different values of m using real-life data sets.

Table 9. The MLEs and the corresponding widths of 95% approximate confidence intervals for the unknown parameters of TR distribution for different values of m.

From the table above, it is observed that:

i.    The MLEs obtained using the EM algorithm have narrower widths of confidence intervals compared to those obtained using NR method except for  when m=40. A smaller width is better because it captures the true parameter value (CV) within a small span and the results are more accurate and precise.

ii.   For both methods, the MLEs and the width of 95% approximate confidence intervals decrease as the number of failures increases (i.e., from 20 to 40) for nearly all the values of m, which indicates the consistency of the estimators.

5. Conclusions

In this study, the problem of estimation of the MLEs for the parameters of the two-parameter Rayleigh distribution based on generated progressive type-II censored samples was addressed. In particular, the MLEs were derived using the NR and the EM algorithms. Approximate asymptotic variances of the MLEs were also derived and used to construct approximate confidence intervals of the parameters.

The simulation results clearly show that the MLEs obtained using the EM algorithm have lower levels of RMSE and narrower widths of the corresponding confidence intervals compared to those obtained using the NR algorithm. However, the NR method may yield better estimates especially when  is greater than 70%. This shows that both the EM and NR algorithms can be used in estimation problem, but we can conclude that the EM algorithm is highly recommended as it provides better estimates. Al-Zahrani and Gindwan [1] and Helu, Samawi, & Raqab [14] obtained similar simulation results.

References

1. Al-Zahrani, Bander M., and Maha S. Gindwan. "Estimating the parameter of the Lindley distribution under progressive type-II censored data." Electronic Journal of Applied Statistical Analysis 8.1 (2015): 100-111.
2. Balakrishnan, Narayanaswamy, and Rita Aggarwala. Progressive Censoring: theory, methods, and applications. Springer Science & Business Media, 2000.
3. Balakrishnan, N., and R. A. Sandhu. "A simple simulational algorithm for generating progressive Type-II censored samples." The American Statistician 49.2 (1995): 229-230.
4. Balakrishnan, N. "Progressive censoring methodology: an appraisal." Test 16.2 (2007): 211-259.
5. Bekker, A., J. J. J. Roux, and P. J. Mosteit. "A generalization of the compound Rayleigh distribution: using a Bayesian method on cancer survival times." Communications in Statistics-Theory and Methods 29.7 (2000): 1419-1433.
6. Cox, D. R., and D. V. Hinkley. "TheoreticalStatistics Chapman and Hall, London." See Also (1974).
7. Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin. "Maximum likelihood from incomplete data via the EM algorithm." Journal of the Royal Statistical Society. Series B (Methodological) (1977): 1-38.
8. Devore, Jay L., and Kenneth N. Berk. Modern mathematical statistics with applications. Cengage Learning, 2007.
9. Dey, Sanku. "Comparison of Bayes Estimators of the parameter and reliability function for Rayleigh distribution under different loss functions." Malaysian Journal of Mathematical Sciences 3 (2009): 247-264.
10. Dey, Sanku, and M. K. Das. "A note on prediction interval for a Rayleigh distribution: Bayesian approach." American Journal of Mathematical and Management Sciences 27.1-2 (2007): 43-48.
11. Dey, Sanku, Tanujit Dey, and Debasis Kundu. "Two-parameter Rayleigh distribution: different methods of estimation." American Journal of Mathematical and Management Sciences 33.1 (2014): 55-74.
12. Dey, Tanujit, Sanku Dey, and Debasis Kundu. "On Progressively Type-II Censored Two-parameter Rayleigh Distribution." Communications in Statistics-Simulation and Computation 45.2 (2016): 438-455.
13. Gulhar, Monika, et al. "A comparison of some confidence intervals for estimating the population coefficient of variation: a simulation study." Sort: Statistics and Operations Research Transactions 36.1 (2012): 45-68.
14. Helu, Amal, Hani Samawi, and Mohammad Z. Raqab. "Estimation on Lomax progressive censoring using the EM algorithm." Journal of Statistical Computation and Simulation 85.5 (2015): 1035-1052.
15. Howlader, H. A., and A. Hossain. "On Bayesian estimation and prediction from Rayleigh based on type II censored data." Communications in Statistics-Theory and Methods 24.9 (1995): 2251-2259.
16. Johnson, Norman L., Samuel Kotz, and N. Balakrishnan. Continuous multivariate distributions, volume 1, models and applications. Vol. 59. New York: John Wiley & Sons, 2002.
17. Khan, Hafiz MR, Serge B. Provost, and Ashima Singh. "Predictive inference from a two-parameter Rayleigh life model given a doubly censored sample." Communications in Statistics-Theory, and Methods 39.7 (2010): 1237-1246.
18. Kundu, Debasis, and Arabin Kumar Dey. "Estimating the parameters of the Marshall–Olkin bivariate Weibull distribution by EM algorithm." Computational Statistics & Data Analysis 53.4 (2009): 956-965.
19. Lio, Y. L., Ding-Geng Chen, and Tzong-Ru Tsai. "Parameter Estimations for Generalized Rayleigh Distribution under Progressively Type-I IntervalCensored Data." Open Journal of Statistics 1.02 (2011): 46.
20. Top of FormMcLachlan, Geoffrey J, and Thriyambakam Krishnan. The Em Algorithm and Extensions. New York: J. Wiley, 1997. Print.
21. Bottom of Form Lee, Jason, Yuekai Sun, and Michael Saunders. "Proximal Newton-type methods for convex optimization." Advances in Neural Information Processing Systems. 2012.
22. Ng, H. K. T., P. S. Chan, and N. Balakrishnan. "Estimation of parameters from progressively censored data using EM algorithm." Computational Statistics & Data Analysis 39.4 (2002): 371-386.
23. Ortega, James M., and Werner C. Rheinboldt. Iterative solution of nonlinear equations in several variables. Vol. 30. Siam, 1970.
24. Pradhan, Biswabrata, and Debasis Kundu. "Bayes estimation and prediction of the two-parameter gamma distribution." Journal of Statistical Computation and Simulation 81.9 (2011): 1187-1198.
25. Rayleigh, Lord. "On the capillary phenomena of jets." Proc. R. Soc. London. Vol. 29. No. 196-199. 1879.
26. Surles, J. G., and W. J. Padgett. "Inference for P (Y< X) in the Burr type X model." Communications in Statistics-Theory and Methods 7 (1998): 225-238.
27. Seo, Eun-Hyung, and Suk-Bok Kang. "AMLEs for Rayleigh distribution based on progressive Type-II censored data." Communications for Statistical Applications and Methods 14.2 (2007): 329-344.
28. Smith, Richard L. "Maximum likelihood estimation in a class of nonregular cases." Biometrika 72.1 (1985): 67-90.
29. Watanabe, Michiko, and Kazunori Yamaguchi, eds. The EM algorithm and related statistical models. CRC Press, 2003.