A Review of Change Point Estimation Methods for Process Monitoring

When one or more observations fall outside the control limits, the chart signals the existence of a change in the process. Change point detection is helpful in modelling and prediction of time series and is found in broader areas of applications including process monitoring. Three approaches were proposed for estimating change point in process for the different types of changes in the literature. they are: Maximum Likelihood Estimator (MLE), the Cumulative Sum (CUSUM), and the Exponentially Weighted Moving Average (EWMA) approaches. This paper gives a synopsis of change point estimation, specifies, categorizes, and evaluates many of the methods that have been recommended for detecting change points in process monitoring. The change points articles in the literature were categorized broadly under five categories, namely: types of process, types of data, types of change, types of phase and methods of estimation. Aside the five broad categories, we also included the parameter involved. Furthermore, the use of control charts and other monitoring tools used to detect abrupt changes in processes were reviewed and the gaps for process monitoring/controlling were examined. A combination of different methods of estimation will be a valuable approach to finding the best estimates of change point models. Further research studies would include assessing the sensitivity of the various change point estimators to deviations in the underlying distributional assumptions.


Introduction
Statistical process control (SPC) is designed for decreasing variation in quality process which leads to increase processes performance. There are two types of variation: they are common causes and assignable causes of variation. The purpose of control charts is to monitor and as well detect assignable causes for nonconforming products that are produced. When one or more observations fall outside the control limits, the chart signals the existence of a change in the process (Pignatiello and Simpson [42]). When a change is observed, it is compulsory for the user to search for the cause of the signal. Amiri and Allahyari [4] reviewed change point estimation approached for control charts post signal diagnosis. It was discovered that the work that the signal does not correspond to the exact time of the change in the process because of a delay in signaling the change by the control chart. In this paper, we extend the review of Amiri and Allahyari [4] to recent research in change point estimation up till the year 2019. It is our believe that the review will serve as a platform for future work on change point estimation.

The Change Point Model
The change point model focuses on procedures of finding a point in time where the process parameters changed because of assignable cause of variation. Let be a quality characteristic under investigation assumed to be from a normal distribution. Then, ~ ( , ), i = 1¸ 2¸⋯¸ and ~ ( , ), i = +1¸ + 2¸⋯¸T. This implies that the process is normal with mean μ , and variance for the change point . A situation where the change point, , which has one or more parameters of the process (mean and/or variance) has changed to mean and variance . The main task is to evaluate the change point . In the literature, three approaches were proposed for estimating change point in process for the different types of changes. They are: Maximum Likelihood Estimator (MLE), the Cumulative Sum (CUSUM), and the Exponentially Weighted Moving Average (EWMA) approaches Pignatiello and Samuel [40]. In MLE, the change point is the time in which the likelihood function is maximized. According to Amiri & Allahyari [4], the estimated change point is: where L(t) is the likelihood function including both incontrol and out-of-control observations, t is the index for the range of possible values for the process change point, and ̂ is the maximum likelihood change point estimator.

Review of Change Point Estimation
This section examines some key considerations that remain unaddressed in change point estimation methods for process monitoring and also note some possible future research directions. The following categories are considered in this work: (i) Process types (ii) Data types (iii) Change types (iv) Phase types (v) Estimation Methods.

Process Types
The process types in change point estimation method for process monitoring refer to the procedures or operations involved in creating an output (desired outcome) from an input. According to Amiri and Allahyari [4], these processes are categorized into six groups, namely: ordinary process, multistage process, process with profile quality characteristics, autocorrelated process, high yield process, and specific process.
(i) Ordinary process: This is the simplest of procedures in achieving a task, and it's the most used in literature as regards change point estimation. (ii) Multistage process: The multistage process involves two or more levels (multiple operation stages), and these may have different conditions. This is a stage where a production line component parts involve different operations before the final stage. At every stage, there exist monitoring of quality characteristics base on operation sequence. At this stage, the outputs at every stage are affected by the operations of previous stages. There may be variations introduced due to transferring of product from one stage to another. (iii) Process with profile quality characteristics: This is a relationship in the quality of a product that are well characterized in performance between a response variable and one or more explanatory variables. (iv) Autocorrelated process: The measurements or observations over time are autocorelated and not independent from each other. (v) High yield process: A high yield process occurs when nonconforming items are observed and the fraction nonconforming is in the range of parts per million. (vi) Specific process: Specific process occurs when there are some special cases like short-run processes or linear processes with long memory.

Data Types
Quality characteristics are classified on the basis of the number and type of variable. Many quality characteristics can be represented in terms of numerical or non-numerical measurements. These data types are classified into five groups, namely: (i) variable (ii) attribute (iii) multivariate (iv) multi-attribute and (v) profile variable (See Table 1).
(i) Variable is a quality characteristic that can be measured on a numerical scale. (ii) Attribute is a quality characteristic that cannot be measured but can be classified or counted. It is synonymous with discrete data. (iii) Multivariate variables involves when the quality characteristics can be described by multiple correlated variables instead of a single variable. The main objective is to study how the variables are associated with one another, and how they work in sequence to differentiate between the cases on which the observations are made. (iv) Profile variable. The data type in a process with profile quality characteristics is called profile variable. A profile is linear relationship between the dependent and independent variable(s). (v) Multi-attribute. In this case, the attribute data contain multiple correlated attributes. The various authors that have worked on the data types are summarized in Table 1.

Change Types
The change type is another characteristic to be considered as regards change point estimation methods for process monitoring. Different change types have been examined in the literature, which has been categorized as single step change, multiple step changes, drift, and monotonic change point (See Table 1).
In a single step change, the unknown parameter changes as a result of time and remains at new level until the changes is detected and corrective action is taken. A change in the raw material supplied during production process resulted in shifts in the mean of the process Samuel et al . [46]. A change point method was also proposed by Mahmoud et al. [21] based on segmented regression technique to detect changes in a linear profile data set using the Likelihood Ratio Test (LRT) approach. Kamzemzadeh et al. [15] developed and investigated three methods for Phase I monitoring polynomial profiles performance.
Multiple step change changes occur severally at different times before the signal is given by the control chart. This type of changes occur due to one or more influential process input variable(s) at different times (Perry et al. [37]).
Drift change is a type of changes that the process drifts off target, either linearly or nonlinearly at unknown point in time. This trend continues until some corrective measure is taken to bring it in control (Perry and Pignatiello [34]).
Monotonic change also known as a priori. The direction of shifts is the same. i.e., the direction of shift is either increasing or decreasing. According to Perry et al. [37], this type of changes is more general because it includes all other changes.

Phase Types
Profile monitoring methods and applications in control charting can be divided into two phases, which are called Phase I and II (Woodall and Montgomery [54]).
In Phase I applications, a set of historical profile data is analyzed with the main goals to understand the variation in a process over time, which includes the study of separating the common variation within a profile from the variation between profiles (profile-to-profile variation). Furthermore, the interest in the Phase I operation is to evaluation of the process stability, which includes separating in-control profiles from out-of-control profiles (with assignable causes); and to Model the in-control process performance. This goal is achieved by estimating the parameters of a parametric model, if parametric approach is used. However, if one decides to use a nonparametric model, or if the data does not fit any parametric model, then one may end up with a nonparametric baseline profile obtained from all incontrol profiles.
The evaluation of Phase I methods is mainly focused on assessing the probability of signal studies, i.e., the probability of at least one out-of-control signal when applying the control chart to the historical profile dataset.
In Phase II applications, the interest is monitoring the process with the help of on-line data. The goal is to detect shifts in the process from the baseline profile obtained in Phase I as soon as possible. The evaluation of Phase II methods is mainly focused on the performances of the average run-length distribution. The average run length (ARL) helps in comparing the performance of competing control charts in Phase II. For instance, a good work on change point estimations using a phase I data set was carried out by Sullivan [51]. In retrospective analysis, where multiple shift or outliers are present, the clustering algorithm gives a computationally easiest way of detecting the presence of the shifts and/or outliers in any quantity. Zhu [62] opined that the use of clustering algorithm can be generalized to detect any small shifts in other out-ofcontrol.
Noorossana and Shadman [29] provided an estimator for change point estimator in phase II, a period in which a step change in the process non-conformity proportion in high yield processes occurs. At this stage the number of items can be modeled by a geometric distribution until the occurrence of the first non-conforming is detected.
In this paper, Phase I and II processes is proposed. This is based on Ghazanfari et al. [10] who proposed a clustering technique to estimate Shewhart control chart change points for control change estimator in both phase I and II.

Methods of Estimation
Previous methods were studies based on the MLE for estimating change point. Other methods of estimation, which include clustering, Artificial Neural Network, and heuristic algorithms have been applied to estimate the change point for step shift. Column 4 of Table 1 present the estimation approaches for the different types of shifts.
The Maximum Likelihood Estimation (MLE) is a method that determines values for the parameters of a model. The parameter values are found such that they maximize the likelihood that the process described by the model produced the data that were actually observed. In terms of the approach for change point estimation, MLE is the predominant approach in earlier articles. Similarly, the Artificial Neural Network is a computational model inspired by networks of biological neurons, wherein the neurons compute output values from inputs. It learns from its past experience and errors in a non-linear parallel processing manner. The learning is based on reinforcement (supervised) and unsupervised (no target) type. The unsupervised mimics the biological neuron pattern of learning. In recent years, according to Puri et al. [43] ANNs approach is more used for change point estimation.
The clustering method is the classification of patterns into groups (clusters). A typical pattern clustering involves: Definition of a pattern proximity, Clustering or grouping, Data abstraction and Assessment of outputs: (Jain and Dubes [13]).
SPC tools and change-point models have some characteristics that are suitable for clustering methods. They are: two possible clusters, pattern classification, proximity to out-of-control and a close relationship between the control of mean in SPC and the definition of between variation in clustering, Therefore, monitoring the process are now addressed via change point models, which is the direction of some recent works in Statistical process control. Thus, the review. Table 1 summarizes the reviewed articles by the various considerations discussed in Section 3.

The Gaps and Direction for Future Research Work
Based on the review presented in Section 3 and the summary in Table 1, the following gaps and direction for future research works were identified: (i) In the change types, single step change has been flooded by researchers due to its simplicity. However, other change types, which are more complex in nature appear to be more relevant in practice. For instance, more attentions are drawn to relevant types of change which seems to be the future area of research. Directions for continuing investigation with profile monitoring should include other change types of change point which are more complicated which include nonlinear profiles, nonparametric profiles, generalized linear model profiles, profiles for geometric specifications, and autocorrelated profiles. Moreover, change point estimators for phase II of multistage processes for different change types can be developed together with the change point estimator for variance. It was observed that most of the literature on SPC has focused on phase II control charting which require more work on phase I subject to the use of robust estimators which will increase the utilization of change point methods. These robust estimators include nonparametric change point estimators, which are flexible in distributional assumptions. Similarly, most authors have focused on single phase whereas there is a need to also conduct research on both phases. Zhou and Liu [59] presented nonparametric estimators for the mean. The studies related to profile monitoring which considered single step shift and with linear profiles. Recent studies have shown a combination of both phases most especially for a multivariate data type in order to give a more robust monitoring process.
(ii) Methods of estimation such as ANN, clustering, and heuristic algorithms have assisted in increasing precision of estimates. Additional modifications of these estimation approaches for the different types of shifts would be apt and add to knowledge. There are combinations of change types and control charts that have not been studied such as multiple change points for multistage processes as an extension to Zou et al. [60] (iii) The Process types is a key consideration in the classification of articles on SPC. Based on the reviewed articles, it was observed that majority of work on SPC focuses on the ordinary processes, while only few has considered more complex processes. Articles on processes with profile quality characteristics have started appearing in recent years. However, multistage, high yield, autocorrelated and special processes are promising areas for interesting research.
(iv) For the data type, most of the work that had been done focuses on variable data. A promising area for research will be the multi-attribute control charts and post-signal diagnostic where little work or none has been done.

Conclusion
The change points articles in the literature were categorized broadly under five categories, namely: types of process, types of data, types of change, types of phase and methods of estimation. Aside the five broad categories, we also included the parameter involved. Finally, we explored the gaps for the essence of possible future investigation. It is our believe that apart from the methods of estimations reviewed in this paper, a combination of different methods of estimation will be a valuable approach to finding the best estimates of change point models. Further research studies would include assessing the sensitivity of the various change point estimators to deviations in the underlying distributional assumptions. Performance of a change point estimator, which is developed for specific change type, can be evaluated under a variety of change types. Moreover, performance of change point estimator developed by Mahmoud et al. [21] for step shift in regression parameters under different change types such as drift or monotonic changes can also be evaluated.