Constrained Blind Separation Algorithm Using Variable Step Size and Variable Momentum Factor

: The traditional blind separation algorithm is mainly for the instantaneous mixing problem in the stable environment. In the practical applications, blind separation often takes into account the interference of the external environment, which requires that the algorithm has strong tracking performance, but the traditional algorithm can’t meet the needs. Aiming at the problem of instantaneous blind separation in non-stationary environment, constrained blind separation algorithm using variable step size and variable momentum factor is proposed in this paper. Based on the nonholonomic natural gradient algorithm, the cost function is constrained by the disturbance of the hybrid system and the constraint factors take the form of self-adaptive adjustment. According to the separation situation, the constraint factors are adjusted adaptively to accelerate the convergence speed. The variable step size based on the cost function gradient is introduced to improve the tracking performance. By incorporating momentum term, the momentum factor is adaptively adjusted to make it have better separation performance. The simulation results show that compared with the traditional algorithm, the proposed algorithm can better balance the contradiction between convergence speed and steady-state error in non-stationary environment, and has better separation performance. In the case of obvious disturbance in the mixed system, the algorithm can effectively improve the shortcomings of the traditional algorithm. In summary, constrained blind separation algorithm using variable step size and variable momentum factor proposed in this paper is effective


Introduction
Blind source separation (BSS) refers to the recovery of source signals from a set of observed signals only according to the statistical independence of each source signal when the source signal and the mixed system are unknown. The "blind" means that the source signal cannot be directly observed and the hybrid system is unknown. In recent years, blind separation technology has made continuous progress and has been widely used in biomedical signal processing, wireless communication, speech recognition, feature extraction and data compression [1,2]. Blind separation has uncertainty, which is reflected in the difference in magnitude and order of the source signals recovered by the separation matrix. In fact, it is multiplying the arbitrary row vector of the separation matrix by a fixed scale factor or exchanging the position of any row vector, but it will not affect blind separation in essence. In the non-stationary environment, blind separation can be divided into three conditions: 1) the source signal is non-stationary when the mixed system is invariable; 2) the source signal is stationary when the mixed system is time-varying; 3) the source signal is non-stationary when the mixed system is time-varying. This paper mainly studies the second conditions, that is, the mixed system is time-varying. At this time, the external environment is non-stationary due to the existence of system disturbance.
In practical applications, blind separation often takes into account the interference of the external environment, which requires that the algorithm has strong tracking performance, while the traditional algorithm can't meet the requirements. Reference [3] aiming at the slow convergence of classical natural gradient algorithm, a natural gradient blind separation algorithm for optimal search direction is proposed. Reference [4] introduced reference separation system, analyzed crosstalk error, and proposed a new variable step-size method based on blind source separation reference separation system. Reference [5] increases the variable step size of the cost function gradient, and proposed the nonholonomic natural gradient algorithm based on sign operator. Reference [6] aiming at the difficulty in selecting nonlinear functions, and proposed an algorithm for selecting the appropriate activation function adaptively by kurtosis. Reference [7] established a new separation measurement index, constructed a nonlinear monotone function, adaptive adjustment of the step size and momentum factor, and proposed an adaptive blind separation method of variable step size momentum term. Reference [8] based on the separation performance index, the approximate optimal parameters are designed, and proposed the step-size adaptive blind source separation algorithm with adding momentum term. Reference [9,10], a variable step size blind source separation algorithm with adaptive momentum factor was proposed by using the construction function of the performance evaluation index of the separation signal. Reference [11] proposed an adaptive variable step size algorithm in non-stationary environment by using the disturbance of mixed system, constraint cost function, adaptive constraint factor and step size.
The above literature uses different methods to improve the algorithm, which has a good separation effect to some extent, but the contradiction between convergence speed and steady-state error hasn't been well improved. Aiming at the shortcomings of the above algorithms, constrained blind separation algorithm using variable step size and momentum factor is proposed, which is suitable for non-stationary environments. Firstly, based on the nonholonomic natural gradient algorithm, introduce gate function and constraint cost function. According to the separation situation, the constraint factors are adjusted adaptively. Secondly the variable step size based on the cost function gradient is introduced to improve the tracking performance. Thirdly the momentum term is integrated, and the momentum factor takes an adaptive form, thereby accelerating the convergence speed and reducing the steady-state error. In the case of obvious disturbance in the mixed system, the algorithm can effectively improve the shortcomings of the traditional algorithm. Finally, the simulation results show that the algorithm is effective.

Constrained Nonholonomic Natural
Gradient Algorithms in Non-stationary Environment

Blind Source Separation and Nonholonomic Natural Gradient Algorithm
Without considering the noise interference, the instantaneous mixing model of blind separation can be presented as: where Α represents the N×N row-full-rank mixed matrix, 1 2 x (  After separating the separation matrix, the estimated source signal is: is the estimation signal vector of the N-dimensional source signal, W is the separation matrix, and G is the N-dimensional global matrix.
The natural gradient algorithm [12] can find the optimal separation matrix W ept by successively correcting W , namely: where ( ) k µ is the step size, its affects the convergence speed and stability of the algorithm. When the value is large, the convergence speed is fast, but the stability is poor; when the value is small, the convergence speed is slow, but the stability is good. (y ) represents a non-linear activation function，define:  [13,14] is obtained by combining equation (3), namely:

Constraint Algorithm in Non-stationary Environment
In the non-stationary environment, when the disturbance on the mixed system changes slightly, the traditional natural gradient algorithm cannot produce an effective response. Here, nonlinear programming techniques are used to introduce gate functions to solve optimization problems and ensure the robustness of mixed systems under disturbance [15]. When the condition that (W) 0 g ≥ and (W) J reach the minimum value is satisfied, Solving optimization problems, that is, Solving the minimum of the augmented cost function. The augmented cost function is: is the logarithmic gate function, used to prevent feasible solutions from leaving the region determined by inequality constraints. (W) R is the constraint factor, used to control the gate function. The setting of constraint conditions is obtained by assuming the initial separation matrix and the disturbance added to the mixed matrix. Assuming 0 A is the mixed system in the stationary environment (the mixed matrix without disturbance), D is the disturbance on the mixed system. Define the disturbance coefficient , according to the definition of matrix norm: Since 0 A D + stands for real-time mixed matrix,  (7) can be written as follows: Squared on both sides: Inequality constraint is expressed as: From this, the logarithmic gate function can be obtained: The augmented cost function can be expressed as ( ) Can be obtained: Therefore, the iteration formula of the nonholonomic natural gradient algorithm (C-NNG) based on the constrained cost function is as follows:

Adaptive Constraint Factor
The selection of the constraint factor will have a certain impact on the algorithm. When the value of constraint factor is large, the convergence is fast but the stability is poor. In order to improve the shortcomings of the algorithm when the constraint factor is constant value, the constant value constraint factor in the algorithm is adjusted to the adaptive constraint factor, and the updated iteration equation of the adaptive constraint factor can be written as: where ρ is a small constant.
According to the matrix inner product operation, the gradient term can be expressed as: tr is the trace of the matrix. Equation (17) can also be represents as: Therefore, the iterative equation of the nonholonomic natural gradient algorithm (AC-NNG) based on the adaptive constraint factor is as follows:

Adaptive Variable Step Size
The step size has a great influence on the convergence speed and stability of the algorithm，fixed step size limits algorithm performance. When the step size is large, the convergence is fast but the stability is poor. When the value is small, the stability is good but the convergence is slow. In order to improve the shortcomings of the algorithm when the step size is fixed, the fixed step size is adjusted to adaptive variable step size, and the updated iteration formula of adaptive variable step size is as follows: where λ is a small constant.
According to the matrix inner product operation, the gradient term can be expressed as: (21) can also be represents as: Therefore, the iteration formula of variable step size nonholonomic natural gradient algorithm (ACV-NNG) based on the adaptive constraint factor is as follows:

Adaptive Momentum Factor
Compared with the AC-NNG algorithm, the convergence speed of the ACV-NNG algorithm is improved, but the steady-state error is not significantly improved. Aiming at this phenomenon, in order to further improve the convergence speed and reduce the steady-state error, this paper incorporated the momentum term, when the momentum term is incorporated and the momentum factor is a constant value, the iterative formula of the algorithm (ACVM-NNG-fixed) is: is the momentum term, and β represents the momentum factor. The above algorithm improves the convergence speed, but the steady-state error has not been significantly improved. In order to further improve the convergence speed and reduce the steady-state error, and better balance the contradiction between the convergence speed and the steady-state error. In this paper, an adaptive form of the momentum factor is adopted, the update iteration formula of the adaptive momentum factor is: where σ is a small constant.
According to the matrix inner product operation, the gradient term can be expressed as: Formula (24) can also be written as: Therefore, the iteration formula of variable step size momentum factor nonholonomic natural gradient algorithm based on adaptive constraint factor (ACVM-NNG) is as follows: Compared with the ACV-NNG algorithm, ACVM-NNG incorporates the momentum term, and the momentum factor can be adaptively adjusted according to the needs, which accelerates the convergence speed of the algorithm, reduces the steady-state error, and has better separation performance. The flow chart of the ACVM-NNG algorithm is shown in Figure 1.

Simulation Experiment
In To compare the separation performance of these six algorithms, select PI [16] as the performance evaluation index: the smaller the value, the better the separation effect, and vice versa.
In the first case: at each sampling point, disturbances occur in the mixed system.
Set the time-varying mixed matrix:    It can be seen from Figure 3 that the source signals estimated by the newly proposed ACVM-NNG-Fixed algorithm and the ACVM-NNG algorithm can reach the convergence state with fewer iterations, the ACVM-NNG algorithm has faster convergence speed. It can be seen more clearly from figure 4 that, compared with the first four algorithms, the two newly proposed algorithms have faster convergence speed and less steady-state error, which can better balance the contradiction between the convergence speed and steady-state error. The ACVM-NNG algorithm has better effect.
In the second case: When the number of sampling points is 4000, there is a sudden change in the hybrid system.
Before the mutation occurs, set the initial mixed matrix: After the mutation occurs, set the mixed matrix: 1 1 A A 0.01( ( (A ))) rand size = + (34) 1 1.19 0.6 A 0.5 0.1 the momentum factor in ACVM-NNG-fixed algorithm 2 0.5 β = . Figure 5 is the simulation result of the separation performance curve of the six algorithms, and the number of iterations is from 1-8000. As can be seen from Figure 5, in the case of mutation, the convergence speed of the two newly algorithms is faster than that the first four algorithms. The ACVM-NNG algorithm has faster convergence speed than the ACVM-NNG-Fixed algorithm, and the steady-state error is smaller, which can better balance the contradiction between convergence speed and steady-state error. In the case of mutation, the separation effect is better than that of the first case before the number of iterations is 4000, because there is no system disturbance before the mutations occurs, while in the first case, there are system disturbances at every sampling point. Comparing Figure 4 and Figure 5, the newly proposed ACVM-NNG algorithm has better effects in the presence or absence of mutation, It improves the convergence speed while reducing the steady-state error and has better separation performance. It can be seen that the variable step size momentum factor nonholonomic natural gradient algorithm based on adaptive constraint factor proposed in this paper is effective.

Conclusion
This paper mainly studies the mixed system with time-varying and stable source signals. On the basis of the nonholonomic natural gradient algorithm, considering the system disturbance, a new improved algorithm is proposed. The algorithm adopts a method of adaptively constraining factor, step size and momentum factor to balance the relationship between convergence speed and steady state error. The simulation results show that the algorithm has better separation effect, and can improve the convergence speed and reduce the steady-state error.