Stability of a regularized Newton method with two potentials

In a Hilbert space setting, we study the stability properties of the regularized continuous Newton method with two potentials, which aims at solving inclusions governed by structured monotone operators. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we can take the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton's methods.


Introduction
Throughout this paper, H is a real Hilbert space with scalar product ., . and norm · . As a guideline of our study, we use the Newton-like dynamic approach to solving monotone inclusions which was introduced in [7]. To adapt it to structured monotone inclusions and splitting methods, this study was developed in [7], [6] , [2] and [5], where the operator is the sum of the subdifferential of a convex lower semicontinuous function, and the gradient of a convex differentiable function. We wish to extend this study to a non potential case, and so enlarge its range of applications. More precisely, we are going to consider some discrete and continuous Newton-like dynamics, which aim at solving structured monotone inclusions of the following type where ∂Φ is the subdifferential of a convex lower semicontinuous function Φ : H → R ∪ {+∞}, and B is a monotone cocoercive operator. Recall that a monotone operator B : H → H is cocoercive if there exists a constant β > 0 such that for all x, y ∈ H Bx − By, x − y ≥ β Bx − By 2 .
The abstract formulation (1) covers a large variety of problems, see for example [1], [2], [5], [6], [8], [10], [13]. It is directly connected to two important areas, namely convex optimization (take B = 0), and the theory of fixed point for nonexpansive mappings (take Φ = 0, and B = I − T with T a nonexpansive mapping). By a classical result, the two operators ∂Φ and B are maximal monotone, as well as their sum A = ∂Φ + B. We are going to exploit the structure of the maximal monotone operator A in order to develop first continuous dynamics, and then, by time discretization, splitting forward-backward algorithms that aim to solve (1). Precisely, our analysis relies on the convergence properties (as t → +∞) of the orbits of the system υ (t) ∈ ∂Φ (x (t)) (2) λẋ (t) +υ (t) + υ (t) + B (x (t)) = 0. ( In (3), λ is a positive constant which acts as a Levenberg-Marquard regularization parameter. We make the following standing assumptions: ϕ and ψ are two functions which act on H and satisfy • ϕ : H −→ R ∪ {+∞} is convex lower semicontinuous, and proper; • ψ : H −→ R is convex differentiable, and ∇ψ is Lipschitz continuous on the bounded subsets of H.
• ϕ + ψ is bounded from below on H.
We are concerned with the study of Newton-like continuous and discrete dynamics attached to solving the the structured minimization problem Note the asymmetry between ϕ, which may be nonsmooth, with extended real values, and ψ which is continuously differentiable, whence the structured property of the above problem. Indeed, we wish to design continuous and discrete dynamics which exploit this particular structure and involve ϕ via implicit operations (like resolvent or proximal operators) and ψ via explicit operations (typically gradient-like methods). So doing we expect obtening new forward-backward splitting methods involving Riemannian metric aspects, and which are close to the Newton method. This approach has been delineated in a series of recent papers, [1], [3], [2], [6], [7]. In this paper we are concerned with the stability properties with respect to the data (λ, x 0 , υ 0 , ...) of the strong solutions of the differential inclusion Let us now make our standing assumption on function λ(·): is continuous, and absolutely continuous on each interval Henceλ(t) exists for almost every t > 0, andλ(·) is Lebesgue integrable on each bounded interval [0, b]. We stress the fact that we assume λ(t) > 0, for any t ≥ 0. By continuity of λ(·), this implies that, for any b > 0, there exists some positive finite lower and upper bounds for λ(·) on [0, b], i.e., for any t ∈ [0, b] Our main interest is to allow λ(t) to go to zero as t → +∞. This makes the corresponding Levenberg-Marquardt regularization method asymptotically close to the Newton's method.
Let us summarize the results obtained in [2], [12] . Under the above assumptions, for any Cauchy data x 0 ∈ dom∂ϕ and υ 0 ∈ ∂ϕ(x 0 ), there exists a unique strong global solution (x (·) , υ (·)) : [0, +∞[→ H × H of the Cauchy problem (4)- (6). Assuming that the solution set is nonempty, if λ (t) tends to zero not too fast, as t −→ +∞, then υ (t) −→ 0 strongly, and x (t) converges weakly to some equilibrium which is a solution of the minimization problem (P). By Minty representation of ∂ϕ, the solution pair (x (·) , υ (·)) of (4)-(6) can be represented as follows: set µ(t) = 1 λ(t) , then for any t ∈ [0, +∞), where z (·) : [0, +∞[→ H is the unique strong global solution of the Cauchy probleṁ = 0 Let us recall that prox µϕ is the proximal mapping associated to µϕ. Equivalently, prox µϕ = (I + µ∂φ) −1 is the resolvent of index µ > 0 of the maximal monotone operator ∂ϕ, and ∇ϕ µ is its Yosida approximation of index µ > 0. Let us stress the fact that, for each t > 0, the operators prox µ(t)ϕ : H −→ H, ∇ϕ µ(t) : H −→ H are everywhere defined and Lipschitz continuous, which makes this system relevant to the Cauchy-Lipschitz theorem in the nonautonomous case, which naturally suggests good stability results of the solution of (4)-(6) with respect to the data. This paper is organized as follows: We first establish a priori energy estimates on the trajectories. Then we consider the case where λ is locally absolutely continuous. Note that it is important, for numerical reasons, to study the stability of the solution with respect to perturbations of the data, and in particular of λ which plays a crucial role in the regularization process. In Theorem 3.1 we prove the Lipschitz continuous dependence of the solution with respect to λ. Moreover, the Lipschitz constant only depends on the L 1 norm of the time derivative of λ. Finally, we extend our analysis to the case where λ is a function with bounded variation (possibly involving jumps). We use a regularization by convolution method in order to reduce to the smooth case, and then pass to the limit in the equations. So doing, in Theorem 4.1 and Corollary 4.1, we prove the existence and uniqueness of a strong solution for (4)- (6), in the case where λ is a function with bounded variation.

A Priori Estimates
The linear space H ×H is equipped with its usual Hilbertian norm (ξ, ζ) = ξ 2 + ζ 2 . In this section we work on a fixed bounded interval [0, T ], and following assumption (8), we suppose that there exists some positive constant c 0 such that We will also assume that ∇ψ is L ψ -Lipschitz continuous. Indeed, this is not a restrictive assumption since one can reduce the study to trajectories belonging to a fixed ball in H. We will often omit the time variable t and write x,υ.... for x (t), υ (t).... when no ambiguity arises.
Since ∂ϕ : H ⇒ H is monotone Dividing by h 2 and passing to the limit preserves the inequality, which yields By taking the inner product of both sides of (5) byẋ(t) we obtain Noticing that x and v are continuous on [0, T ], hence bounded, and that λ is bounded from below on [0, T ] by a positive number, one easily gets from (18) thaṫ For our stability analysis, we now establish a precise estimate of the L 2 norm ofẋ. By the classical derivation chain We appeal to a similar formula which is still valid for a convex lower semicontinuous proper function ϕ : ii) v is continuous on [0, T ], and hence belongs to L 2 (0, T ; H); iii)ẋ ∈ L 2 (0, T ; H) by (19).
Combining (18) with (20) and (21) we obtain By integrating the above inequality from 0 to T we obtain Since ϕ+ψ is bounded from below on H, and λ is minorized by the positive constant c 0 on [0, T ] (see (14)), we infer is absolutely continuous on bounded sets, for any t ∈ [0, T ] Passing to the norm, and using Cauchy-Schwarz inequality yields Combining the above inequality with (24) gives This being true for any t ∈ [0, T ] Let us now exploit another a priori energy estimate.
Proof. By taking the scalar product of (5) byv(t) we obtain Hence By integrating the above inequality we deduce that, for any t Since ∇ψ is L ψ -Lipschitz continuous A careful look at the proof of (28) gives the more precise estimate Combining (34) with (37) gives As a consequence and which ends the proof.
We can now deduce from the two preceding propositions an a priori bound on the L ∞ norm ofẋ andv.
Proof. a) Let us return to the equation obtained by taking the inner product of both sides of (5) byẋ(t) By (17), and λ is minorized by the positive constant c 0 on [0, T ],we infer Hence By (36) and (40) we deduce that b) Let us return to the equation obtained by taking the inner product of both sides of (5) byv(t) A similar argument as above yields v(t) ≤ v(t) + ∇ψ(x(t)) from which we deduce the result.
Let us enunciate some straight consequences of Proposition 2.3. Corollary 2.1. The following properties hold: for any 0 < T < +∞

Stability Results
In the next theorem we analyze the Lipschitz continuous dependence of the solution (x, υ) of the Cauchy problem (4)-(6) with respect to the function λ and the initial point (x 0 , υ 0 ). Our demonstration is based on that followed in [6], [9], [14].
Remark 3.2. Another approach is to study the equivalent problem (13)- (13), based on the known stability results for the Cauchy-Lipschitz problem. Although conceptually simple, this approach seems more technical.

Bounded Variation Regularization
Coefficient λ (·) the supremum being taken over all p ∈ N and all strictly increasing sequences τ 0 < τ 1 < · · · < τ p of points of [0, T ]. Function λ may involve jumps. We also suppose that λ is bounded away from 0: The following lemmas which are proved in [6] gather some classical facts concerning the approximation of functions of bounded variation by smooth functions together with some technical results useful for sequel. In particular, . Lemma 4.2. Let z n , z ∈ C ([0, T ] , H) be such that z n −→ z uniformly and (z n ) n is L-Lipschitz continuous for some positive constant L independent of n ∈ N. Let λ n −→ λ in L 2 (0, T ) . Then λ nżn converges weakly to λż in L 2 (0, T ; H) .
We can now state the main result of this section.

Conclusion
In a Hilbert space setting, we studeid the stability properties of the regularized continuous Newton method with two potentials, which aims at solving inclusions governed by structured monotone operators. The Levenberg-Marquardt regularization term acts in an open loop way. As a byproduct of our study, we took the regularization coefficient of bounded variation. These stability results are directly related to the study of numerical algorithms that combine forward-backward and Newton's methods Let us list some interesting questions to be examined in the future: 1. Study the asymptotic stability properties, corresponding to the case where T = +∞, in connection with the convergence results of [2] and [7]. 2. Extend the results to the case where the Levenberg-Marquart regularization term is given in a closed-loop form, λ(t) = α( ẋ(t) ) as in [6]. 3. Study the same questions for the corresponding forwardbackward algorithms, see [1].