Distributed Subgradient Algorithm for Multi-Agent Convex Optimization with Global Inequality and Equality Constraints

: In this paper, we present an improved subgradient algorithm for solving a general multi-agent convex optimization problem in a distributed way, where the agents are to jointly minimize a global objective function subject to a global inequality constraint, a global equality constraint and a global constraint set. The global objective function is a combination of local agent objective functions and the global constraint set is the intersection of each agent local constraint set. Our motivation comes from networking applications where dual and primal-dual subgradient methods have attracted much attention in the design of decentralized network protocols. Our main focus is on constrained problems where the local constraint sets are identical. Thus, we propose a distributed primal-dual subgradient algorithm, which is based on the description of the primal-dual optimal solutions as the saddle points of the penalty functions. We show that, the algorithm can be implemented over networks with changing topologies but satisfying a standard connectivity property, and allow the agents to asymptotically converge to optimal solution with optimal value of the optimization problem under the Slater’s condition


Introduction
In recent years, distributed optimization and control have developed rapidly, and have been welcomed in the fields of industry and national defense, including smart grid, sensor network, social network and information system (Cyber-Physical system). Distributed optimization problems of multi-agent systems appear different kinds of distributed processing issues such as distributed estimation, distributed motion planning, distributed resource allocation and distributed congestion control [1][2][3][4][5][6][7][8][9][10][11][12]. The main focus is to solve a distributed optimization problem where the global objective function is composed of a sum of local objective functions, each of which is only known by one agent. Distributed optimization problems were first studied systematically in [1] where the union of the graphs was assumed to be strongly connected among each time interval of a certain bounded length and the adjacency matrices were doubly stochastic. A distributed subgradient method was introduced to solve the distributed optimization and error bounds on the performance index functions were given. As a continuation of [1], a distributed subgradient projection algorithm was developed in [2] for distributed optimization where each agent was constrained to remain in a closed convex set and the paper gave corresponding convergence analysis on identical closed convex sets and on fully connected graphs with non-identical closed convex sets. Inspired by the works of [1,2], the algorithms proposed in [1] and [2] were studied in the random environment [3] and [4], where the agents had the same state constraint. In [5], the communication topology was undirected and each possible communication link was functioning with a given probability. Thus, the expected communication topology is essentially fixed and undirected. Different from [1]- [5], a dual averaging subgradient algorithm was developed and analyzed for randomized graphs under the assumption that all agents remain in the same closed convex set in [6] and it was shown that the number of iterations were required by their algorithm scales inversely in the spectral gap of the network. Moreover, distributed optimization problems with asynchronous step-sizes or inequality-equality constraints or using other algorithms were studied in [7]- [12] and corresponding conditions were given to ensure the system converge to the optimal point or its neighborhood. However, as in [1]- [5], it was assumed in [6]- [12] that the state sets of agents to be identical or the objective function finally converge to only a neighborhood of the optimal set.
In this paper our work is to extend [14] to study the penalty primal-dual subgradient projection algorithm in a more general method. In [14], the authors solved a multiagent convex optimization problem where the agents subject to a global inequality constraint, a global equality constrain and a global constraint set. In order to solved these constraints, the author in [14] presented two different distributed projection algorithms with three assumptions that the union of the graphs is assumed to be strongly connected among each time interval of a certain bounded length and the adjacency matrices were doubly stochastic and nondegeneracy. However, [14] guaranteed the edge weight matrices of graphs were doubly stochastic (i.e., ∑ for all j V ∈ and 0 k ≥ ). Previous work did not perform well on the application of the distributed algorithms in multiagent network.
Contributions: The subgradient algorithm (we proposed) is different with the approach proposed in [14] in properties and analysis. In our approach, the communication topology is without loss of generality. This paper does not recur to the assumption that the adjacency matrices are doubly stochastic, and we only require the network is weight-balanced, which makes our algorithm more practical. In this paper, we consider a general multi-agent optimization problem where the main focus is to minimize a global objective function which is a sum of local objective functions, subject to global constraints, including an inequality constraint, an equality constraint and a (state) constraint set. Each local objective function is convex and only known by one particular agent. On the other hand, the inequality (resp. equality) constraint is given by a convex (resp. affine) function and known by all agents. Each node has its own convex constraint set, and the global constraint set is defined as their intersection. Particularly, we assume that the local constraint sets are identical. Our main interest is in computing approximate saddle points of the Lagrangian function of a convex constrained optimization problem. To set the stage, we first study the computation of approximate saddle points (as opposed to asymptotically exact solutions) by using the subgradient method with a constant step-size. We consider constant step-size rule because of its simplicity and practical relevance, and because our interest is in generating approximate solutions in finite number of iterations.
The paper is organized as follows. In Section II, we give some basic preliminaries and concepts. Then, in Section III, we present our problem formulation as well as distributed consensus algorithm preliminaries. We then introduce the distributed penalty primal-dual subgradient algorithm with some supporting lemmas and continue with a convergence analysis of the algorithm in Section IV. Furthermore, the properties of the algorithm are explored by employing a numerical example in Section V. Finally, we conclude the paper with a discussion in Section VI.

Preliminaries and Notations
In this section, we first introduce some preliminary results about graph theory, the properties of the projection operation on a closed convex set and convex analysis (referring to [13], [14] [ ] arg min || || In the subsequent development, the properties of the projection operation on a closed convex set play an important role. In particular, we use the projection inequality, i.e., for any vector x We also use the standard non-expansiveness property, i.e.

|| [ ] [ ] || || ||
for any x and y (2) In addition, we use the properties given in the following lemma.
Lemma 2.1: Let X be a nonempty closed convex set in n R . Then, we have for any By using the inequality of part (a), we obtain , for all y X ∈ Part (b) of the preceding lemma establishes a relation between the projection error vector and the feasible directions of the convex set X at the projection vector.
The following notations besides those aforementioned will be used throughout this paper. n R denotes the set of all ndimensional real vector spaces. Given a set S , we denote co( ) S by its convex hull. We write T x or T A to denote the transpose of a vector x or a matrix A . We let the function [ ] : denote the projection operator onto the non- x is the standard Euclidean norm in the Euclidean space. In this paper, the quantities (e.g., functions, scalars and sets) associated with agent i will be indexed by the superscript [ ] i .

Problem Statement
We consider a multi-agent network model. The nodes connectively at time 0 k ≥ can be represented by a directed weighted graph ( ) ( , ( ), ( )) Please note that the set ( ) is the set of edges with non-zero weights ( ) ij a k . In this paper the agents are to correspondingly solve the following optimization problem: where [ ] : i n f R R → is a convex objective function of agent i , and X is a nonempty, closed, compact and convex subset of n R . In particular, we study the cases where the local constraint sets are identical i.e., [  , and represents a global inequality constraint. The function : n h R R → , represents a global equality constraint, and is known by all the agents. Let * f denote the optimal value of (3) and * x denote an optimal solution of (3). We assume that the optimal value * f to be finite. We also represent the optimal solution set by * X , i.e., . We will assume that in general f is non-differentiable.
To generate optional solutions to the primal problem of Eq. (3), we consider optional solutions to its dual problem. Here, the dual problem is the one arising from penalty relaxation of the inequality constraints ( ) 0 g x ≤ and equality constraints ( ) 0 h x = . Note that the primal problem (3) is trivially equivalent to the following: with associated dual problem given by Here, the dual function, The dual optimal value of problem (7) is denote by * a and the set of dual optimal solutions is denoted by * Q . Since X is convex, f and g ℓ , for {1,..., } m ∈ ℓ , are convex, and * f is finite and the Slater's condition holds, we can conclude that * * f a = and * Q ≠ ∅ . We now proceed to characterize * q and * M . Pick any On the other hand, pick any * * x X ∈ . Then * x is feasible, i.e., * x X ∈ Therefore, we have * To prove the non-empty of 1) The set X is closed and convex.

2) Each function [ ] :
i n f R R → is convex.

3) All functions
When X is compact, the Assumption 3.2(4) holds. We here make the following assumptions on the network communication graphs ( ) G k .
Assumption 3.5 (Periodical Strong Connectivity): There is a positive integer B such that, for all 0 0 k ≥ , the directed is strongly connected. Inequality and Equality Constraints. x µ λ is a saddle point of the function H over if and only if it is a pair of primal and penalty dual optimal solutions and the following penalty minimax equality holds: Based on this characterization, we will use the subgradient method of the following section for finding the saddle points of the penalty function. We denote ( , ) r t is referred to as the reference signal (or input) of node i at time t .
We propose the First-Order Dynamic Average Consensus Algorithm below to reach the dynamic average consensus: there exists a real number 0 Without loss of generality, we only consider the case where 0 s = , being identical with the proof for a general s . Fixing some i , it holds that Applying recursive method, it follows that where we are using the property of (10) in the last two inequalities. Applying repeatedly (11), we have that, for any integer [ ,( , the following holds for t ph = Now we proceed by induction on ℓ . Suppose that (6) Consequently, as in (11), we have Following along the same lines as in (11), we obtain This establishes (6) for 1 i N + ∈ ℓ . By induction, we have shown that (6) holds. The proof for (7) is analogous.
Similarly, we can see that Combining the above two inequalities gives that For any 0 t ≥ , let t ℓ be the largest integer such that It follows from (13) that Consider the following Distributed projected subgradient algorithm proposed in [13]: Suppose The following is a slight modification of Lemma 8 and its proof in [13].

Distributed Subgradient Methods
In this section, we introduce a distributed penalty primaldual subgradient algorithm to solve the optimization problem (3), followed by its convergence properties. Distributed is the convex combination of dual estimates of agent i and its neighbors'.
[ ] ( ) i x S k keeps to the following rules: ( ) | | ( ( )) Proof: Modifying the second term on the right-hand side in the above formula, we then have In the following, we study the convergence behavior of the subgradient algorithm introduced in this section where the optimal solution and the optimal value is asymptotically agreed upon. unbounded. Therefore, unlike other subgradient algorithm, e.g., [15], [16], the distributed penalty primal-dual subgradient algorithm does not involve the dual projection steps onto compact sets. So we do not guarantee the subgradient [ ] ( ) i x S k not to be absolutely bounded, while the boundedness of subgradients is a standard assumption in the analysis of subgradient methods, e.g., see [6], [13], [17], [18], [19], [20]. The step-size of Assumption 3.1 is stronger than the more standard diminishing step-size scheme in [22] and this will correctly deal with the difficulty of the In this paper, we apply the harmonic series  satisfies the step-size Assumption 4.1 (for more details, one may refer to [14]).

A. Convergence Analysis
In the following, we will prove convergence of the distributed penalty primal-dual subgradient algorithm. First, we rewrite our algorithm into the following form:    , and the sequences of . This implies that the following inequalities hold for all We now consider the evolution

∑ ∑
, then we can say that the first term on the right-hand side in the above estimate is summable. Inequality and Equality Constraints.

Numerical Example
In this section, we study a simple numerical example to illustrate the effectiveness of the proposed distributed penalty primal-dual subgradient algorithm.

Conclusion and Future Work
In this paper, we formulated a distributed optimization problem with local objective functions, a global equality, a global inequality and a global constraint set defined as the intersection of local constraint sets. In particular, we considered the local constraint sets to be identical. Then, we proposed a distributed penalty primal-dual subgradient algorithm for the constrained optimization with a convergence analysis. Moreover, we employed a numerical example to show that the algorithm was asymptotically converge to primal solutions and optimal values. Future work may aim at the analysis that the local constraint sets of each agent are imparities. Also, we will pay attention to the convergence rates of the algorithms in this paper.