Nonnegativity Constraints in Numerical Analysis - CiteSeer

the active set method fnnls. It also enables their method to incorporate multiple activeconstraints at each iteration. By employing non-diagonal gradient scaling, PQN-NNLSovercomes some of the deficiencies of a projected gradient method such as slow convergenceand zigzagging. An important characteristic of PQN-NNLS algorithm is that despite theefficiencies, it still remains relatively simple in comparison with other optimization-orientedalgorithms. Also in this paper, Kim et al. gave experiments to show that their methodoutperforms other standard approaches to solving the NNLS problem, especially for largescaleproblems.Algorithm PQN-NNLS:Input: A ∈ R m×n , b ∈ R mOutput: x ∗ ≥ 0 such that x ∗ = arg min ‖Ax − b‖ 2 .Initialization: x 0 ∈ R n +, S 0 ← I and k ← 0repeat1. Compute fixed variable set I k = {i : x k i = 0, [∇f(x k )] i > 0}2. Partition x k = [y k ; z k ], where y k i /∈ I k and z k i ∈ I k3. Solve equality-constrained subproblem:3.1. Find appropriate values for α k and β k3.2. γ k (β k ; y k ) ← P(y k − β k ¯Sk ∇f(y k ))3.3. ỹ ← y k + α(γ k (β k ; y k ) − y k )4. Update gradient scaling matrix S k to obtain S k+15. Update x k+1 ← [ỹ; z k ]6. k ← k + 1until Stopping criteria are met.Sequential Coordinate-wise Algorithm for NNLSIn [29], the authors propose a novel sequential coordinate-wise (SCA) algorithm whichis easy to implement and it is able to cope with large scale problems. They also derivestopping conditions which allow control of the distance of the solution found to the optimalone in terms of the optimized objective function. The algorithm produces a sequence ofvectors x 0 , x 1 , . . . , x t which converges to the optimal x ∗ . The idea is to optimize in eachiteration with respect to a single coordinate while the remaining coordinates are fixed. Theoptimization with respect to a single coordinate has an analytical solution, thus it can becomputed efficiently.10