Nonnegativity Constraints in Numerical Analysis - CiteSeer

4.1 Nonnegative Matrix FactorizationIn Nonnegative Matrix Factorization (NMF), an m × n (nonnegative) mixed data matrix Xis approximately factored into a product of two nonnegative rank-k matrices, with k smallcompared to m and n, X ≈ WH. This factorization has the advantage that W and H canprovide a physically realizable representation of the mixed data. NMF is widely used in avariety of applications, including air emission control, image and spectral data processing,text mining, chemometric analysis, neural learning processes, sound recognition, remotesensing, and object characterization, see, e.g. [9].NMF problem: Given a nonnegative matrix X ∈ R m×n and a positive integer k ≤min{m, n}, find nonnegative matrices W ∈ R m×k and H ∈ R k×n to minimize the functionf(W, H) = 1‖X − 2 WH‖2 F , i.e.k∑min f(H) = ‖X − W (i) ◦ H (i) ‖ subject to W, H ≥ 0 (15)Hi=1where ′ ◦ ′ denotes outer product, W (i) is ith column of W, H (i) is ith column of H TFigure 1: An illustration of nonnegative matrix factorization.See Figure 1 which provides an illustration of matrix approximation by a sum of rankone matrices determined by W and H. The sum is truncated after k terms.Quite a few numerical algorithms have been developed for solving the NMF. The methodologiesadapted are following more or less the principles of alternating direction iterations,the projected Newton, the reduced quadratic approximation, and the descent search. Specificimplementations generally can be categorized into alternating least squares algorithms[64], multiplicative update algorithms [40, 51, 52], gradient descent algorithms, and hybridalgorithms [67, 69]. Some general assessments of these methods can be found in [20, 56]. Itappears that there is much room for improvement of numerical methods. Although schemesand approaches are different, any numerical method is essentially centered around satisfyingthe first order optimality conditions derived from the Kuhn-Tucker theory. Note that thecomputed factors W and H may only be local minimizers of (15).14