Nuclear norm system identification with missing inputs and outputs

where A adj is the adjoint of the mapping A and M is the positive semidefinite matrix definedby the identityMz = A adj (A(z)) ∀z. (22)Step 2 of the algorithm is discussed in more detail in the following two sections.The minimizer X in step 3 can be expressed as(X = argmin ‖ ˆX‖ ∗ + ρˆX2 ‖ ˆX)−A(x)−A 0 −(1/ρ)Z‖ 2 F=min{p,q}∑i=1where u i , v i , σ i are given by a singular value decompositionmax{0,σ i − 1 ρ } u iv T i (23)A(x)+A 0 + 1 min{p,q}ρ Z = ∑i=1σ i u i v T i(see [27, theorem 2]). This operation is called ‘singular value soft-thresholding’.The residuals and tolerances in the stopping criterion in step 5 are defined as follows [26]:r p = A(x)+A 0 −X (24)r d = ρA adj (X prev −X) (25)ǫ p = √ pqǫ abs +ǫ rel max{‖A(x)‖ F ,‖X‖ F ,‖A 0 ‖ F } (26)ǫ d = √ nǫ abs +ǫ rel ‖A adj (Z)‖ 2 , (27)Typical values for the relative and absolute tolerances are ǫ rel = 10 −3 and ǫ abs = 10 −6 . Thematrix X prev in (25) is the value of X in the previous iteration.Instead of using a fixed penalty parameter ρ, one can vary ρ to improve the speed ofconvergence. An example of such a scheme is to adapt ρ at the end of each ADMM iterationas follows [28] ⎧⎨ τρ if ‖r p ‖ F > µ‖r d ‖ 2ρ := ρ/τ if ‖r d ‖ 2 > µ‖r p ‖ F⎩ρ otherwise.This scheme depends on parameters µ > 1, τ > 1 (typical values are µ = 10 and τ = 2).Note that varying ρ has an important consequence on the algorithm in table 1. If ρ is fixed,the coefficient matrix H +ρM in the equation (21) that is solved in step 2 of each iterationis constant throughout the algorithm. Therefore only one costly factorization of H + ρMis required. If we change ρ after step 5, a new factorization of H + ρM is needed beforereturning to step 2. We explain in §4.2 how the extra cost of repeated factorizations can beavoided.10