Principles of Modern Radar - Volume 2 1891121537

5.3 SR Algorithms 169the value of one parameter into the values for the other two. However, once a solution toone problem is available, we can calculate (to at least some accuracy) the parameters forthe other two solutions. 22First, notice that only a certain range of parameters makes sense. Consider solvingBP σ with σ = ‖ y‖ 2 . The solution to this problem is obviously ˆx σ = 0. Any largervalue of σ will yield the same solution. Similarly, imagine solving LS τ with τ = ‖ ˆx BP ‖ 1 .(Recall that ˆx BP is the solution to BP σ with σ = 0.) In other words, this is the minimuml 1 solution such that A ˆx BP = y. Any larger value of τ will produce the same solution.Thus, the solution with x = 0 corresponds to a large value of λ, while the solution ˆx BPcorresponds to the limit of the solution to QP λ as λ approaches zero. Values outside thisrange will not alter the resulting solution.The fact that the BP solution is the limit of the solution to QP λ is important. Thealgorithms that solve the unconstrained problem cannot be used to precisely compute BPsolutions. Algorithms that solve QP λ exhibit a fundamental deficiency in solving BP, ascan be seen by their phase transition. See [66] for results on this issue. Notice that thisproblem does not arise when dealing with noisy data and solving the problem for σ>0,as the corresponding positive λ then exists. We will emphasize recovery from noisy datathroughout this chapter. In contrast, much of the CS literature centers around solving thenoise-free BP problem. From a coding or compression standpoint, this makes a great dealof sense. This distinction, σ>0 vs. σ = 0, colors our discussion, since algorithms thatwork beautifully for BPDN may work poorly for BP and vice versa. Indeed, an examplewould be the approximate message passing (AMP) algorithm [66], whose developmentwas at least partially motivated by the inability of algorithms like Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) to solve the BP problem exactly.We can create a plot of ‖A ˆx − y‖ 2 versus ‖ ˆx‖ 1 which is parametrized by λ (or byτ or σ ) to obtain what is known as the Pareto frontier for our problem of interest. Wewill denote the Pareto frontier as φ(τ). This curve represents the minimum l 2 error thatcan be achieved for a given l 1 bound on the solution norm. Pairs above this curve aresub-optimal, and pairs below the curve are unattainable. It turns out that this curve isconvex. Furthermore, for a given point on the curve, the three parameters associated withthe corresponding solution ˆx are given by φ(τ) = σ = ‖A ˆx − y‖ 2 , τ = ‖ ˆx‖ 1 , and λ isrelated to the slope of the Pareto curve at that point [63]. In particular, the slope of thePareto curve can be calculated explicitly from the solution ˆx at that point as∥ φ ′ A H r ∥∥∥∥∞(τ) =−∥ ‖r‖ 2where r = y − A ˆx τ [63]. This expression is closely related to λ, which is given by λ =2 ∥∥ A H r ∥ ∥∞, as shown in [67]. 23 These results are proven and discussed in detail in [63].Thus, much like the L-curve [68,69] that may be familiar from Tikhonov regularization,the parameter λ can be viewed as a setting which allows a tradeoff between a family ofPareto optimal solutions. An example Pareto frontier plot is shown in Figure 5-6. In thefigure, we have labeled the values of the end points already discussed.22 A good discussion of the numerical issues in moving between the parameters is provided in [61]. In anutshell, determining λ from the solution to one of the constrained problems is fairly difficult. The othermappings are somewhat more reliable.23 Note that the factor of 2 stems from the choice to not include a 1/2 in the definition of ˆx λ in (5.19).