Downloaded from https://www.everyspec.com IAACP 706-1117 2)The wnemd (positive definite) quadratic function can be written as q(x) - a + brx + xTAx (124) where the matrix A is postive definite and symmetric. The procedure for finding the minimum value q(x*) consists of starting at some initial point x. and taking succesive steps along the directions s, s, ... , S,.. All these directions are choseti to be A-conjugate; i.e., for all i*j, i,j=O, 1, ... , n- 1, these directions satisfy the relationship sIs = 0. (12-7) Successive points in the minimization proce- dure are computed from x1+1 = x1 + ais,.
(12.8) As in the steepest descent method, the value of the step size a 5 is found by minimizing f(x, + as). It can be shown that, regardless of the starting point, this sequential process leads to the desired minimum value of q(x*) in n steps The parabolas are equialue contours of the ob- or less (where n is the number of variables in jective function y = 16x1 + (x 2 - 4) . The heavy the vector x) (Ref. 8). Thus, conjugate direc- zig-zag line shows the path taken by a steepest descent procedure seeking the minimum value of tions minimize a quadratic very efficiently. this function. 12-2.2.2 The Fletcher-Powell Method FIGURE 12.1.
Finding the Minimum Using the Steepest Descent Method' The method presented by Fletcher and IPowell (outlined in Table 12-1) is probably the most powerful general procedure now known for finding a local minimum of a gen- f(x) = f(X*) + l(x - X*)T H,(x*)(x - x*), eral function f(x) (Refs. 8 and 9). (12-5) Central to the method is a symmetric, positive definite matrix H,, which is updated where at each iteration, and which supplies the cur- T indicates the transpose of a matrix rent direction of motion s, when multiplied and by the gradient vector. The numerators A, H,(x*) = matrix of second partials of f and B, in Steps 8 and 9 of the Fletcher-Powell evaluated at x*.
method are both matrices, while the denomi- H r is assumed to be positive definite; thus, the nators are scalars. Fletche and Powell have function behaves like a pure quadratic in the demonstrated that their method will always vicinity of x*. converge, since the objective function f is ini- tially decreasing along the direction s,. When 12-2.2.1 Conjugate Directions the method is applied to a quadratic (Eq. 12-5), the directions s, are A-conjugate, and Most, if not all, of the newer, more effi- the process converges to a minimum in n cient unconstrained minimization procedures steps.
The matrix 3l, converges to the inverse are based on the idea of conjugate directions matrix A-' after n steps. When applied to a (Refs. 6-8). general function, H, tends to become the 12-5