.
                                                D
                                             Matrix
                                          Calculus
                                D–1
               Appendix D: MATRIX CALCULUS                                                         D–2
               In this Appendix we collect some useful formulas of matrix calculus that often appear in finite
               element derivations.
               §D.1 THEDERIVATIVESOFVECTORFUNCTIONS
               Let x and y be vectors of orders n and m respectively:
                                                 x             y 
                                                    1                1
                                                   x                y
                                                  2             2 
                                             x =  . ,      y =  . ,(D.1)
                                                  .             . 
                                                    .               .
                                                   x               y
                                                    n               m
               where each component y may be a function of all the x , a fact represented by saying that y is a
                                      i                           j
               function of x,or
                                                       y = y(x).                                  (D.2)
               If n = 1, x reduces to a scalar, which we call x.Ifm = 1, y reduces to a scalar, which we call y.
               Various applications are studied in the following subsections.
               §D.1.1 Derivative of Vector with Respect to Vector
               Thederivative of the vector y with respect to vector x is the n × m matrix
                                                  ∂y     ∂y         ∂y 
                                                      1      2  ···    m
                                                    ∂x1   ∂x1        ∂x1
                                                                        
                                                  ∂y     ∂y         ∂y 
                                                      1      2  ···    m
                                           ∂y def  ∂x    ∂x         ∂x 
                                               =      2     2          2                        (D.3)
                                           ∂x      .       .   .      . 
                                                   .       .    ..    . 
                                                      .     .          .
                                                  ∂y     ∂y         ∂y 
                                                      1      2  ···    m
                                                    ∂xn   ∂xn        ∂xn
               §D.1.2 Derivative of a Scalar with Respect to Vector
               If y is a scalar,
                                                           ∂y 
                                                            ∂x1
                                                               
                                                           ∂y 
                                                   ∂y def  ∂x 
                                                       =  2.(D.4)
                                                   ∂x      . 
                                                           . 
                                                           . 
                                                             ∂y
                                                            ∂xn
               §D.1.3 Derivative of Vector with Respect to Scalar
               If x is a scalar,
                                            ∂y def  ∂y   ∂y         ∂y 	
                                                =      1     2  ...    m                          (D.5)
                                            ∂x       ∂x    ∂x         ∂x
                                                         D–2
               D–3                                 §D.1 THE DERIVATIVES OF VECTOR FUNCTIONS
               REMARKD.1
               Many authors, notably in statistics and economics, define the derivatives as the transposes of those given
               above.1 This has the advantage of better agreement of matrix products with composition schemes such as the
               chain rule. Evidently the notation is not yet stable.
               EXAMPLE D.1
               Given
                                                   
y           x1

                                                y =   1 ,    x =   x2                             (D.6)
                                                     y
                                                      2            x3
               and
                                                     y =x2−x
                                                      1    1   2                                  (D.7)
                                                     y =x2+3x
                                                      2    3    2
               the partial derivative matrix ∂y/∂x is computed as follows:
                                                ∂y     ∂y 
                                                     1    2
                                           ∂y   ∂x1    ∂x1    2x1   0 

                                                   ∂y   ∂y
                                                     1    2
                                              =            = −13                                (D.8)
                                           ∂x   ∂x2    ∂x2      02x
                                                   ∂y   ∂y              3
                                                   ∂x1  ∂x2
                                                     3    3
               §D.1.4 Jacobian of a Variable Transformation
               Inmultivariateanalysis,ifxandyareofthesameorder,thedeterminantofthesquarematrix∂x/∂y,
               that is
                                                              
                                                           ∂x
                                                       J =                                      (D.9)
                                                           ∂y
               is called the Jacobian of the transformation determined by y = y(x). The inverse determinant is
                                                              
                                                       −1   ∂y
                                                     J   = .(D.10)
                                                            ∂x
                1 Oneauthorputsitthisway: “Whenonedoesmatrixcalculus,onequicklyfindsthattherearetwokindsofpeopleinthis
                  world: those who think the gradient is a row vector, and those who think it is a column vector.”
                                                         D–3
                  Appendix D: MATRIX CALCULUS                                                                       D–4
                  EXAMPLE D.2
                  Thetransformation from spherical to Cartesian coordinates is defined by
                                          x =rsinθcosψ,         y =rsinθ sinψ,        z = r cosθ(D.11)
                  wherer > 0,0 <θ<πand0≤ψ<2π. ToobtaintheJacobianofthetransformation,let
                                                      x ≡ x1,     y ≡ x2,     z ≡ x3                              (D.12)
                                                      r ≡ y ,θ≡y ,ψ≡y
                                                           1           2            3
                  Then
                                                                                               
                                                       sin y cos y      sin y sin y      cos y   
                                                            2     3          2     3          2
                                             ∂x                                                
                                        J =      =  y cos y cos y    y cosy siny     −y siny 
                                                        1     2     3    1     2    3      1    2
                                             ∂y    −y siny siny      y sin y cos y       0                    (D.13)
                                                         1     2    3    1    2     3
                                                  =y2siny =r2sinθ.
                                                      1     2
                  The foregoing definitions can be used to obtain derivatives to many frequently used expressions,
                  including quadratic and bilinear forms.
                  EXAMPLE D.3
                  Consider the quadratic form
                                                                y = xTAx                                          (D.14)
                  where A is a square matrix of order n. Using the definition (D.3) one obtains
                                                             ∂y =Ax+ATx                                           (D.15)
                                                             ∂x
                  and if A is symmetric,
                                                                ∂y =2Ax.(D.16)
                                                                ∂x
                  Wecanofcoursecontinuethedifferentiation process:
                                                       ∂2y = ∂ ∂y=A+AT,(D.17)
                                                          2
                                                       ∂x     ∂x    ∂x
                  and if A is symmetric,
                                                                ∂2y
                                                                   2 = 2A.(D.18)
                                                                ∂x
                  Thefollowing table collects several useful vector derivative formulas.
                                                            y            ∂y
                                                                         ∂x
                                                           Ax            AT
                                                          xTAA
                                                           xTx           2x
                                                          xTAx       Ax+ATx
                                                                   D–4