General Inverse

Page Contents
These notes follow the development in chapter 2 of , though the exposition has been somewhat summarized and simplified. ;
 

Rank Factorization

If #A is a ~m # ~n matrix of rank &rho. ( &rho. =< ~m , &rho. =< ~n ) _ then &exist. ~m # &rho. matrix #B , _ and &rho. # ~n matrix #C , both of rank &rho. such that _ #A _ = _ #B#C.

Such a composition exists for all matrices of non-zero rank (For proof see 1.4.3 (page 9), but is not necessarily unique.

The above decomposition is called a #~{rank factorization} of #A

g-inverse

If #A is a ~m # ~n matrix, the ~n # ~m matrix #G is a #~{generalized inverse} or #~{g-inverse} of #A if _ _ #A#G#A _ = _ #A.

If #A is a reqular square matrix, then #A^{-1} is the unique g-inverse of #A, otherwise #A has infinitely many g-inverses ( proof below).

We write _ #G _ = _ #A^{-} _ when G is a g-inverse of #A _ [ but note that this is #{not} (necessarily) unique ].

 

 

Generalized Inverse Solutions

#G is a g-inverse of #A _ <=> _ #~x = #G#~y is a solution of #A#~x = #~y , _ for any _ #~y &in. CSp(#A) (column space).

[ I.e. in particlar _ #A#G#~y _ = _ #~y _ for any _ #~y &in. CSp(#A) .]

Proof:

(" => ") _ #~y &in. CSp(#A) _ => _ #~y = #A#~z _ some #~z , _ then _ _ #A(#G#~y) _ = _ #A#G##A#~z _ = _ #A=~z _ = _ #~y

(" <= ") _ #A#G#~y = #~y _ => _ #A#G##A#~z = #A#~z , _ &forall. #~z , _ in particular for _ #~z = #~e_~i , _ the ~i^{th} column vector of #I_~n , _ ~i = 1 ... ~n. _ Then #A#G##A _ = _ #A .

 

 

Multiplicity of g-inverses

If _ #A _ = _ #B#C _ is a rank factorization of #A, then &exist. right and left inverses _ #B_~l and #C_~r _ of _ #B and #C _ respectively, _ ( i.e. #B_~l#B _ = _ #I_{&rho.} _ and _ #C#C_~r _ = _ #I_{&rho.} ) _ then #C_~r#B_~l is a g-inverse of #A.

Proof: _ _ _ #A#C_~r#B_~l#A _ = _ #B#C#C_~r#B_~l#B#C _ = _ #B#C _ = _ #A .

Now recall &exist. regular #M, #N such that _ #M#A#N _ = _ #C , _ ( &therefore. _ #A _ = _ #M^{-1}#C#N^{-1} ) , _ where

#C _ = _ matrix{#I_{&rho.},#0/#0,#0}

For ~{any} matrices #U, #V, #W of the appropriate dimensions, the matrix

#D _ = _ matrix{#I_{&rho.},#U/#V,#W} _ _ is a g-inverse of #C:

matrix{#I_{&rho.},#0/#0,#0}matrix{#I_{&rho.},#U/#V,#W}matrix{#I_{&rho.},#0/#0,#0} _ = _ matrix{#I_{&rho.},#U/#0,#0}matrix{#I_{&rho.},#0/#0,#0} _ = _ matrix{#I_{&rho.},#0/#0,#0} .

Then _ #N#D#M _ is a g-inverse of #A:

#A #N#D#M #A _ = _ #M^{-1}#C#N^{-1} #N#D#M #M^{-1}#C#N^{-1} _ = _ #M^{-1}#C#D#C#N^{-1} _ = _ #M^{-1}#C#N^{-1} _ = _ #A .

So every matrix that is not square and regular has infinitely many g-inverses.

 

Rank of g-inverse

Recall that for any compatible matrices _ rank(#A#B) =< rank(#A) , _ rank(#A#B) =< rank(#B) .

If #G is a g-inverse of #A , _ then _ rank(#A) _ = _ rank(#A#G) _ = _ rank(#G#A)

Proof: _ _ _ rank(#A) _ = _ rank(#A#G#A) _ =< _ rank(#A#G) _ =< _ rank(#A) _ _ etc.

 

 

 

Reflexive g-inverse

#G is a g-inverse of #A . _ #G is a #~{reflexive} g-inverse if #A is also a g-inverse of #G. _ I.e.
_ _ _ _ _ #A#G#A _ = _ #A _ _ and _ _ #G#A#G _ = _ #G

#G is a g-inverse of #A , _ then

  • rank(#A) _ =< _ rank(#G)
  • rank(#A) _ = _ rank(#G) _ _ <=> _ _ #G is reflexive.

 

 

 

Solution of Linear System

We have shown that if #G is a g-inverse of #A _ then _ #A#G#~y = #~y , _ for any _ #~y &in. CSp(#A), and conversely, _ #~x = #G#~y _ is a solution of _ #A#~x = #~y , _ only if _ #G is a g-inverse of #A. This will be used in the proof of the following:

If #G is a g-inverse of #A , _ #~y &in. CSp(#A) , _ then

_ _ _ _ _ _ #A#~x _ = _ #~y _ _ <=> _ _ #~x _ = _ #G#~y + ( #I - #G#A ) #~z , _ _ some or any #~z .

Proof:

(" => ") _ #A#~x = #~y . _ Put _ #~z = #~x - #G#~y , _ then
_ _ _ _ #G#~y + ( #I - #G#A ) #~z _ = _ #G#~y + ( #I - #G#A )( #~x - #G#~y ) _ = _ #G#~y + #~x - #G#~y - #G#A#~x + #G#A#G#~y _ = _ #~x - #G#A#~x + #G#A#G#~y _ = _ #~x

(" <= ") _ #A ( #G#~y + ( #I - #G#A ) #~z ) _ = _ #A#G#~y + #A#~z - #A#G#A#~z _ = _ #A#G#~y _ = _ #~y

 

Minimum Norm g-inverse

Let #G be a g-inverse of #A . _ #G is a #~{minimum norm} g-inverse of #A _ if _ #~x = #G#~y _ is the solution of _ ~#A#~x = #~y _ with minimum norm for any #~y &in. CSp(#A). _ [ I.e. _ if _ #~x _ is a solution of _ ~#A#~x = #~y _ then _ || #G#~y || _ =< _ || #~x || ].

#G is a minimum norm g-inverse of #A_ _ <=> _ _ (#G#A)^T = #G#A _ [ i.e. #G#A symmetric ]

Proof:

(" => ") _ #~y &in. CSp(#A) _ then _ #~x = #G#~y is a solution of #A#~x = #~y _ [ proved above ].
#G#~y has minimum norm _ => _ || #G#~y || _ =< _ || #G#~y + ( #I - #G#A ) #~z || _ &forall. #~z .
=> _ || ( #I - #G#A ) #~z ||^2 + 2#~y^T #G^T ( #I - #G#A ) #~z _ >= _ 0
=> _ || ( #I - #G#A ) #~z ||^2 + 2#~u^T #A^T #G^T ( #I - #G#A ) #~z _ >= _ 0 _ (#*) _ _ _ [ #A#~u &in. CSp(#A) _ any #~u ] .
Now suppose _ #~u^T #A^T #G^T ( #I - #G#A ) #~z _ != _ 0 , _ then _ 2&alpha.#~u^T #A^T #G^T ( #I - #G#A ) #~z _ > _ || ( #I - #G#A ) #~z || ,
for | &alpha. | sufficiently large ( &alpha. positive or negative as appropriate ) , which contradicts (#*).
So _ #~u^T #A^T #G^T ( #I - #G#A ) #~z _ != _ 0 , _ _ &forall. #~u ,#~z ,
=> _ _ #A^T #G^T ( #I - #G#A ) _ = _ #0 _ _ => _ _ #A^T#G^T _ = _ (#G#A)^T #G#A _ - symmetric [ #X^T #X symmetric, any matrix #X ]
so #G#A symmetric.

(" <= ") _ We know that any solution is of the form _ #G#~y + ( #I - #G#A ) #~z .
Now we must show that _ || #G#~y || _ =< _ || #G#~y + ( #I - #G#A ) #~z || .
|| #G#~y + ( #I - #G#A ) #~z ||^2 ­_ = _ || #G#~y ||^2 _ + _ || ( #I - #G#A ) #~z ||^2 _ + _ 2 #~y^T#G^T ( #I - #G#A ) #~z .
#~y &in. CSp(#A) _ => _ #~y = #A#~u , _ some #~u .
#~y^T#G^T ( #I - #G#A ) #~z _ = _ #~u^T#A^T#G^T ( #I - #G#A ) #~z _ = _ #~u^T#G#A ( #I - #G#A ) #~z _ = _ #~u^T ( #G#A - #G#A#G#A ) #~z _ = _ 0 .
As _ || ( #I - #G#A ) #~z ||^2 _ >= _ 0 , _ this completes the proof.

 

Least Squares g-inverse

Let #G be a g-inverse of #A . _ #G is a #~{least squares} g-inverse of #A _ if _ for any _ #~x , #~y , _ || #A#G#~y - #~y || _ =< _ || #A#~x - #~y || .

#G is a least squares g-inverse of #A_ _ <=> _ _ (#A#G)^T = #A#G _ [ i.e. #A#G symmetric ]

 

Moore-Penrose Inverse

If #G is a reflexive g-inverse of #A which is both minimum norm and least squares then it is said to be the #~{Moore-Penrose Inverse} of #A. _ I.e. _ #G is the Moore-Penrose inverse of #A if

  • #A#G#A _ = _ #A
  • #G#A#G _ = _ #G
  • #A#G _ = _ (#A#G)^T _ = _ #G^T#A^T
  • #G#A _ = _ (#G#A)^T _ = _ #A^T#G^T

For any matrix _ #A , _ the Moore-Penrose inverse exists and is unique, and is denoted by _ #A^+ .

Proof:

Uniqueness: _ Suppose #H is also a Moore-Penrose inverse of #A, i.e.

  • #A#H#A _ = _ #A
  • #H#A#H _ = _ #H
  • #A#H _ = _ (#A#H)^T _ = _ #H^T#A^T
  • #H#A _ = _ (#H#A)^T _ = _ #A^T#H^T

then

  1. #A#G _ = _ #A#H#A#G _ = _ #H^T#A^T#G^T#A^T _ = _ #H^T#A^T _ = _ #A#H
  2. #G#A _ = _ #G#A#H#A _ = _ #A^T#G^T#A^T#H^T _ =_ #A^T#H^T _ = _ #H#A

So _ #G _ = _ #G#A#G _ = _ #G#A#H _ = _ #H#A#H _ = _ #H .

Existence: _ If _ #A _ = _ #B#C _ is a rank factorization of #A , _ then

_ _ _ _ _ _ _ #A^+ _ _ = _ _ #C^T ( #C#C^T )^{-1} ( #B^T#B )^{-1} #B^T