Orthogonality

Page Contents

Geometric Orthogonality

Two subspaces L_1 and L_2 of &reals.^{~n} are #~{orthogonal}, and we write _ L_1 &perp. L_2 , _ if _ &forall. ~{#v}_1 &in. L_1, ~{#v}_2 &in. L_2, _ ~{#v}_1 #. ~{#v}_2 = #0. _ (I.e. any vector in L_1 is orthogonal to any vector in L_2.) _ Note that orthogonality implies that L_1 &intersect. L_2 = \{#0\}.

Now consider any two subspaces of &reals.^~n, not neccessarily orthogonal, _ if L_1 &intersect. L_2 != \{#0\} then we can find V_1 and V_2 , _ where _ L_1 = V_1 &oplus. (L_1 &intersect. L_2), _ L_2 = V_2 &oplus. (L_1 &intersect. L_2) , _ so that _ V_1 &perp. (L_1 &intersect. L_2) _ and _ V_2 &perp. (L_1 &intersect. L_2). _ If L_1 &intersect. L_2 = \{#0\} then put V_1 = L_1 and V_2 = L_2. _ [The symbol &oplus. represents the direct sum of two spaces.]

L_1 and L_2 are said to be ~#{geometrically orthogonal} if _ V_1 &perp. V_2 .

Of course orthogonality implies geometric orthogonality. As an example consider three dimensional space. A line (through the origin) perpendicular to a plane (through the origin) are orthogonal, their intersection being just the origin (i.e. the null vector). Two planes (through the origin) which are perpendicular to each other are geometrically orthogonal.

Commutative Projections

#{Lemma}: _ If ~p_1 and ~p_2 are orthogonal projections onto L_1 and L_2 respectively, then

L_1 and L_2 are geometrically orthogonal _ &iff. _ ~p_1~p_2 = ~p_2~p_1.

Proof: _ If L_1 and L_2 are geometrically orthogonal then

&reals.^{~n} _ _ _ = _ _ _ V_0 &oplus. V_1 &oplus. V_2 &oplus. V_3

where _ V_1 and V_2 are as before, i.e. _ L_1 = V_1 &oplus. ( L_1 &intersect. L_2 ) , _ and _ L_2 = V_2 &oplus. ( L_1 &intersect. L_2 ) . _ Then put _ V_3 = L_1 &intersect. L_2 _ and _ V_0 = (V_1 &oplus. V_2 &oplus. V_3)^{&perp.}.

If ~{#v} &in. &reals.^{~n}, then ~{#v} = ~{#v}_0 + ~{#v}_1 + ~{#v}_2 + ~{#v}_3 _ (where ~{#v}_{~i} &in. V_{~i}). _ Then ~p_1(~{#v}) = ~{#v}_1 + ~{#v}_3 &in. V_1 &oplus. V_3 = L_1, etc.
So _ ~p_1~p_2~{#v} = ~p_1(~{#v}_2 + ~{#v}_3) = ~{#v}_3 _ and _ ~p_2~p_1~{#v} = ~p_2(~{#v}_1 + ~{#v}_3) = ~{#v}_3. _ (I.e. the projection onto L_1 &intersect. L_2 )

Convesely, if _ ~p_1~p_2 = ~p_2~p_1 _ then ~p_1~p_2~{#v} &in. L_1, ~p_2~p_1~{#v} &in. L_2, so ~p_1~p_2~{#v} &in. L_1 &intersect. L_2. _ If ~{#v}_1 &in. V_1 then _ ~p_2~{#v}_1 = ~p_2~p_1~{#v}_1 &in. L_1 &intersect. L_2. _ But V_1 &perp. (L_1 &intersect. L_2), so ~p_2~{#v}_1 = #0 for any ~{#v}_1 &in. V_1 _ i.e. V_1 &perp. L_2, in particular V_1 &perp. V_2, _ so L_1 and L_2 are geometrically orthogonal.

#{Corollary}: _ If L_1 and L_2 are geometrically orthogonal _ then _ ~p_1~p_2 is the projection onto L_1 &intersect. L_2.
(In fact if L_1 ... L_{~k} are pairwise geometrically orthogonal then ~p_1~p_2 ... ~p_{~k} is the projection onto L_1&intersect.L_2&intersect. ... &intersect.L_{~k}.)

Orthogonal Factors

Two Factors, F and G are said to be ~{orthogonal} if their corresponding subspaces L_F and L_G are geometrically orthogonal.

The following follows immediately from the lemma:

F and G are orthogonal _ _ _ _ <=> _ _ _ _ ~p_F ~p_G _ = _ ~p_G ~p_F

Furthermore we can show that:

F and G are orthogonal _ _ _ _ <=> _ _ _ _ ~p_F ~p_G _ = _ ~p_{F&min.G}

To see this consider the corresponding matrices:

P_F P_G = P_{F&min.G} _ => _ P_F P_G = P_{F&min.G} = P_{F&min.G}^T = (P_F P_G)^T = P_G^T P_F^T = P_G P_F _ => _ F, G orthogonal.

Conversely if F, G orthogonal then ~p_F ~p_G is the projection onto L_F &intersect. L_G = L_{F&min.G}, _ so ~p_F ~p_G = ~p_{F&min.G}

To sum up, in terms of matrices:

F and G are orthogonal _ _ _ _ <=> _ _ _ _ P_G P_F _ = _ P_F P_G _ = _ P_{F&min.G}

Orthogonality Conditions

If F and G are orthogonal if, and only if

&hash.(F # G)^{-1}(~{f, g}) _ # _ &hash.(F&min.G)^{-1}(~h) _ _ = _ _ ~{n_f} ~{n_g} , _ _ _ _ &forall. ~f , ~g &in. G _ for which _ &hash.(F # G)^{-1}(~{f, g}) != 0,

where ~h is the level of F&min.G for which _ F^{-1}(~f) &in. (F&min.G)^{-1}(~h) _ and _ F^{-1}(~g) &in. (F&min.G)^{-1}(~h), _ i.e. the level that "contains" ~f and ~g.

Putting _ ~{n_{fg}} = &hash.(F # G)^{-1}(~{f, g}) _ and _ ~{n_h} = &hash.(F&min.G)^{-1}(~h) , _ then we can write more succinctly

~{n_{fg}} ~{n_h} _ = _ ~{n_f} ~{n_g} _ <=> _ F and G are orthogonal.

#{Proof}

It was shown above that _ F and G are orthogonal _ <=> _ P_F P_G _ = _ P_{F&min.G} . _ Now recall that

( P_F )_{~i, ~j} _ = _ array{ {1/~n_{F({~i})}}, _ _ ,if _ F(~i) = F(~j)// 0,,otherwise}

so

(P_F P_G)~{_{i, j}} _ = _ sum{ ~f_{~i,~k} ~g_{~k,~j},~k = 1 ... ~n,}

where

~{f_{i,k} g_{k,j}} _ = _ array{ fract{1,~n_{F({~i})}}fract{1,~n_{G({~j})}}, _ _ , if _ F(~i) = F(~k) _ and _ G(~j) = G(~k)/ _ /0,, otherwise. }

but, due to orthogonality

(P_F P_G)~{_{i, j}} _ = _ sum{(P_F)_{~{i,k}}(P_G)_{~{k,j}},~k,}

_ _ _ _ _ = _ sum{fract{1,~n_{F(~i)}} _ array{ &chi. ( ~k )/\{~m | F(~m) = F(~i)\}} _ _ fract{1,~n_{G(~j)}} _ array{ &chi. ( ~k )/\{~m | G(~m) = G(~j)\}} ,~k,}

( Putting _ ~f = F_{~i}, _ ~g = G_{~j}, _ _ and &chi._A( ) represents the characteristic function [ &chi._A(~x) = 1 if ~x &in. A, 0 otherwise.] )

_ _ _ _ _ = _ sum{fract{1,~{n_f n_g}} array{ &chi. ( ~k )/\{~m | F(~m) = ~f and G(~m) = ~g\} } ,~k,}

_ _ _ _ _ = _ fract{~{n_{fg}},~{n_f n_g}}

Now, turning to matrix corresponding to the minimum

( P_{F&min.G} )_{~i, ~j} _ = _ array{ {1/~n_{F&min.G({~i})}}, _ _ ,if F&min.G(~i) = F&min.G(~j)// 0,,otherwise}

and since _ P_F P_G _ = _ P_{F&min.G} , _ then by comparing the matrix elements the result follows.


Note: _ If _ F&min.G = 0 _ we have: _ F and G are orthogonal _ &iff. _ ~{n_{fg}} = (~{n_f} ~{n_g}) ./ ~n

Example

&hash.(F # G)^{-1}(~{f, g})

Consider the example of the group of students, summarized here on the right.

This table represents the quantity _ &hash.(F # G)^{-1}(~{f, g}) . In the three tables below the other quantities are displayed.
Remember that F&min.G has two levels: _ \{ (Blond, Blue) , (Blond, Green) \} _ and _ \{ (Dark, Brown) , (Red, Brown) \}.

Note that in the last two tables, there is correspondence on all the elements which are not zero in the first table, so these two factors are orthogonal.

&hash.(F&min.G)^{-1}(~h)
&hash.(F # G)^{-1}(~{f, g}) # &hash.(F&min.G)^{-1}(~h)
~{n_f} ~{n_g}