\documentclass{article} %\pagestyle{empty} \setlength{\topmargin}{0in} \setlength{\oddsidemargin}{0in} \setlength{\textheight}{8in} \setlength{\textwidth}{6in} \usepackage{amssymb} \usepackage{amsthm} \usepackage{enumerate} \newcommand{\stbar}{\;|\;} \newcommand{\R}{\mathbb{R}} \newcommand{\ra}{\rightarrow} \newcommand{\dotprod}{\bullet} \newcommand{\proj}{\mathrm{Proj}} \renewcommand{\span}{\mathrm{Span}} \newtheorem{theorem}{Theorem} \theoremstyle{definition} \newtheorem{example}[theorem]{Example} \newtheorem{claim}[theorem]{Claim} \newtheorem{fact}[theorem]{Fact} \newtheorem{definition}[theorem]{Example} \newtheorem{exercise}{Exercise} \newtheorem{maple}{Maple Exercise} \begin{document} \title{Math 315 - Notes on Section 4.2} \date{October 23, 2002} %\author{Blake Thornton} \maketitle \section{Projections} To project a vector to a subspace, you first need an orthogonal basis for that subspace. Assuming you have an orthogonal subspace, you can use the formula on page~217 of the text. Let $\{Q_1, \ldots, Q_m\}$ be the basis for your subspace $W$ and you want to project $B$ to $W$. Then: \[ \proj_W B = \frac{B \dotprod Q_1}{Q_1 \dotprod Q_1} Q_1 + \frac{B \dotprod Q_2}{Q_2 \dotprod Q_2} Q_2 + \cdots + \frac{B \dotprod Q_m}{Q_m \dotprod Q_m} Q_m \] The projection has the following important properties: \begin{enumerate} \item The vector $B_1$ defined as $B_1 = B- \proj_W B$ is orthogonal to $W$. (i.e., $B_1 \dotprod Q_i = 0$ for all $i$). \item The vector $\proj_W B$ is in $W$, and in fact, it is the vector that is ``closest'' to $B$. What this means is that if $v$ is any other vector in $W$ ($v \neq \proj_WB$), then \[ |B-\proj_WB| < |B-v| \] (You should draw the picture to see what this means geometrically.) \end{enumerate} \begin{exercise} Let $W$ be the subspace of $\R^4$ with basis $\{Q_1, Q_2\}$ with \[ Q_1 = [1,-1,1,3]^t \qquad Q_2=[1, -1, 1, -1] \] \begin{enumerate}[(a)] \item Compute the projection of $B=[1,2,3,4]^t$ onto $W$. \item Find the orthogonal vector $B_1$. \item Compute $|B_1|$. \item Pick an arbitrary vector $v \in W$ (not equal to $\proj_WB$) and show that $|B_1| < |B-v|$. \end{enumerate} \end{exercise} \section{Gram-Schmidt} Gram-Schmidt is the process of using our knowlege of vector projections to find an orthogonal basis. First, before we can do ``Gram-Schmidt,'' we actually need a basis for our space. (We have talked a lot about finding bases for spaces and the connection to row reduction. For example, if you want to find a basis for row space, you can row reduce and use the non-zero rows of the reduced matrix. Or, if you want to find a basis for column space, you can row reduce and use the pivot columns of the \emph{original} matrix as a basis.) So, lets work though an example. Lets start with the basis $\{A_1, A_2, A_3 \}$ for $W \subset \R^4$ given here: \[ A_1 = [1,3,-1,1]^t \quad A_2=[2,-6,0,1]^t \quad A_3=[4,12,-1,2]^t \] \begin{exercise} Show that this is not an orthogonal basis. \end{exercise} We are going to build an orthogonal basis up one vector at a time. We first let $Q_1=A_1$, this is our first vector in our new basis: First, consider the space: \[ W_1 = \span \{ Q_1 \} \] $W_1$ is a one-dimensional space, and in fact, $\{A_1\}$ is an orthogonal basis for $W_1$. Now, lets add $A_2$: \[ W_2 = \span \{ Q_1, A_2 \} \] $W_2$ is 2 dimensional, but this is not an orthogonal basis. Lets compute some projections: \[ \proj_{W_1}A_2 = \frac{A_2 \dotprod Q_1}{Q_1 \dotprod Q_1} Q_1 = \frac{-15}{12}Q1 \] Lets now compute the orthogonal vector (vector $B_1$ in the Fourier formula): We'll call this vector $Q_2$: \[ Q_2 = A_2 - \proj_{W_1}A_2 = \left[\frac{13}{4},\frac{-9}{4},\frac{-5}{4},\frac{-9}{4}\right]^t \] \begin{exercise} \label{dim 2} Show the following things about the new situation: \begin{enumerate}[(a)] \item Vector $Q_1$ and $Q_2$ are orthogonal. \item $W_2 = \span \{ Q_1, Q_2 \}$. \\ Hints: \begin{enumerate} \item Remember that $W_2$ is defined to be $\span \{Q_1, A_2\}$, so I'm asking you to show that: \[ \span \{ Q_1, A_2 \} = \span \{ Q_1, Q_2 \} \] \item The easiest way is probabaly to show this is to show that $Q_1$ and $Q_2$ can be written as a combiniation of $Q_1$ and $A_2$, which shows that \[ \span \{ Q_1, Q_2 \} \subset \span \{ Q_1, A_2 \} \] and then show that $Q_1$ and $A_2$ can be written as a combiniation of $Q_1$ and $Q_2$ which shows that \[ \span \{ Q_1, A_2 \} \subset \span \{ Q_1, Q_2 \} \] \item You can also use our results from dimension theory to make this less work too! \item This is probably easier to do symbolicaly using the formulas for projections. \end{enumerate} \end{enumerate} \end{exercise} % What we did was transformed the basis $\{A_1, A_2\}$ into the basis $\{Q_1, Q_2\}$. (I could have said this for the subspace $W_1$ too and said we transformed the basis $\{A_1\}$ to $\{Q_1\}$, but this would have been a bit silly since $A_1=Q_1$.) Now, we need to look at the space $W_3 = \span \{A_1, A_2, A_3\}$ (and of course, $W=W_3$). By the exercise, we already know that \[ W = \span \{ A_1, A_2, A_3 \} = \span\{ Q_1, Q_2, A_3 \} \] Our basis is now almost orthogonal, we just have to get rid of $A_3$ and put something orthogonal in its place. We do play the same game as above, first projecting to $W_2$: \begin{eqnarray*} \proj_{W_2}A_3 & = & \frac{A_3 \dotprod Q_1}{Q_1 \dotprod Q_1} Q_1 + \frac{A_3 \dotprod Q_2}{Q_2 \dotprod Q_2} Q_2 = \frac{43}{12}Q_1 + \frac{-33/4}{89/4}Q_2 \\ & = & \left[ \frac{635}{267}, \frac{1031}{89}, \frac{-833}{267}, \frac{734}{267} \right]^t \end{eqnarray*} Now, we compute the orthogonal vector: \[ Q_3 = A_3 - \proj_{W_2}A_3 = \left[ \frac{433}{267}, \frac{37}{89}, \frac{566}{267}, \frac{-200}{267} \right]^t \] \begin{exercise} Show that $\{ Q_1, Q_2, Q_3 \}$ is an orthogonal basis for $W$. Specifically, this means that you will need to show the following: \begin{enumerate}[(a)] \item These vectors are mutually orthogonal. \item These vectors are linearly independent. \\ (Explain how this relates to the statement in the text on page 210: ``orthogonal sets of vectors are independent.'') \item $\{ Q_1, Q_2, Q_3 \}$ spans $W$ (this is similar to problem~\ref{dim 2}). \end{enumerate} \end{exercise} \begin{exercise} Find an orthonormal basis for $W$. \\ (How can you turn our orthogonal basis into a normal basis?) \end{exercise} \subsection{Improvements} If you're like me, you get tired of working with fractions (unless maple is doing it for us). So, lets do things just a bit different. Above, we first found $Q_1$ (which was easy since it was just $A_1$). Then, we found $Q_2$, which had some fractions in it, but consider the vector $P_2 = 4Q_2$ (in other words, I got rid of the fractions). Then, in this case, $\{Q_1, P_2 \}$ are an orthogonal basis for $W_2$ and now we can continue from there just clearing fractions as we go. You do have to be careful though. Here are some important things to notice: \begin{enumerate} \item Only multiply the basis element by a scalar, not the projection! \item The basis you end up with will likely be different then the basis you end up without clearing fractions. But, they will still represent the same subspaces and still be orthogonal. \end{enumerate} \begin{exercise} Do this process to get a different orthogonal basis for $W$. \end{exercise} \section{Process} Here I outline the process just in case you didn't notice the main points above: \begin{enumerate} \item Start with an arbitrary basis for your vector space: \[ \{ A_1, \ldots, A_m \} \] \item Let $Q_1=A_1$. \item Let $W_1 = \span \{ Q_1 \} = \span\{A_1\}$. \item Project $A_2$ to $W_1$ (Fourier formula) \[ \proj_{W_1}A_2 \] \item Compute the orthogonal vector: \[ Q_2 = A_2 - \proj_{W_1}A_2 \] \item (Optional) Multily $Q_2$ by a scalar so that fractions are cleared. \item Let $W_2=\span\{Q_1, Q_2\} = \span\{A_1, A_2\}$. \item Project $A_3$ to $W_2$ (Fourier formula) \[ \proj_{W_2}A_3 \] \item Compute the orthogonal vector: \[ Q_3 = A_3 - \proj_{W_2}A_3 \] \item (Optional) Multily $Q_3$ by a scalar so that fractions are cleared. \item Keep going until you have used all the $A$'s. \end{enumerate} \section{Theorems and Definitions} Remember that theorems are here so that we don't have to work as hard. Here are some good theorems and definitions. \begin{definition} A set $\{Q_1, \ldots, Q_m\}$ is an \emph{orthogonal set} if each $Q_i$ is orthogonal to $Q_j$ for $i \neq j$ and none of the $Q_i$ is the zero vector. \end{definition} Notice that a set with the zero vector in it is not orthogonal. Excluding the orthogonal vector is important so that the next theorem is true: \begin{theorem} Any orthogonal set is linearly indenpendent. \end{theorem} \begin{theorem}[Fourier formula] Let $\{Q_1, \ldots Q_m\}$ be an orthogonal basis for $W \subset \R^n$. Let $B\in \R^n$ be an arbitrary vector. Then, the projection of $B$ to $W$ is given by \[ \proj_WB = \frac{B \dotprod Q_1}{Q_1 \dotprod Q_1} Q_1 + \cdots + \frac{B \dotprod Q_m}{Q_m \dotprod Q_m} Q_m \] And, the vector $B-\proj_WB$ is orthogonal to $W$. \end{theorem} \begin{theorem}[Gram-Schmidt] Let $\{A_1, \ldots, A_m\}$ be a basis for $W \subset \R^n$. The process above will always turn this basis into an orthogonal basis $\{Q_1, \ldots, Q_m\}$. Therefore, every subspace of $\R^n$ has an orthogonal basis. \end{theorem} \begin{theorem} Every subspace of $\R^n$ has an orthonormal basis. \end{theorem} \section{Back to Coordinates} Lets take our bases from before (which I will give labels to): \begin{eqnarray} A & = & \{ A_1, A_2, A_3 \} \\ Q & = & \{ Q_1, Q_2, Q_3 \} \\ N & = & \{ N_1, N_2, N_3 \} \end{eqnarray} where these vectors are: \[ A_1= \left[ \begin{array}{r} 1\\ 3\\ -1\\ 1 \end{array} \right] \quad A_2= \left[ \begin{array}{r} 2\\ -6\\ 0\\ 1 \end{array} \right] \quad A_3= \left[ \begin{array}{r} 4\\ 12\\ -1\\ 2 \end{array} \right] \] \[ Q_1= \left[ \begin{array}{r} 1\\ 3\\ -1\\ 1\end{array} \right] \quad Q_2= \left[ \begin{array}{r} 13\\ -9\\ -5\\ -9\end{array} \right] \quad Q_3= \left[ \begin{array}{r} 433\\ 111\\ 566 \\ -200\end{array} \right] \] \[ N_1= \left[ \begin{array}{r} \frac{1}{\sqrt{12}}\\ \frac{3}{2\sqrt{3}}\\ -\frac{1}{2\sqrt{3}}\\ \frac{1}{2\sqrt{3}} \end{array} \right] \quad N_2= \left[ \begin{array}{r} \frac{13}{2\sqrt{89}}\\ -\frac{9}{2\sqrt{89}}\\ -\frac{5}{2\sqrt{89}}\\ -\frac{9}{2\sqrt{89}}\end{array} \right] \quad N_3= \left[ \begin{array}{r} \frac{433}{\sqrt{560166}}\\ \frac{111}{\sqrt{560166}}\\ \frac{566}{\sqrt{560166}} \\ -\frac{200}{\sqrt{560166}} \end{array} \right] \] OK, these aren't the $Q$'s that I had before, but you should be able to see what I did and you should be able to see that they are an orthogonal basis. The $N$ basis is derived from the $Q$ basis, but it is orthonormal and a bit ugly. Lets talk about coordinates. The book uses the words \emph{skewed coordinate vector with respect to the given basis}. I will rephrase Theorem~2 on page 222 in a different language. Consider the vector: \begin{eqnarray} v & = & 4A_1 + 3A_2 - A_3 \nonumber \\ v & = & \left[ \begin{array}{r} 6\\ -18\\ -3\\ 5 \end{array} \right] \label{v in R4} \end{eqnarray} $v$ is clearly in $W$. The way I have been writing these coordinates is: \begin{equation} \label{v in A} v = [4,3,-1]^t_A \end{equation} Can we write this vector in terms of the $Q$-coordinates? In other words, can we find $a,b,c$ so that \[ v = aQ_1 + bQ_2 + cQ_3 \] Sure, we can solve equations, which turns into the following matrix to reduce: \[ \left[ \begin{array}{rrrr} 1 & 13 & 433 & 6 \\ 3 & -9 & 111 & -18 \\ -1 & -5 & 566 & -3 \\ 1 & -9 & -200 & 5 \end{array} \right] \longrightarrow \left[ \begin{array}{rrrr} 1 & 0 & 0 & -\frac{10}{3} \\ 0 & 1 & 0 & \frac{75}{89} \\ 0 & 0 & 1 & -\frac{1}{267} \\ 0 & 0 & 0 & 0 \end{array} \right] \] What this means is that \[ v = -\frac{10}{3}Q_1 + \frac{75}{89}Q_2 -\frac{1}{267}Q_3 \] or \begin{equation} \label{v in Q} v = \left[-\frac{10}{3}, \frac{75}{89}, -\frac{1}{267}\right]^t_Q \end{equation} Similarly, we can try to find $v$ in $N$ coordinates. You will certainly want to use Maple here! Here is what I got: \begin{eqnarray} v & = & -\frac{20}{\sqrt{3}}N_1 + \frac{150}{\sqrt{89}}N_2 - \frac{\sqrt{560166}}{267}N_2 \nonumber \\ v & = & \left[ -\frac{20}{\sqrt{3}}, \frac{150}{\sqrt{89}}, - \frac{\sqrt{560166}}{267} \right]^t_N \label{v in N} \end{eqnarray} Here are some things to notice. \begin{enumerate} \item We can write $v$ in many different ways: \begin{enumerate} \item As a vector in $\R^4$ (this is equation~(\ref{v in R4})) \item As a vector in $\R^3$ in $A$-coordinates (this is equation~(\ref{v in A})) (lets call this vector $v_A$) \item As a vector in $\R^3$ in $Q$-coordinates (this is equation~(\ref{v in Q})) (lets call this vector $v_Q$) \item As a vector in $\R^3$ in $N$-coordinates (this is equation~(\ref{v in N})) (lets call this vector $v_N$) \end{enumerate} \item We can easily compute the norm of all the vectors above. In fact, using maple I get: \begin{enumerate} \item \[ v \dotprod v = 394 \] \item \[ v_A \dotprod v_A = 26 \] \item \[ v_Q \dotprod v_Q = \frac{842726}{71289} \] \item \[ v_N \dotprod v_N = 394 \] \end{enumerate} \end{enumerate} (The norms of the vectors are the square roots of these numbers.) Wow!! Two of these dot products are the same! In fact, this is part of Theorem~2 on page 222 of the text: \begin{theorem} Let $W$ be a subspace of $\R^n$ with orthonormal basis \[ N = \{ N_1, \ldots, N_m \} \] Let $v \in W$. Then, we can find the coordinates of $v$ with repsect to the basis $N$. Let $v_N$ denote the vector in $\R^m$ representing the coordinates of $v$ with respect to $N$. Then \[ v \dotprod v = v_N \dotprod v_N \] \end{theorem} In fact, you should be able to convince yourself that if I give you any vector in $W$, you can come up with the $A$ or $Q$ or $N$ coordinates of this vector. In fact, this transformation from ``normal'' coordinate to these other coordinates defines a linear transformation from $W$ to $\R^3$. Perhaps a better way to think of this is to think about transforming between coordinates systems in $W$. Here are some exercises: \begin{exercise} \begin{enumerate}[(a)] \item Let $v=[1,0,0]_A^t$. \begin{enumerate} \item What is $v$ in standard coordinates for $\R^4$? \item What is $v$ in $Q$-coordinates for $W$? \item What is $v$ in $N$-coordinates for $W$? \end{enumerate} \item Let $v=[0,1,0]_A^t$. \begin{enumerate} \item What is $v$ in standard coordinates for $\R^4$? \item What is $v$ in $Q$-coordinates for $W$? \item What is $v$ in $N$-coordinates for $W$? \end{enumerate} \item Let $v=[0,0,1]_A^t$. \begin{enumerate} \item What is $v$ in standard coordinates for $\R^4$? \item What is $v$ in $Q$-coordinates for $W$? \item What is $v$ in $N$-coordinates for $W$? \end{enumerate} \item The transformation from $A$-coordinates in $W$ to $Q$ coordinates in $W$ is a transformation from $\R^3$ to $\R^3$. Find the matrix of this tranformation (this should be interpreting the above work). \end{enumerate} \end{exercise} \newpage \section{Maple Homework} Here are some maple exercises for you. The due date will be announce in class. (The other exercises above are not due, but they should certainly help you do these!) First, start with the following vectors: \[ A_1 = \left[ \begin{array}{r} 1 \\ 2 \\ 4 \\ -5 \\ 2 \end{array} \right] \quad A_2 = \left[ \begin{array}{r} 3 \\ 2 \\ 4 \\ -5 \\ 2 \end{array} \right] \quad A_3 = \left[ \begin{array}{r} 5 \\ 4 \\ 4 \\ -5 \\ -2 \end{array} \right] \quad A_4 = \left[ \begin{array}{r} 1 \\ 0 \\ 4 \\ 2 \\ 0 \end{array} \right] \] \begin{maple} \begin{enumerate}[(a)] \item Show that the set $A=\{A_1, A_2, A_3, A_4\}$ is a basis for a subspace, $W \subset \R^5$. \item What is $\dim W$? \end{enumerate} \end{maple} \begin{maple} Using Gram-Schmidt, transform the basis $A$ into an orthogonal basis $Q$. \end{maple} \begin{maple} Tranform your basis $Q$ into an orthonormal basis $N$. \end{maple} \begin{maple} Take the vector $v=2A_1-3A_2+4A_3-5A_4$. \begin{enumerate}[(a)] \item Write $v$ in standard coordinates for $\R^5$. \item Write $v$ in $A$-coordinates. (So this should be a vector in $\R^4$.) \item Write $v$ in $Q$-coordinates. (So this should be a vector in $\R^4$.) \item Write $v$ in $N$-coordinates. (So this should be a vector in $\R^4$.) \item Find the norms of the vectors above. Explain how this verifies Theorem~2 of page~222. \end{enumerate} \end{maple} \begin{maple} \begin{enumerate}[(a)] \item Compute the change of coordinate matrix to transform from $A$ coordinates to $Q$ coordinates. \item Using your change of coordinate matrix that you just found, verify what you found above for the $Q$-coordinates for $v$. \end{enumerate} \end{maple} \section{Isomorphisms} I will talk about this later in class and try to write something up later too. \end{document}