Linear Algebra (I)

Motivation

When I was learning Linear Algebra in University, I found it was hard to understand what would applications of this course. I remember that we learned determinant on Chapter 1, then we learned a lot of other concept such as Rank and EigenValue. I could only memorize one after another theorems without knowing why these happened to appear this way. However, when I was learning matrices from lectures taught by Gilbert Strang, I found every conclusion is so nature that I could easily understand and associate a bunch of concepts with each other. I'd like to conclude what I had learned from him as well as hoping that someone can learn from that.

Why this subject exists?

First of all, what is the origin of linear algebra? Why we make the world increasingly complicated? The truth is that the purpose of mathmatics is to make the world simpler, not the opposite, Linear Algebra as well. For example, we want to solve equations with a lot of variables, the original way is to write all the parameter out： $$\{\begin{array}{l}3x+4y+6z=0\\ \\ 2x+3y+4z=24\\ \\ x+y+z=15\end{array}$$

If there are only 3 parameters, the problems is not so hard for us to handle. But what if there are 100 parameters? When we transfer the equation to matrices, the outcome becomes so grateful: $$\begin{array}{r}\left[\begin{array}{ccc}3& 4& 6\\ \\ 2& 3& 4\\ \\ 1& 1& 1\end{array}\right]\left[\begin{array}{c}x\\ \\ y\\ \\ z\end{array}\right]=\left[\begin{array}{c}0\\ \\ 24\\ \\ 15\end{array}\right]\end{array}$$ ### Solve the equation group Now, we try to solve the problems, our instinction is to draw the three planes, under most circumstance, there will be a cross point, octave code and the plot is as followed:

octave:1> x=linspace(-20, 20, 100);
octave:2> [xx, yy] = meshgrid(x,y);
octave:3> z = 15-xx-yy;
octave:4> mesh(xx, yy, z);
octave:5> l = (-3*xx-4*yy)./6;
octave:6> mesh(xx,yy, l);
octave:7> t = (24-(2*xx+3*yy))./4;
octave:8> mesh(xx,yy,z);
octave:9> hold on;
octave:11> mesh(xx,yy,t);
octave:12> mesh(xx,yy,l);
octave:13> xlabel("x")
octave:14> ylabel("y")
octave:15> zlabel("z")

The answer of this equation group is that $$\begin{array}{r}\left[\begin{array}{c}x\\ \\ y\\ \\ z\end{array}\right]=\left[\begin{array}{c}-18\\ \\ 72\\ -39\end{array}\right]\end{array}$$x} \end{align}

Another perspective

We can find another perspective to solve this equation: see the equation as a combination of columns, that means: $$\begin{array}{r}x\text{*}\left[\begin{array}{c}3\\ \\ 2\\ \\ 1\end{array}\right]+y\text{*}\left[\begin{array}{c}4\\ \\ 3\\ \\ 1\end{array}\right]+z\text{*}\left[\begin{array}{c}6\\ \\ 4\\ \\ 1\end{array}\right]=\left[\begin{array}{c}0\\ \\ 24\\ \\ 15\end{array}\right]\end{array}$$ Now, we can acquire the new vector through three vectors' combination, from the plot blow demonstrates that we need to find a combination of red&blue&black vectors to generate another vector(the green one). Obviously, $x=-18,y=72,z=-39$ is the only solution of the combination problem.

octave:1> a = quiver3(0, 0, 0, 3,2,1);
octave:2> hold on;
octave:3> b = quiver3(0, 0, 0, 4,3,1);
octave:4> c = quiver3(0, 0, 0, 6,4,1);
octave:5> set(b, "Color", "r");
octave:6> set(c, "Color", "k");
octave:7> set(a, "maxheadsize", 0.02);
octave:8> set(b, "maxheadsize", 0.02);
octave:9> set(c, "maxheadsize", 0.02);
octave:10> d = quiver3(0, 0, 0, 0,24,15);
octave:11> set(d, "Color", "g");
octave:12> set(d, "maxheadsize", 0.02);

How to solve equations using program, and is there always an answer of equations? An easy way is to do elimination of these there equations. What now I want to do right now is to change these equation as follows: $$\{\begin{array}{l}3x+4y+6z=0\\ \\ (1/3)y+0z=24\\ \\ -z=39\end{array}$$

Step1, calclate z, step2, calclate y, step3, calclate x. We first leave a variables z alone with an answer in the third equation. Then, we substitute z to the second equation, and get y. At last we substitute x and y to the first equation and get x. if we represent above through matrices language, it is like this: $$\begin{array}{r}\left[\begin{array}{ccc}3& 4& 6\\ \\ 0& 1/3& 0\\ \\ 0& 0& -1\end{array}\right]\left[\begin{array}{c}x\\ \\ y\\ \\ z\end{array}\right]=\left[\begin{array}{c}0\\ \\ 24\\ \\ 39\end{array}\right]\end{array}$$ The triangle at bottom-left side is all zeros which ensure we can solve equations one by one. The process is named back-substitution. ### Additional thinkings But what if the equation group looks like: $$\begin{array}{r}\left[\begin{array}{ccc}1& 2& 3\\ \\ 1& 2& 3\\ \\ 1& 2& 3\end{array}\right]\left[\begin{array}{c}x\\ \\ y\\ \\ z\end{array}\right]=\left[\begin{array}{c}5\\ \\ 6\\ \\ 7\end{array}\right]or\left[\begin{array}{cc}3& 4\\ \\ 4& 5\\ \\ 5& 6\\ \\ 6& 7\\ \\ 7& 9\end{array}\right]\left[\begin{array}{c}x\\ \\ y\end{array}\right]=\left[\begin{array}{c}5\\ \\ 5\\ \\ 5\\ \\ 5\\ \\ 5\end{array}\right]\end{array}$$
Next time we will talk about LU decomposition and Rank of a matrix.