SVD

name: mid
layout: true
class: center, middle 
---
<br/>
<br/>
<br/>
## How to Use iPhone with 32G Storage?
<br/>
<br/>
<br/>
![center](http://cmb.oss-cn-qingdao.aliyuncs.com/2017-09-14-iphone7.png)
.footnote[powered by <a href="https://github.com/gnab/remark"><font color=blue>remark.js</font></a>]

---
##  BLACK TECHNOLOGY DIMENSIONALITY REDUCTION!

---
name:inverse
layout: true
class: center, middle, inverse
---
# Singular Value Decomposition
## ChengMingbo
## 2017-09-14
---
layout: false
# SVD ABC
- Matrix Linear Transformation
- EigenVector & EigenVaule
- Singular Value Decomposition 
- Other Applications 
- Summary

---
template: inverse
# Linear Transformation

---
layout:false
## Essense of Linear Transformation

.vertmid[
<p>
$$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$$
</p>
]
???
[2 0;0 2] 相当于把一个坐标翻了两倍

---
## Essense of Linear Transformation
<p>
$$\vec{x}=\begin{bmatrix}1 \\3\end{bmatrix}$$
</p>

---
## Essense of Linear Transformation
<p>
$$A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}$$
</p>

---
## Essense of Linear Transformation

<p>
$$\vec{y}=
\begin{bmatrix}
5 \\
2
\end{bmatrix}$$
</p>

---
## Essense of Linear Transformation
.vertmid[
<p>
$$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$$
<br/>
$$A\vec{x}= \begin{bmatrix}2 & 1\\ -1 & 1\end{bmatrix}\begin{bmatrix}1\\3\end{bmatrix}=\begin{bmatrix}5 \\2\end{bmatrix}$$
</p>
]
---
template: inverse

## EigenVector & EigenVaule

---
## EigenVector & EigenVaule

### Multiplication 100 times

<br/>
<p>
$$\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\cdots \begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}=$$
</p>

---
## EigenVector & EigenVaule
.vertmid[
<p>
$$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$$
<br/>
$$\begin{bmatrix}3 & 1 \\ 0& 2\end{bmatrix}\begin{bmatrix}1 \\2\end{bmatrix}=?$$
</p>
]

---
## EigenVector & EigenVaule
<br/>
<br/>

<p>
$$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$$

$$\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}
\begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}=
2\begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}\qquad\quad\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}
\begin{bmatrix}1 \\ 0\end{bmatrix}=
3\begin{bmatrix}1 \\ 0\end{bmatrix}
$$

$$\vec{x_1}=\begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}\qquad\, \vec{x_2}=\begin{bmatrix}1 \\ 0\end{bmatrix}$$
$$\lambda_1=2\qquad\qquad\qquad\lambda_2=3$$
</p>

---
## EigenVector & EigenVaule
<p>
$$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$$
</p>

---
## EigenVector & EigenVaule
.vertmid[
<p>
$$AQ=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}
\begin{bmatrix}
\frac{-1}{\sqrt(2)}&1\\
\frac{1}{\sqrt(2)}&0
\end{bmatrix}=
\begin{bmatrix}
\frac{-1}{\sqrt(2)}&1 \\ \frac{1}{\sqrt(2)}&0
\end{bmatrix}
\begin{bmatrix}
2 & 0\\
0 & 3
\end{bmatrix}
$$
<br/>

$$AQ=Q\Lambda$$
$$A = Q\Lambda Q^{-1}$$

</p>
]
---
## EigenVector & EigenVaule
<br/>
<p> 
$$A\vec{x_1}=\lambda\vec{x_1}\\
A\vec{x_2}=\lambda\vec{x_2}\\
\,\qquad\vdots\\
A\vec{x_k}=\lambda\vec{x_k}$$
</p>

<br/>
<p>
$$Q=
\begin{bmatrix}
	x_{11}& x_{21} &\cdots x_{k1}&\\
    x_{12}& x_{22} &\cdots x_{k2}&\\
    &\vdots&&\\
    x_{1m}& x_{22} &\cdots x_{km}&
\end{bmatrix}
\qquad\Lambda=
\begin{bmatrix}
\lambda_1 & 0 & \cdots&0\\
0 &\lambda_2&\cdots&0\\
\vdots&\vdots&\ddots\\
0&\cdots&\cdots&\lambda_k
\end{bmatrix}$$
</p>

---
## EigenVector & EigenVaule
.vertmid[
$$A = Q\Lambda Q^{-1}$$
]
---
## EigenVector & EigenVaule
### Multiplication 100 times 
<br/>
<p>
$$\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\cdots \begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}=$$
$$AAA\cdots A = Q\Lambda Q^{-1}Q\Lambda Q^{-1}Q\Lambda Q^{-1}\cdots Q\Lambda Q^{-1}$$
$$AAA\cdots A =Q\Lambda\Lambda\cdots \Lambda Q^{-1}=
Q\begin{bmatrix}2^{100} & 0 \\\\0 & 3^{100}\end{bmatrix}Q^{-1}
$$
</p>

---
template: inverse

## Singular Value Decomposition(SVD)

---
## Singular Value Decomposition(SVD)
<br/>
<div align=center>
  <img src="http://cmb.oss-cn-qingdao.aliyuncs.com/2017-09-14-rectangle1.png" height=80% width=80% />
</div>

---
## Singular Value Decomposition(SVD)
<p>
$$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$$
</p>

???
What if we change coordinate from red to blue?

---
## Singular Value Decomposition(SVD)

---
## Singular Value Decomposition(SVD)

---
## Singular Value Decomposition(SVD)

<p>
$$\vec{v_1} \vec{v_2} \vec{v_3},...\vec{v_n} \qquad\rightarrow \qquad \vec{u_1},\vec{u_2},\vec{u_3},...\vec{u_n}$$
$$\qquad\qquad\qquad\qquad\qquad\ \,\sigma_1,\sigma_2, \sigma_3,...\sigma_n$$
</p>

---
## Singular Value Decomposition(SVD)
<br/>
<p>
$$ A \vec{v_1}=\sigma_1 \vec{u_1}$$
$$\vdots$$
$$ A \vec{v_j}=\sigma_j \vec{u_j}$$

$$\begin{bmatrix}
\\A\\\\\end{bmatrix}
\begin{bmatrix}\\ \vec{v_1},\vec{v_2},\cdots,\vec{v_n}\\\\\end{bmatrix}
=\begin{bmatrix}\\ \vec{u_1}, \vec{u_2},\cdots,\vec{u_n}\\\\ \end{bmatrix}
\begin{bmatrix}
\sigma_1 & 0 & \cdots & 0 \\
0 & \sigma_2 & \cdots & 0 \\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \sigma_n 
\end{bmatrix}$$
</p>

---
## Singular Value Decomposition(SVD)
<br/>
<p>
$$\begin{bmatrix}\\A\\\\\end{bmatrix}
\begin{bmatrix}\\ \vec{v_1},\vec{v_2},\cdots,\vec{v_n}\\\\\end{bmatrix}
=\begin{bmatrix}\\ \vec{u_1}, \vec{u_2},\cdots,\vec{u_n}\\\\ \end{bmatrix}
\begin{bmatrix}
\sigma_1 & 0 & \cdots & 0 \\
0 & \sigma_2 & \cdots & 0 \\
\vdots & \vdots & \ddots & \vdots \\
0 & 0 & \cdots & \sigma_n
\end{bmatrix}$$
$$\Uparrow$$
</p>

$$C^{m\times n}\qquad\quad C^{n\times n}\qquad\qquad C^{m\times n}\qquad \qquad \qquad C^{n\times n}$$

<p>
<br/>
$$A_{m\times n}V_{n\times n} = \hat{U}_{m\times n}\hat{\Sigma}_{n\times n}$$

</p>

???
V^{-1}=V^T      V^{-1}=V^T

---
## Singular Value Decomposition(SVD)
.vertmid[
<p>
$$A_{m\times n} = \hat{U}_{m\times n}\hat{\Sigma}_{n\times n}V^T_{n\times n}$$
</p>
]
---
## Singular Value Decomposition(SVD)

---
## Singular Value Decomposition(SVD)
.vertmid[
<p>
$$A_{m\times n} = U_{m\times m}\Sigma_{m\times n}V^T_{n\times n}$$
</p>
]
---
template: mid

## How to resolve these three variables？

---
template: mid
<p>
$$(AA^T)U=U\Sigma^2 $$
<br/>
$$(A^TA)V^T=V^T\Sigma^2$$
</p>

---
## Compression

---
## Compression 
<br/>
<br/>
<div align=center>
  <img src="http://cmb.oss-cn-qingdao.aliyuncs.com/2017-04-29-image.gif" height=80% width=80% />
</div>

---
## Other Applications 
- Recommendation 
- noise filtering 
- Compression

???
netflix

---
## Summary

- Matrix Linear Transformation
- EigenVector & EigenVaule
- Singular Value Decomposition 
- Other Applications

---

## Reference
<div class="reference">
<ul >
<li>https://www.youtube.com/watch?v=EokL7E6o1AE</li>
<li>https://www.youtube.com/watch?v=cOUTpqlX-Xs</li>
<li>https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw</li>
<li> https://itunes.apple.com/cn/itunes-u/linear-algebra/id354869137</li>
<li> http://www.ams.org/samplings/feature-column/fcarc-svd</li>
<li> https://www.cnblogs.com/LeftNotEasy/archive/2011/01/19/svd-and-applications.html</li>
</ul>
</div>

---
template: mid
# Thanks!
# Q&A