SVD

How to Use iPhone with 32G Storage?

1 / 38

BLACK TECHNOLOGY DIMENSIONALITY REDUCTION!2 / 38

Singular Value DecompositionChengMingbo2017-09-143 / 38

SVD ABCMatrix Linear Transformation
EigenVector & EigenVaule
Singular Value Decomposition 
Other Applications 
Summary 
4 / 38

Linear Transformation5 / 38

Essense of Linear Transformation

$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$

6 / 38

[2 0;0 2] 相当于把一个坐标翻了两倍

Essense of Linear Transformation

$\vec{x}=\begin{bmatrix}1 \\3\end{bmatrix}$

7 / 38

Essense of Linear Transformation

$A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}$

8 / 38

Essense of Linear Transformation

$\vec{y}= \begin{bmatrix} 5 \\ 2 \end{bmatrix}$

9 / 38

Essense of Linear Transformation

$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$
$A\vec{x}= \begin{bmatrix}2 & 1\\ -1 & 1\end{bmatrix}\begin{bmatrix}1\\3\end{bmatrix}=\begin{bmatrix}5 \\2\end{bmatrix}$

10 / 38

EigenVector & EigenVaule11 / 38

EigenVector & EigenVaule

Multiplication 100 times

$\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\cdots \begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}=$

12 / 38

EigenVector & EigenVaule

$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$
$\begin{bmatrix}3 & 1 \\ 0& 2\end{bmatrix}\begin{bmatrix}1 \\2\end{bmatrix}=?$

13 / 38

EigenVector & EigenVaule

$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$ $\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix} \begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}= 2\begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}\qquad\quad\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix} \begin{bmatrix}1 \\ 0\end{bmatrix}= 3\begin{bmatrix}1 \\ 0\end{bmatrix}$ $\vec{x_1}=\begin{bmatrix}\frac{-1}{\sqrt(2)} \\ \frac{1}{\sqrt(2)}\end{bmatrix}\qquad\, \vec{x_2}=\begin{bmatrix}1 \\ 0\end{bmatrix}$ $\lambda_1=2\qquad\qquad\qquad\lambda_2=3$

14 / 38

EigenVector & EigenVaule

$A=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix}$

15 / 38

EigenVector & EigenVaule

$AQ=\begin{bmatrix}3 & 1 \\0& 2\end{bmatrix} \begin{bmatrix} \frac{-1}{\sqrt(2)}&1\\ \frac{1}{\sqrt(2)}&0 \end{bmatrix}= \begin{bmatrix} \frac{-1}{\sqrt(2)}&1 \\ \frac{1}{\sqrt(2)}&0 \end{bmatrix} \begin{bmatrix} 2 & 0\\ 0 & 3 \end{bmatrix}$
$AQ=Q\Lambda$ $A = Q\Lambda Q^{-1}$

16 / 38

EigenVector & EigenVaule

$A\vec{x_1}=\lambda\vec{x_1}\\ A\vec{x_2}=\lambda\vec{x_2}\\ \,\qquad\vdots\\ A\vec{x_k}=\lambda\vec{x_k}$

$Q= \begin{bmatrix} x_{11}& x_{21} &\cdots x_{k1}&\\ x_{12}& x_{22} &\cdots x_{k2}&\\ &\vdots&&\\ x_{1m}& x_{22} &\cdots x_{km}& \end{bmatrix} \qquad\Lambda= \begin{bmatrix} \lambda_1 & 0 & \cdots&0\\ 0 &\lambda_2&\cdots&0\\ \vdots&\vdots&\ddots\\ 0&\cdots&\cdots&\lambda_k \end{bmatrix}$

17 / 38

EigenVector & EigenVaule

$A = Q\Lambda Q^{-1}$

18 / 38

EigenVector & EigenVaule

Multiplication 100 times

$\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}\cdots \begin{bmatrix}3& 1 \\0 & 2\end{bmatrix}=$ $AAA\cdots A = Q\Lambda Q^{-1}Q\Lambda Q^{-1}Q\Lambda Q^{-1}\cdots Q\Lambda Q^{-1}$ $AAA\cdots A =Q\Lambda\Lambda\cdots \Lambda Q^{-1}= Q\begin{bmatrix}2^{100} & 0 \\\\0 & 3^{100}\end{bmatrix}Q^{-1}$

19 / 38

Singular Value Decomposition(SVD)20 / 38

Singular Value Decomposition(SVD)

21 / 38

Singular Value Decomposition(SVD)

$\vec{x}=\begin{bmatrix}1\\3\end{bmatrix}\qquad A=\begin{bmatrix}2 & 1 \\ -1 & 1 \end{bmatrix}\qquad\vec{y}=A\vec{x}$

22 / 38

What if we change coordinate from red to blue?

Singular Value Decomposition(SVD)

23 / 38

Singular Value Decomposition(SVD)

24 / 38

Singular Value Decomposition(SVD)

$\vec{v_1} \vec{v_2} \vec{v_3},...\vec{v_n} \qquad\rightarrow \qquad \vec{u_1},\vec{u_2},\vec{u_3},...\vec{u_n}$ $\qquad\qquad\qquad\qquad\qquad\ \,\sigma_1,\sigma_2, \sigma_3,...\sigma_n$

25 / 38

Singular Value Decomposition(SVD)

$A \vec{v_1}=\sigma_1 \vec{u_1}$ $\vdots$ $A \vec{v_j}=\sigma_j \vec{u_j}$ $\begin{bmatrix} \\A\\\\\end{bmatrix} \begin{bmatrix}\\ \vec{v_1},\vec{v_2},\cdots,\vec{v_n}\\\\\end{bmatrix} =\begin{bmatrix}\\ \vec{u_1}, \vec{u_2},\cdots,\vec{u_n}\\\\ \end{bmatrix} \begin{bmatrix} \sigma_1 & 0 & \cdots & 0 \\ 0 & \sigma_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \sigma_n \end{bmatrix}$

26 / 38

Singular Value Decomposition(SVD)

$\begin{bmatrix}\\A\\\\\end{bmatrix} \begin{bmatrix}\\ \vec{v_1},\vec{v_2},\cdots,\vec{v_n}\\\\\end{bmatrix} =\begin{bmatrix}\\ \vec{u_1}, \vec{u_2},\cdots,\vec{u_n}\\\\ \end{bmatrix} \begin{bmatrix} \sigma_1 & 0 & \cdots & 0 \\ 0 & \sigma_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \sigma_n \end{bmatrix}$ $\Uparrow$

$C^{m\times n}\qquad\quad C^{n\times n}\qquad\qquad C^{m\times n}\qquad \qquad \qquad C^{n\times n}$

$A_{m\times n}V_{n\times n} = \hat{U}_{m\times n}\hat{\Sigma}_{n\times n}$

27 / 38

V^{-1}=V^T V^{-1}=V^T

Singular Value Decomposition(SVD)

$A_{m\times n} = \hat{U}_{m\times n}\hat{\Sigma}_{n\times n}V^T_{n\times n}$

28 / 38

Singular Value Decomposition(SVD)

29 / 38

Singular Value Decomposition(SVD)

$A_{m\times n} = U_{m\times m}\Sigma_{m\times n}V^T_{n\times n}$

30 / 38

How to resolve these three variables？31 / 38

$(AA^T)U=U\Sigma^2$
$(A^TA)V^T=V^T\Sigma^2$

32 / 38

Compression

33 / 38

Compression

34 / 38

Other ApplicationsRecommendation 
noise filtering 
Compression
35 / 38

netflix

SummaryMatrix Linear Transformation
EigenVector & EigenVaule
Singular Value Decomposition 
Other Applications 
36 / 38

Reference
https://www.youtube.com/watch?v=EokL7E6o1AE
https://www.youtube.com/watch?v=cOUTpqlX-Xs
https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw
 https://itunes.apple.com/cn/itunes-u/linear-algebra/id354869137
 http://www.ams.org/samplings/feature-column/fcarc-svd
 https://www.cnblogs.com/LeftNotEasy/archive/2011/01/19/svd-and-applications.html

37 / 38

Thanks!Q&A38 / 38

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help

How to Use iPhone with 32G Storage?

BLACK TECHNOLOGY DIMENSIONALITY REDUCTION!

Singular Value Decomposition

ChengMingbo

2017-09-14

SVD ABC

Linear Transformation

Essense of Linear Transformation

Essense of Linear Transformation

Essense of Linear Transformation

Essense of Linear Transformation

Essense of Linear Transformation

EigenVector & EigenVaule

EigenVector & EigenVaule

Multiplication 100 times

EigenVector & EigenVaule

EigenVector & EigenVaule

EigenVector & EigenVaule

EigenVector & EigenVaule

EigenVector & EigenVaule

EigenVector & EigenVaule

EigenVector & EigenVaule

Multiplication 100 times

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

Singular Value Decomposition(SVD)

How to resolve these three variables？

Compression

Compression

Other Applications

Summary

Reference

Thanks!

Q&A

BLACK TECHNOLOGY DIMENSIONALITY REDUCTION!

Help