Clustering

name: quotation
layout: true
class: center, middle, inverse
---

# <center>An Overview on Cluster Analysis</center>

### By Mingbo Cheng
### 2019-1-17
.footnote[powered by <a href="https://github.com/gnab/remark"><font color=yellow>remark.js</font></a>]

---
layout: false

# <center>OUTLINE</center>

- Where are Clustering Applied
- What kinds of Clustering Methods
- How to Evaluate
- What Makes Better Performance
- Conclusion
- References

---
# <center>OUTLINE</center>

- **Where are Clustering Applied**
- What kinds of Clustering Methods
- How to Evaluate
- What Makes Better Performance
- Conclusion
- References

---
## Facebook Sentiment Analysis through LDA

<br/>

<div align=center>
<img src="http://cmb.oss-cn-qingdao.aliyuncs.com/facebook-clustering.png" width=60% height=60%/>
</div>
---
### Google News Clustering
<br/>
<br/>

???
google paper in 2007 talked about 
MinHashing is a probabilistic clustering method 
Google news personalization: scalable online collaborative filtering

---
### Anomaly Detection
<br/>

<br/>

<font size=1 color=green>https://www.semanticscholar.org/paper/Traffic-Anomaly-Detection-Using-K-Means-Clustering-M%C3%BCnz-Li/634e2f1a20755e7ab18e8e8094f48e140a32dacd</font>

???
1. paypal account thieves behaviors different from
2. aircraft	engine, detect whether an engine good enough
---

#### Clustering cancer gene expression data: A comparative study

???
2008 
BMC Bioinformatics 
7 clustering algorithms
shared nearest-neighbor algorithm(SNN)
nearest neighbors (NN)
spectral clustering(SPC)  similarity matrix  dimensionality reduction
mixture of multivariate Gaussians(FMG)
average linkage(AL)
complete linkage(CL)
single linkage (SL)
4 proximity measures
Pearson's Correlation coefficient(P)
Cosine (C)
Spearman's correlation coefficient(C)
Euclidean Distance(E) -> four version, Z0 Z1 Z2 Z3
Genetic conditions(can be seen as illness), Down's

---

# <center>OUTLINE</center>

- Where are Clustering Applied
- **What kinds of Clustering Methods**
- How to Evaluate
- What Makes Better Performance
- Conclusion
- References

---
## Typical cluster models

- Connectivity models

<font size=1 color=green>https://www.datanovia.com/en/courses/hierarchical-clustering-in-r-the-essentials/</font>

???
connectivity models:
https://www.datanovia.com/en/courses/hierarchical-clustering-in-r-the-essentials/

---
## Typical cluster models

- Centroid models

<font size=1 color=green>https://www.mathworks.com/matlabcentral/mlc-downloads/downloads/submissions/26182/versions/11/screenshot.jpg</font>

???
kmeans
---
## Typical cluster models

- Distribution models
<br/>

---
## Typical cluster models

- Density models
<br/>

<br/>
<font size=1 color=green>https://medium.com/@elutins/dbscan-what-is-it-when-to-use-it-how-to-use-it-8bd506293818</font>

---
## Typical cluster models

- Subspace models

<br/>

<font size=1 color=green>http://cw.fel.cvut.cz/old/_media/courses/a6m33bin/biclustering.pdf</font>

---
## Typical cluster models

- Subspace models

<font size=1 color=green>http://cw.fel.cvut.cz/old/_media/courses/a6m33bin/biclustering.pdf</font>

???
different time point, if the gene express
---
## Typical cluster models

- Graph-based models

<div align=center>
<img src="http://cmb.oss-cn-qingdao.aliyuncs.com/graph_based_model.png" width=50% height=50%/>
</div>
<br/>
<font size=1 color=green>https://en.wikipedia.org/wiki/Clique_(graph_theory)</font>
---
## Typical cluster models

- Neural models

<div align=center>
<img src="http://cmb.oss-cn-qingdao.aliyuncs.com/SOM.gif" width=50% height=50%/>
</div>
<br/>
<font size=1 color=green>http://www.lohninger.com/helpcsuite/kohonen_network_-_background_information.htm</font>

???
tutorial: http://www.pitt.edu/~is2470pb/Spring05/FinalProjects/Group1a/tutorial/som.html

---
# <center>OUTLINE</center>

- Where are Clustering Applied
- What kinds of Clustering Methods
- **How to Evaluate**
- What Makes Better Performance
- Conclusion
- References

---
## Cluster evaluation

- Internal Evaluation
    - Davies-Bouldin index
    - Dunn index
- External Evaluation
	- Purity
	- Classification Criteria

???
internal: based on data itself
1. Devies-Bouldin index: centroids far from each other, inner the cluster, point is near to each other Smaller, better
2. Dunn index density and well-separated Higher, better
::::::::::::::::::::::::::::::::::::::::::::::::
external: data not used for clustering, labeled data
Purity, each point is a cluster, imbalance data problem
Classificatioin Criteria: Precision, Recall __ FMeasure
TRUE Positive False Positive True Negative False Negative

---
# <center>OUTLINE</center>

- Where are Clustering Applied
- What kinds of Clustering Methods
- How to Evaluate
- **What Makes Better Performance**
- Conclusion
- References

---
## Data Preprocessing

- Dimensionality Reduction
- Noise Reduction
- Standardization
- Scaling
- Ranking

???
Dimensionality Reduction: PCA, Non-negative matrix factorization(NMF)
noise reduction: LDA, high-frequency words. THE may offer us no information
Stardardization: zero mean, then unit variance
Scaling: Hight and weight, S or L size shirt Different dimension contribute different 
Ranking: Function(x_i)

---
## Similarity Measures

- Minkowski Metric

<font size=3>      q=1: Manhattan distance<br/>       q=2: Euclidean distance </font>

$$d(x, y) = L\_q(x, y) = \sqrt[q]{\sum\_{i=1}^n(x\_i-y\_i)^q}$$

- Kullback-Leibler Divergence
- Cosin

???
included angle
---

# <center>OUTLINE</center>

- Where are Clustering Applied
- What kinds of Clustering Methods
- How to Evaluate
- What Makes Better Performance
- **Conclusion**
- References

---
## Conclusion

- Clustering Applications
- Kinds of Clustering Methods
- Clustering Evaluation
- Gain a Better Performance
---
## References
<div class="reference">
<ul >
<li> https://en.wikipedia.org/wiki/Cluster_analysis</li>
<li> Costa IG: Clustering cancer gene expression data: A comparative study</li>
<li> Mahamed G.H. Omran etc.: An overview of clustering methods</li>
<li> T. Soni Mahulatha etc.: An overview on clustering methods by</li>
<li> TAOUFIQ ZARRA etc.: TOPIC MODELING AND SENTIMENT ANALYSIS IN FACEBOOK TO ENHANCE STUDENTS LEARNING</li>
<li> Abhinandan Das:Online Collaborative Filtering Online Collaborative Filtering</li>
</ul>
</div>

---
## References cont.
<div class="reference">
<ul >
<li>Sara C. Madeira: Biclustering Gene Expression Data</li>
<li>https://www.youtube.com/watch?v=uCaPP4blYAg</li>
<li>http://www.mit.edu/~9.54/fall14/slides/Class13.pdf</li>
<li>http://www.lohninger.com/helpcsuite/kohonen_network_-_background_information.htm</li>
<li>http://www.ims.uni-stuttgart.de/institut/mitarbeiter/schulte/theses/phd/algorithm.pdf</li>
</ul>
</div>

---
## References cont.
<div class="reference">
<ul >
<li> Marcilio C. P. de Souto etc.: Comparative Study on Normalization Procedures for Cluster Analysis of Gene Expression Datasets</li>
<li> https://datascience.stackexchange.com/questions/22795/do-clustering-algorithms-need-feature-scaling-in-the-pre-processing-stage</li>
<li> http://mlwiki.org/index.php/SNN_Clustering</li>
<li> https://www.ncbi.nlm.nih.gov/pmc/articles/PMC430175/</li>
</ul>
</div>
---
template: inverse 
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
# <center>Thanks!<center>
## <center> Q&A<center>