Graph convolutional network

name: inverse
layout: true
class: center, middle, inverse

---
# Graph convolutional network
### By Mingbo
### 2024-10-28
.footnote[powered by <a href="https://github.com/gnab/remark"><font color=yellow>remark.js</font></a>]
???
I am a note, will be only shown when under presentation mode

---
layout: false
## Definition

$$G = (V, E)$$

where $G$ is a graph, $V$ is the set of nodes, and $E$ is the set of edges.

The Feature matrix $X$ is given by:

$$X\in \mathbb{R}^{N\times D}$$

where $x_i\in X$ is the feature description for every node. N is the number of nodes and D is the number of features.

---
## Definition

The Adjacency matrix $A$ of $G=(V,E)$ is given by:
$$\begin{aligned}A_{ij} = \begin{cases} 1 \quad \text{if } (v_i, v_j)\in E \\\\ 0 \quad \text{otherwise} \end{cases}\end{aligned}$$

It produces a node-level output $Z\in \mathbb{R}^{N\times F}$, where $F$ is the number of output features. Thus every neural network layer can be represented as:

$$H^{(l+1)} = f(H^{(l)}, A)$$

where $H^{(0)}=X$ and $H^{(L)}=Z$, $L$ is the number of layers.

---
layout: false

## Simplest Form

Given the activation function $\sigma$, the simplest form of a graph neural network layer is:

$$f(H^{(l)}, A) = \sigma(AH^{(l)}W^{(l)})$$

where $W^{(l)}$ is the weight matrix of the $l$-th layer and $\sigma$ can be a non-linear activation function like ReLU.

---
## Normalization

First we normalize the adjacency matrix $A$ by adding the identity matrix $I$ and the degree matrix $D$:
$$\hat{A} = A + I$$

Next we normalize the rows of $\hat{A}$ by the degree matrix $D$:

$$\hat{D} = D^{-\frac{1}{2}}\hat{A}D^{-\frac{1}{2}}$$

---
## Normalization Form

The normalized form of the graph neural network layer is:
$$f(H^{(l)}, A) = \sigma(\hat{D}^{-\frac{1}{2}}\hat{A}\hat{D}^{-\frac{1}{2}}H^{(l)}W^{(l)})$$

---
layout: false
## Karate club network

We will use the Karate club network as an example to illustrate the graph neural network.

![](https://cmb.oss-cn-qingdao.aliyuncs.com/karate.png)

.footnote[powered by <a href="https://tkipf.github.io/graph-convolutional-networks/"><font color=grey>tkipf.github.io/graph-convolutional-networks/</font></a>]

---
layout: false
## Karate club network

Three layers of GCN are applied to the Karate club network with randomly initialized weight. Even without training, the output of the last layer is already able to separate the four classes of the Karate club network:

![](https://cmb.oss-cn-qingdao.aliyuncs.com/karate_emb.png)

.footnote[powered by <a href="https://tkipf.github.io/graph-convolutional-networks/"><font color=grey>tkipf.github.io/graph-convolutional-networks/</font></a>]
---
layout: false
## Future reading

<div class="smallfont">
        1. Kipf & Welling (ICLR 2017), Semi-Supervised Classification with Graph Convolutional Networks (disclaimer: I'm the first author)<br/>
        2. Defferrard et al. (NIPS 2016), Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering<br/>
3. https://tkipf.github.io/graph-convolutional-networks/
</div>

![](https://cmb.oss-cn-qingdao.aliyuncs.com/gcn_web.png)

---
template: inverse
# Thanks!
# Q&A

---
class: middle, center
layout: false

# Reference

1. https://arxiv.org/pdf/1609.02907
2. https://tkipf.github.io/graph-convolutional-networks/