howaboutyu/quarto-site

Website made with quarto

quarto-site's Issues

Jax blog issues

nerf algo

The explaination needs revision:

• "Consider a line or ray directed pointing away from the camera,"
• $a_i$ explanation does not make sense

GNN notes - foundations

https://www.youtube.com/watch?v=uF53xsT7mjc&ab_channel=PetarVeli%C4%8Dkovi%C4%87

Molecules can be represented as graphs:
Atoms can be thought of a s nodes and bonds are edges, featrues can be atom type and charge. Then we can use GNNs will inhibit a potential bacteria. After training the model can predict whether a new molecule that is not in the training set will perform at targeting a specific bacteria.

Transportation networks can also be through of as graphs. it can predict the transportation time from one city to another.

it is a growing field, experiencing their "IMAGENET" moment.

Some echo systems:

pytorch geometric
spektral
dgl.ai
deemind: graph_nets and jraph

convolutional neural networks are translation invarient, patterns are "internesting" irrespective of where they are in the image. Locality is another key property of conv nets, as closer pixels are more important to each other.

in the case of graphs, nodes are not in any-order.

permutation invarience vs variance

if we have a set of nodes $V$ and $x_i \in \mathbb{R}^k$ be the features of node $i$. we can stack them into a single node feature $X\in\mathbb{R}^{n\times k}$. This representation is not node invarience, as we have an order here. we can have permutation matrices $P$ that changes the order of the nodes.

Defining permutation invariance, we say $f(X)$ is permutation invariant if $f(PX) = f(X)$. This will mean that the order of the nodes are not importnat, some operations are sum, averages, or max. for insatnat $sum(P* [1, 0,2, 3]) = sum([1, 0, 2, 3])$. Permutation invariance is good at defining global level outputs, where are not quite useful interms of GNNs.

To work at the level of nodes, we seek functions that don't change the node ordering. $f(PX) = Pf(X)$, this is *permutation equivariant$. This is saying that we have to apply a function to each node individually, i.e. computing the latet vector for each node $i$, $h_i = \psi (x_i)$.

This is a useful blueprint, stacking equivariant functions with an invariant tiail. It will be first applying an equivariance function, then an invariant function to get globlal values.

Learning on graphs

$E \subseteq V\times V$, we can then represent the edges with an adjacency matrix $a_{ij}$. the permutation matrix applied before now also needs to be performed on the edges ($PAP^\top$). Thus invariance now needs to be $f(PX, PAP^\top)=f(X, A)$, and equivariance $f(PX, PAP^\top = Pf(X, A)$.

locality on graphs for a node $i$ its 1-hop neighbourhood is defined as

$N_i = {j : \in \epsilon \text{ or } (j, i) \in \epsilon}$

we can also consider the set of features in the neighbourhood as $X_{N_i} = {{x_j : j \in N_i}}$, and define a local function $g$ operating over this multiset : $g(x_i, X_{N_i})$. This is a permutation equivariant function for operating over a graph $f(X, A) = [ g(x_1, X_{N_1}) ,... ] $, to ensure equivariance, we need $g$ not to depend on the order of the verticies in $X_{N_i}$, so $g$ should be permutation invariant.