Every notebook in the repository as a readable page: the text and the code, without outputs. Open the live marimo server ↗ to run them (reachable while the workstation tunnel is up).
CS224W companions
L01_introductionL01 · Introduction — graphs as data1. A graph from raw records — Zachary's Karate Club in NetworkX, then as PyG tensors (edgeindex). 2. Why ordinary deep learning chokes — relabel the nodes and watch an MLP change its mind. 3. Choosing a representation — one Twitter-like log, four different graL02_node_embeddingsL02 · Node embeddings1. The template — encoder (a lookup table), decoder (dot product), similarity. 2. Random walks — uniform walks and what "co-occurrence" looks like. 3. node2vec's biased walk — the p,q step by hand, then BFS-like vs DFS-like walks. 4. Skip-gram with negative saL03_graph_neural_networksL03 · Graph Neural Networks1. One GCN layer by hand — the lecture's 4-node example, then its matrix form D^{-1}A. 2. Computation graphs — unroll a Karate node's K-layer tree. 3. Permutation equivariance — a numerical test. 4. GCN from scratch — symmetric normalisation, checked against PL04_general_perspective_gnnsL04 · A general perspective on GNNs1. One layer = message + aggregation — PyG's MessagePassing in 10 lines. 2. GCN, GraphSAGE, GAT from scratch — each checked against PyG with identical weights. 3. GAT attention by hand — the lecture's worked example, interactively; why weights that sum to 1 caL05_gnn_augmentation_trainingL05 · GNN augmentation and training1. Feature augmentation — constant vs. one-hot vs. cycle-count features on a "triangle or square?" task. 2. Virtual nodes — fix long-range message passing on long paths. 3. Neighbour sampling — a hand-written mini-batch sampler; cost vs. accuracy on Cora. 4. PL06_theory_of_gnnsL06 · Theory of GNNs — how expressive are they?1. Aggregators on multisets — mean, max, sum: which can tell which neighbourhoods apart? 2. The WL test — colour refinement from scratch; hexagon vs. two triangles; more pairs. 3. GIN — sum + MLP; a counting task where GCN fails and GIN wins. 4. Real benchmarkL07_powerful_graph_encodersL07 · Designing powerful graph encoders — breaking the WL limit1. The spectral view — what a GNN with constant inputs can and cannot see. 2. Fix B: structural features — closed walks \mathrm{diag}(A^k); triangle-vs-square nodes; connected vs. disconnected graphs. 3. Fix A: eigenvector positional encodings — Laplacian PE, L08_graph_transformersL08 · Graph transformers1. Self-attention by hand — the lecture's two-token example, plus a Graphormer bias. 2. Attention is message passing — full attention = complete graph; masked attention = neighbours only. 3. Putting the graph back — PE vs. attention bias vs. local MP; a long-rL09_heterogeneous_graphsL09 · Heterogeneous graphs1. What makes a graph heterogeneous — DBLP: authors, papers, terms, conferences. 2. R-GCN from scratch — per-relation weights, normalisation, self-loop; basis decomposition. 3. Counting parameters — the lecture's worked example, interactive. 4. Benchmarks on DL10_knowledge_graphsL10 · Knowledge graph embeddings1. Score functions by hand — TransE's Obama example; symmetry with TransE vs. RotatE. 2. Relation patterns — the expressiveness table. 3. A family KG with known patterns — train TransE, DistMult, ComplEx, RotatE (PyG) and measure per-relation filtered MRR. 4. L11_recommender_systemsL11 · GNNs for recommender systems1. Data — MovieLens (100k ratings) as an implicit-feedback bipartite graph; a per-user temporal split. 2. Recall@K and a popularity baseline. 3. Surrogate losses — binary vs. BPR, the lecture's worked example. 4. Matrix factorisation vs. LightGCN — from scratcL12_relational_deep_learningL12 · Relational deep learning1. A small e-commerce database — users, products, transactions (synthetic, with a planted signal). 2. A database is a graph — the relational entity graph as HeteroData. 3. The training table — churn labels at several cutoff times; temporal split. 4. GNN layer L13_advanced_rdlL13 · Advanced architectures in relational deep learning1. A junction-table database — customers, products, transactions; a task that needs price × category pairing. 2. Standard heterogeneous message passing with 1, 2, 3 layers. 3. Composite message passing (RelGNN) — one step along customer ← (transaction ⊕ producL14_advanced_topicsL14 · Advanced topics — in-context learning on graphs, relational foundation models, conformal prediction1. In-context learning on a graph (PRODIGY-lite) — label-free pretraining, then few-shot prompts with no weight update. 2. Relational Transformer tokens and masks — cells as tokens; column / feature / neighbour / full attention. 3. Conformal prediction for GNNL15_foundation_models_kgsL15 · Towards foundation models for knowledge graphs1. The inductiveness ladder. 2. The relation graph — the lecture's movie-KG worked example. 3. ULTRA-lite — relation GNN + query-conditioned entity GNN, from scratch. 4. Zero-shot transfer to a new KG with unseen entities and shuffled relation IDs. 5. ExerciseL16_llms_gnnsL16 · LLMs + GNNs1. A company knowledge graph (synthetic) — people with short bios and typed relations (workswith, reportsto, mentors). 2. VectorRAG vs. GraphRAG — which retrieval finds the facts a two-hop question needs? 3. GNN feeds LLM — a GNN reader that answers from the rL17_agents_graphsL17 · Agents + graphs — building intelligent and collaborative agents1. RLHF's reward model is BPR — train one on synthetic preference pairs. 2. A semi-structured knowledge base — product texts + a KG; text-only vs. KG-only vs. hybrid retrieval. 3. AvaTaR-lite — an LLM comparator optimises the agent's tool weights by contrastinL18_generative_modelsL18 · Deep generative models for graphs1. The likelihood of one generated graph — the lecture's worked example. 2. Node orderings — random vs. BFS: why GraphRNN only needs to look back M nodes. 3. GraphRNN-lite — train on grids, sample, evaluate. 4. GCPN-lite — goal-directed generation with a GNN pL19_conclusion_design_spaceL19 · Conclusion — the GNN design space1. Kendall's τ by hand — the lecture's worked example. 2. A small design space and task space — 72 designs × 4 node-classification tasks. 3. Controlled random search — is BatchNorm useful? Are skip connections? 4. Task similarity and transfer — anchors, τ matrL20_bonus_classic_ml_scalingL20 · Bonus A — classic graph ML and scaling1. Node-level features on the notes' 5-node example (degree, clustering, betweenness, closeness, graphlet orbits). 2. Link-level features — CN, Jaccard, Adamic–Adar, Katz; then link prediction on Cora. 3. Labels as signals — label propagation; Correct & SmoothL21_bonus_geometry_subgraphs_trustL21 · Bonus B — geometry, subgraphs and trust1. Geometric GNNs — invariance (SchNet) and equivariance (PaiNN), checked numerically; a toy energy-regression task. 2. Network motifs — feed-forward loops, null models, Z-scores; the notes' retweet example. 3. Neural subgraph matching — order embeddings. 4. E