Blog
Fragments of code, concepts, and curiosity.
·
Sparse Attention · Part I
Why Sparse, Why Adaptive, and the Classical Problem Behind Thresholds
Softmax and α-entmax from one regularized family, why softmax disperses under fixed score spread and temperature, and the classical allocation problem behind thresholds.
Sparse Attention
α-entmax
Optimization
·
Sparse Attention · Part II
How Sparse Attention Runs Fast and How Frontier Labs Do It
How DeepSeek and MiniMax train sparse indexers, when sparsity saves computation, and how AdaSplash-2 uses threshold estimates and block masks.
Sparse Attention
GPU Kernels
·
50 min read
SLURM in the Wild: A Practical Guide for Academic Labs
A complete guide from basic concepts to production deployment — covering multi-node setup, GPU scheduling, advanced monitoring, and hard-learned lessons from scaling a research lab from 2 to 30+ users across heterogeneous hardware.
Infrastructure
GPU Computing
SLURM
More posts coming soon.