Dispatches from the edge of chaos — on nonlinear dynamics, AI, emergence, and the mathematics of complex systems.
VQ-VAE (Vector Quantized Variational Autoencoder) and VQGAN (Vector Quantized GAN) learn discrete codebook representations of images, enabling powerful image generation by converting the continuous pixel space
WaveNet is a deep generative model for raw audio waveforms that uses dilated causal convolutions to model long-range temporal dependencies, producing remarkably natural-sounding speech and revolutionizing
Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and GraphSAGE are foundational graph neural network architectures that learn representations of nodes, edges, and graphs by aggregating
Stacked LSTMs and GRUs are deep recurrent neural network architectures that process sequential data by maintaining hidden states across time steps, with gating mechanisms that control
Latent Diffusion Models (LDMs), commercialized as Stable Diffusion, generate high-quality images by performing the diffusion process in a compressed latent space rather than pixel space, dramatically
U-Net is an encoder-decoder architecture with skip connections designed for biomedical image segmentation, producing pixel-precise segmentation masks by combining high-level semantic features with fine-grained spatial details.
ResNet (Residual Networks) and DenseNet (Densely Connected Networks) are landmark CNN architectures that solved the degradation problem in deep networks through skip connections, enabling training of
Mixture of Experts (MoE) is an architecture paradigm that scales model capacity dramatically while keeping computational cost manageable by routing each input to only a subset
The Evoformer is the core neural network module of AlphaFold2, DeepMind's breakthrough protein structure prediction system. It processes evolutionary and pairwise residue information through
Hyena is a sub-quadratic attention replacement that uses long convolutions and element-wise gating to achieve Transformer-quality performance with significantly reduced computational cost, particularly for long sequences.