Notes from an ongoing exploration
Writing.
Sometimes code. Sometimes life.
Always an attempt to understand a little more.
Rotary Positional Embeddings in CUDA: From Math to High-Performance Kernels
A practical deep dive into Rotary Positional Embeddings (RoPE) in CUDA: mathematical derivation, tensor layouts, coalesced and vectorized memory access, and effective bandwidth optimization on modern GPUs.
ClickHouse Physical Merge Strategies: Horizontal, Vertical, and a Hybrid in C++
A practical deep dive into physical merge algorithms in MergeTree: why ClickHouse uses horizontal and vertical merges, what trade-offs they optimize, and what we learn from building a hybrid adaptive chunked merge in C++.
Boltzmann Generators: Sampling Equilibrium States with Deep Learning
A technical deep dive into Boltzmann Generators, exploring how Normalizing Flows can directly sample equilibrium states of many-body systems, complete with a from-scratch PyTorch implementation of RealNVP and Flow Matching on 2D potentials.
TileKernels in the DeepSeek-V4 Era
How TileKernels maps low-level GPU kernels to DeepSeek-V4 ideas like compressed long-context attention and manifold-constrained residual routing, with a small worked example and a practical test-reading workflow.
Building a Go CLI to Compare Diff Algorithms
A practical case study in building a Go CLI that compares LCS, Myers, Patience, and Histogram diff algorithms, with unified output, benchmarks, and the trade-offs that actually matter.
Making Matrix Multiplication Fast: From Naive Loops to AVX2, and Why Neural Networks Care
A hands-on journey through optimizing GEMM in C++17 — cache blocking, AVX2 intrinsics, OpenMP parallelism — and how every dense layer in a neural network is just a matrix multiply in disguise.
Building a Kafka-Style Distributed Queue in Rust from Scratch
A practical deep dive into implementing a distributed message queue in Rust: WAL segments, partition replication, ZooKeeper coordination, consumer groups, and real failure handling trade-offs.
Minimal Implementation of ClickHouse MergeTree from Scratch
Reflections from working at Tinybird processing millions of events per second. A deep dive into building a minimal but functional MergeTree engine in C++, connecting academic theory with production reality.