Articles
- one hundred events became one
A delta-debugging reducer deleted ninety nine of a trace's one hundred events and every configuration ranking survived. Why preserving the decision is the wrong gate on its own, and how CommCanary compresses a distributed LLM communication trace while proving the compression kept the answer.
- my canary got bigger
Running CommCanary end to end on a laptop: a compression tool that doubled the size of its input, an error budget the compiler refused, a search whose own receipts outweighed what it saved, and the one number in the whole pipeline that turned out to mean what it said.
- the address where the value lived
A pointer remembers the first element of a vector. One push_back later, the value survives and the pointer does not. Following the missing address through arrays, layout, lifetime, vector growth, arenas, and pools.
- the syscall the kernel refused
The same open call returns a file descriptor, and then returns Operation not permitted. Following the result through the arm64 trap, XNU's file authorization path, a process policy, Linux seccomp, and executable memory.
- the head pointer came back
A delayed dequeue sees the same head address twice and succeeds with a stale compare-and-swap. Following that returned address through FIFO invariants, helping, linearizability, hazard pointers, reclamation, and the actual limits of lock-free progress.
- Rings, Trees, and Butterflies: HPX Collectives (Part 1)
Eight ranks need the same sum. The HPX call leads into one communicator server, then into five executed all-reduce algorithms, a byte-and-message cost model, fresh MPI measurements, and a provenance-stamped cluster result that changes what the source alone seemed to imply.
- idle gaps are not compute
A collective benchmark put one NCCL configuration safely in the middle. The workload put it last. Following that disagreement through trace capture, replay scheduling, CUDA streams, ranking statistics, missing evidence, queueing, and one source line that changed the explanation.
- the page that was not there
A mapping succeeds with no resident pages, then one ordinary load per page enters the kernel 256 times. Following those loads through virtual addresses, arm64 translation, XNU faults, zero fill, protection, copy-on-write, file sharing, wiring, and the measurements that cannot reveal a TLB.
- how your computer fakes randomness
rand() returns the same numbers after every process start. The first value leads into state recovery, entropy, biased mappings, finite tails, forked streams, and benchmark results whose meaning changes when order and uncertainty are handled.
- one line to machine code
One line enters torch.compile. What comes back is a guarded region containing a library matrix multiply and a generated C++ loop, plus a crash that four threads can reproduce and one thread cannot.
- teaching a model
A rank-8 patch fit the training target while the frozen model forgot unrelated text. The loss said it learned. The held-out prompts forced a more careful answer.
- one gradient, backward
A thirty-line tape and PyTorch agree on one gradient. A finite difference does not. Following the discrepancy reaches the chain rule, reverse and forward accumulation, matrix backward, GPT-2, checkpointed activations, optimizer state, and the numbers fp16 cannot keep.
- one forward pass
GPT-2 put Paris fifth. Following the exact rows behind that result reaches byte-pair tokens, matrix products, attention, residual state, a growing KV cache, batch reuse, and the difference between simulating 4-bit error and running a packed kernel.
- how machines agree
One counter, two machines, one dropped packet, and the state that must survive retries, partitions, and a killed process before an acknowledged increment can be trusted.
- two machines talking to each other
What really happens when two machines talk to each other?
- where the time goes
One program that sums every byte of a 1 GiB file, and the same arithmetic made 4000x slower and then fast again, a walk through syscalls, the user/kernel boundary, 16 KiB pages, page faults, the page cache, mmap, P and E cores, sampling profilers, and using the roofline as a bottleneck detector.
- a file is just bytes
A column read that looked thirty-eight times faster led into Parquet pages, byte ranges, encodings, pruning, Arrow buffers, and the commit race hidden behind a table.
- the fast path out of python
Summing ten million numbers takes 290 ms in a Python loop and 3.4 ms in numpy. Following the gap reaches PyObject boxing, pairwise reduction, native buffers, ownership, the GIL, and the point where zero-copy stops being a sufficient claim.
- two threads to a task runtime
Two threads each add one to a counter a million times, and it comes out to 1,012,274, a different wrong number every run. Following that lost million down through the load-add-store race, atomics, the weakly-ordered memory model, false sharing, mutexes and deadlock, and up to why real systems hand you tasks and futures instead of raw threads.
- from a C loop to a GPU kernel
A walk from a slow 512x512 C matrix multiply through cache, SIMD, FMA, tiling, roofline thinking, Metal kernels, threadgroup memory, GEMM, reductions, softmax, attention, and fusion.
- sixteen million and one
A full walk through computer numerics from integer representation and IEEE-754 float bits to rounding, subnormals, cancellation, compensated summation, SIMD/FMA/BLAS, mixed precision, and quantization.
- from source to a running process
A C++ toolchain walk from preprocessing and templates to object files, ODR traps, static and dynamic linking, ABI symbols, relocations, GOT/stubs, lazy binding, dyld, and the first instruction of a process.
- Python is a C program
Five lines of Python followed all the way down: a 52 KB Mach-O launcher, tokens, AST, bytecode, _PyEval_EvalFrameDefault, PyObject, refcounts and immortals, cyclic GC, the GIL, free-threading, specialization, the experimental JIT, and the C API.
- std::move doesn't move anything
Value categories, RAII, move semantics, copy elision, smart pointers, templates, and concepts, traced from one line of libc++ source. With real benchmarks on an M4 and real HPX code throughout.