Clustered Thoughts

Building a GPT-2 Tokenizer in Go

Rebuilding a GPT-2 BPE tokenizer from scratch in Go, with benchmarks and optimization deep-dives.

11 min read

Large Language Models Go · Large Language Models
Building an LSM Storage Engine from Scratch in Go, Part 4

Benchmarking the LSM: throughput-latency curves, group commit tuning, and the methodology behind it all.

16 min read

Storage Engines Go Systems Programming · Storage Engines
Building an LSM Storage Engine from Scratch in Go, Part 3

Leveled compaction, per-directory SSTable layout, and dropping the manifest file.

8 min read

Storage Engines Go Systems Programming · Storage Engines
Building an LSM Storage Engine from Scratch in Go, Part 2

WAL for crash recovery, group commit for throughput, and bloom filters for read performance.

10 min read

Storage Engines Go Systems Programming · Storage Engines
Building an LSM Storage Engine from Scratch in Go, Part 1

Writing a log-structured merge-tree storage engine from first principles in Go: skiplists, binary SSTables, and the flush path.

9 min read

Storage Engines Go Systems Programming · Storage Engines