AI Systems Essays 12
Essays on AI infrastructure, evaluation, inference, agentic systems, and the engineering trade-offs behind production AI.
Where does an Agent actually run?
A look at the runtime around agent loops: harnesses, sandboxes, tool execution, state, permissions, and the infrastructure needed once a model can repeatedly act.
What changes when a Model can ReAct
How agentic behavior changes the workload shape: one request becomes many model calls, tool invocations, observations, and decisions that need orchestration around the model.
You control more of the bill than you think
Four practical levers for inference cost control, from routing and prompt shape to model choice and system design, beyond the headline token price.
Why Token Price Isn't the Whole Cost
Why inference cost is not just token pricing: latency, throughput, retries, context length, model choice, and system overhead all shape what AI actually costs to run.
Knowing Whether the Model Actually Works
A practical look at evaluation: building task-specific checks, choosing useful datasets, and measuring model changes against the production behavior that actually matters.
Post-training is the new Pretraining
A freshly pretrained model is not completely useful to talk to. This essay walks through supervised fine-tuning, preference optimization, and reasoning RL.
Prompt, Retrieve, Fine-Tune, Distill - In that order
A practical ladder for improving AI systems: start with prompting, move to retrieval when context is missing, fine-tune when behavior must shift, and distill when smaller models need to inherit capability.
How Models Learn (It's Simpler Than You Think)
A clear starting point for pretraining: how models learn patterns from massive text corpora, why next-token prediction scales into useful capability, and where infrastructure bottlenecks start to matter.
A GPU Cluster isn't a Cluster of GPUs
A thousand-GPU cluster is more than chips. The real system includes HBM, interconnect, networking, storage, power, cooling, and scheduling choices that determine whether accelerators stay fed.
Chip Is the Easy Part
Why comparing AI chips by spec sheets misses the larger architecture question: the best accelerator depends on software maturity, memory behavior, workload shape, and the system around the silicon.
Not all "AI Chips" do the same thing
A tour through AI chip families and the trade-offs each one makes against compute, memory capacity, and memory bandwidth.
Why I Stopped Thinking About AI Infrastructure as "Just GPUs"
AI infrastructure has evolved past stacking GPUs. Success means balancing accelerators for tensor math, CPUs for orchestration logic, and DPUs for networking.