Research

How results are made, and how to read them. The series here are about method rather than any one field: what a number from an experiment can and cannot tell you, what the words in a methods section mean, and the design choices that decide whether anyone should believe a finding. The worked examples come from studies of AI coding agents, because those are the ones I ran, but nothing in them is specific to AI.

Research Basics

A plain-language series on the words in a methods section: hypotheses and the null, the permutation test done by shuffling cards, what a p-value means, confidence intervals built by resampling, the Holm correction, how to defend a null result, blinding and preregistration, then the bootstrap test, the classical tests, regression and mixed models, leaderboards and pass@k, and the Bayesian alternative. 15 parts, each with a worked example from two real studies, a demo you can shuffle, resample or break, and Python that reproduces its numbers, plus a glossary.

Research

Research Basics