How an experiment is designed, how its evidence is evaluated, and how the next experiment is chosen are themselves methodological decisions. Lab Loop makes them explicit and swappable, grounded in the theory of empirical research, not locked to one approach.
The research question determines the method, not the other way around. Design, evaluation and search can change as the programme evolves.
Five methodological families, each grounded in a distinct theoretical tradition, compose into the research loop.
The design determines what a result can teach. Factorial and fractional designs (Fisher) isolate main effects and interactions; response surface methodology (Box & Wilson) maps optima; optimal design theory (Kiefer) allocates trials efficiently under a model.
A result only counts as evidence against a method for evaluating it. Frequentist testing (Neyman & Pearson) controls error rates; Bayesian updating (Laplace, Jeffreys) revises belief quantitatively; likelihood methods compare support across hypotheses directly.
When experiments are expensive, the choice of the next trial is itself a decision under uncertainty. Model-based search (Bayesian optimization) builds a surrogate and exploits it; direct search (Nelder & Mead, CMA-ES) needs no model; random and quasi-random baselines establish what structured search is worth.
Real research questions rarely have a single objective. Pareto-based methods (Edgeworth, Pareto; Deb) return a non-dominated set rather than compressing objectives into one arbitrary score; scalarization and preference elicitation let a decision-maker steer the search without pre-committing to weights.
Research runs under a finite budget of trials, time and cost. Multi-armed bandit theory (Thompson, Gittins, Lai & Robbins) formalizes the explore-exploit tradeoff; active learning chooses experiments that most reduce uncertainty; optimal stopping decides when enough evidence has accumulated.
A result that cannot be reproduced cannot inform a decision. Replication and blocking (Fisher) separate signal from noise; variance estimation and resampling (Efron) quantify uncertainty without distributional assumptions; randomization guards against confounding the factor under test with uncontrolled variation.
The research programme remains stable while the method at each stage changes according to the structure of the question.
Goal, constraints, budget and the active question.
A trial configuration produced by the chosen design and search method.
Evidence assessed by the chosen evaluation method, with variance reported.
Accept, reject or revise, and choose the next experiment under the remaining budget.