Why small experiments matter

The temptation of the large bet

There is a particular kind of ambition that wants to skip straight to the significant move. The big product launch. The comprehensive platform. The bold creative statement that arrives fully formed. It is understandable. Large bets feel proportionate to large ambitions, and if you believe in what you are building, the impulse to go all-in makes a certain emotional sense.

The problem is that large bets, made early, are usually bets on assumptions rather than knowledge. You commit resources, time, and reputation to a hypothesis you have not yet tested. If the hypothesis is wrong, you have paid a high price to learn something you could have learned more cheaply. And at the early stage, most hypotheses are wrong in at least one important dimension, often in several.

This is where we are at MSG in mid-2015. Resource-constrained, early in the building of several different systems across technology, performance, and creative work. The honest description of our situation is that we know a lot about what we are trying to accomplish and relatively little about the exact shape the work will take. That is a perfectly normal place to be. The question is what discipline follows from it.

What makes an experiment genuinely small

Not every modest-looking attempt qualifies as a small experiment in the useful sense. There is an important distinction between a small experiment and a half-hearted attempt, and collapsing that distinction is one of the more common early-stage mistakes.

A half-hearted attempt is small in effort but unclear in intention. You do a version of the thing without being certain what you are trying to learn from it. The output exists but cannot be evaluated against any specific question. When it does not produce results, you are not sure whether the approach failed or the execution did.

A small experiment is different. It is small in scope but sharp in intention. You define, in advance, what question you are asking. You create the minimum conditions needed to get a meaningful answer to that question. You execute with full commitment within those conditions. And you interpret the result against the question you asked, not against some vague hope about outcomes.

The scope constraint is about efficiency, not about effort. You are not trying to do less. You are trying to direct effort toward learning rather than toward premature scale.

In practical terms, this means that a small experiment in software development might be a working prototype that answers one specific interaction question. A small experiment in creative work might be a single finished piece that tests whether a particular voice lands with a particular audience. A small experiment in performance work might be a structured training block of defined length with specific outcome metrics. In each case, smallness comes from the boundary you draw around the question, not from reduced commitment to the work inside that boundary.

The cost of being wrong, renegotiated

One of the underrated benefits of small experiments is what they do to the cost of error. Being wrong is not only inevitable, it is necessary. You cannot build an accurate model of a new domain without updating that model against reality, and updating requires encountering disconfirming evidence. The question is not whether you will be wrong but how much it costs you when you are.

Large bets that prove incorrect carry compounding costs. There is the direct cost of resources spent. There is the opportunity cost of the time and attention that could have gone elsewhere. There is often a reputational cost if the bet was visible. And there is a subtler cost: the difficulty of changing course after you have publicly or internally committed to a direction. Sunk cost reasoning is not rational, but it is deeply human, and the larger the bet, the more powerfully it operates.

Small experiments reframe the relationship with error. When an experiment gives you an answer you did not expect, the cost of that discovery is bounded by the size of the experiment. You do not carry it forward. You take the learning, revise your model, and design the next experiment against the updated understanding. Error becomes part of the process rather than a threat to it.

This is easier to describe than to practise. There is genuine psychological work involved in treating a failed experiment as a successful piece of learning. But the structural logic is sound, and over time the habit becomes possible.

Knowledge compounds; ignorance does not

There is an asymmetry between the value of experimental knowledge and the value of unmade decisions. Unmade decisions do not compound. Leaving a question unanswered because you are waiting for the right moment to run a large program does not accumulate value. The question stays open. The uncertainty persists.

Experimental knowledge, by contrast, does compound. Each experiment that produces a clear answer narrows the space of uncertainty for the next experiment. You know which variables matter and which do not. You know which assumptions survived contact with reality and which need revision. You know which directions to pursue with more investment and which to set aside.

Over twelve months of consistent small experiments across different domains, you can accumulate a body of knowledge that would have been impossible to acquire through the same twelve months of planning toward a single large bet. This is not a claim about the pace of output. It is a claim about the rate of learning. And in the early stage of building anything, learning is the primary product.

At MSG, this principle applies across the portfolio. Work on performance systems, technology infrastructure, creative development, and product thinking all benefit from the same underlying discipline: run the smallest experiment that gives you a real answer, take that answer seriously, and build the next experiment on what you have learned.

The tension between patience and speed

There is a genuine tension in this approach that deserves honest treatment rather than resolution by slogan.

Building anything significant requires patience. The timeline for a meaningful product, a durable creative practice, or a robust performance system is measured in years, not weeks. Impatience at the level of the long arc is one of the most reliable ways to undermine the work. You cut short the compounding, chase premature recognition, and make decisions from a narrow time horizon that the work itself requires you to abandon.

At the same time, small experiments operate on a fast cycle. They are designed to produce answers quickly and to feed those answers back into the work. This is not the same as impatience. It is a distinction between the pace of iteration and the timescale of ambition.

The resolution, to whatever extent one is available, is to hold both simultaneously. Commit to the long arc with genuine patience. Within that long arc, move quickly through experimental cycles and take the learning seriously. Do not confuse patience with slowness at the level of iteration. Do not confuse fast iteration with impatience about the destination.

In practical terms: we are not trying to build everything this year. We are trying to learn enough this year to be meaningfully better positioned next year. The experiments that matter most are the ones that move our understanding forward, not the ones that produce the most visible activity.

Across domains, the same discipline

What makes the small experiment framework valuable for a portfolio organisation like MSG is that it applies across genuinely different kinds of work.

In technology development, the unit of experiment might be a component, an integration, or a specific user interaction. The question might concern reliability, speed, or whether a particular abstraction serves its purpose under realistic conditions.

In performance and fitness work, the unit might be a training protocol or a recovery intervention. The question might concern adaptation rate, compliance, or the relationship between a specific input and a specific measured output.

In creative work, including music, media, and content development, the unit might be a finished piece or a structural format. The question might concern resonance, reach, or whether a particular approach to storytelling produces the intended effect on its intended audience.

These are different domains with different methods and different kinds of evidence. But the underlying logic is the same in each case: define the question first, design the smallest thing that can answer it, execute with commitment, and take the answer seriously.

The portfolio structure means that learning in one domain can sometimes transfer to another. A lesson about iteration speed learned in software development can inform how creative work is structured. A lesson about measurement learned in performance work can inform how product decisions are made. This cross-domain transfer is not automatic, but it is real, and it is one of the structural advantages of working across multiple kinds of problems simultaneously.

What this requires

None of this is easy to sustain. Small experiments require a tolerance for ambiguity: you are committing to a process rather than a predetermined outcome, and the process involves regularly discovering that your assumptions were incomplete. They require a genuine culture of learning rather than a culture of performance, where the point is what you know rather than what you appear to be doing. They require the discipline to define questions before designing experiments, which means sitting with uncertainty for longer than feels comfortable.

They also require honest evaluation. An experiment whose results are interpreted to confirm what you already believed is not an experiment in any useful sense. It is confirmation theatre. The standard is whether you would genuinely update your behaviour based on a negative result, and then whether you do update it when one arrives.

At this stage of MSG, the commitment to small experiments is as much a character commitment as a strategic one. It is a decision about what kind of organisation we are building: one that takes its own uncertainty seriously, one that treats learning as a first-order activity rather than a byproduct, and one that is willing to be wrong repeatedly and cheaply in service of eventually being right about the things that matter.

That orientation is not naturally dramatic. It does not produce the kind of progress that is easy to narrate in the moment. But it is the most reliable path we have found from genuine uncertainty to genuine knowledge, and genuine knowledge is the only foundation on which anything durable gets built.