Growth Amplified via Autonomous Optimization

Olcan Sercinoglu

Sep 06, 2018

Growth wins. The most successful products of our time are ones that are expertly designed and engineered to deliver more value as they grow, creating a virtuous cycle of compounding value and growth.

Growing products are naturally capable of generating more data, so the ability to leverage that data to drive even more value/growth is a universal source of compounding growth for any product or business.

A fascinating consequence of this is the emergence of an increasingly formal and cross-functional discipline of “Growth” that demands an unprecedented marriage of expert intuition and data.

Standard practice is for experts to maintain a hypothesis that informs a sequence of experiments that drive growth and/or improve the hypothesis. We can visualize this process and its impact on metrics along the funnel as follows:

growth-experiment.png#asset:678

This process falls short in many ways. First, it is not designed to maximize growth, but rather to execute a sequence of decisions where a certain type of risk (usually what is called a Type 1 Error) is reduced. The decisions are constrained to predefined segments (if any), and the risk calculation assumes (inevitably wrongly) that the future will mirror the past.

The following chart illustrates the difference between a typical A/B test compared to an algorithm that is actually designed to maximize growth. Everything else held equal (including the variants), maximizing growth can drive 9x more growth over the same time period.

growth-maximize.png#asset:677

Surprisingly, this result does not necessarily depend on the outcome of the A/B test. This is mainly because the test must be restricted to predefined segments that are virtually never the best segments for those specific variants (and specific metric chosen). In contrast, a growth-maximizing algorithm could be free to match segments to variants and vice versa, essentially personalizing and contextualizing behavior as needed to drive growth with those variants.

The following is a vivid (and intentionally provocative) illustration of how hypothesis testing, though obviously better than making critical decisions based on pure hunch, still leaves way too much to chance and thus falls very short of delivering on the promise of data-driven compounding growth.

growth-rolling-the-dice.png#asset:676

Can we go all the way to absolutely maximize data-driven growth? The answer is YES, thanks to cutting edge AI (specifically Machine Learning) driven optimization algorithms that can handle exponentially larger hypothesis spaces that are far beyond what we could manage with traditional hypothesis testing.

If we also consider the problem of sustaining growth longer term, across inevitable shifts and trends in metrics over time, we see that the complete solution must also include a certain level of autonomy. That is, an ability to execute independently of the experts, who can instead focus on refining their hypotheses and honing the intuition that is critical to driving long-term growth.

Although machines can relentlessly search exponentially large hypothesis spaces and even take actions to maximize gains, they certainly cannot define the metrics or understand how those metrics actually drive long-term growth in a business. They also cannot consider any information that is not (readily) available to them, such as the maintenance costs associated with the variants.

Thus, we propose a new Growth process where experts drive growth not by iterating on a sequence of experiments, but on a system of many Autonomous Optimization (AO) projects that continuously and independently drive growth in key metrics. We can visualize this new process as follows:

growth-autonomous.png#asset:679

The fundamental principles (e.g. statistical significance and rigor) remain largely the same, but unlike simple experiments, AO projects can reliably improve metrics while producing insights that go far beyond pass/fail conclusions on predefined segments, instead revealing novel combinations of segments and variants that have already been successfully applied to maximize growth in past projects.

This new process enables much higher traction and leverage than the standard process, with several extremely desirable compounding effects, such as:

In future posts we will provide a describe how to design and implement effective AO projects and dig deep into various specific use cases across the customer funnel and software stack.