Jeff Bezos’s letters to shareholders are, of course, famous for their insight, not only in to how Amazon functions, but also for their advice on how to run a certain type of business. One of my favourite excerpts, from the 2005 letter is:
As our shareholders know, we have made a decision to continuously and significantly lower prices for customers year after year as our efficiency and scale make it possible. This is an example of a very important decision that cannot be made in a math-based way. In fact, when we lower prices, we go against the math that we can do, which always says that the smart move is to raise prices. We have significant data related to price elasticity. With fair accuracy, we can predict that a price reduction of a certain percentage will result in an increase in units sold of a certain percentage. With rare exceptions, the volume increase in the short term is never enough to pay for the price decrease. However, our quantitative understanding of elasticity is short-term. We can estimate what a price reduction will do this week and this quarter. But we cannot numerically estimate the effect that consistently lowering prices will have on our business over five years or ten years or more. Our judgment is that relentlessly returning efficiency improvements and scale economies to customers in the form of lower prices create a virtuous cycle that leads over the long term to a much larger dollar amount of free cash flow, and thereby to a much more valuable Amazon.com. We’ve made similar judgments around Free Super Saver Shipping and Amazon Prime, both of which are expensive in the short term and—we believe—important and valuable in the long term.What Jeff is saying here, as he has in many of his other letters is that Amazon is in for the long term, making long term market decisions (essentially, that Amazon is taking a market position that it will always be the cheapest, getting cheaper all the time, whether for a book, a baby stroller, a Kindle or 50 TB of cloud storage) in preference to short term fixes. But more than that, he’s saying that even when he knows that a short term (and presumably, very tempting!) decision will make more cash – because the maths shows it – that will be usurped by their strategy to play the lower-price long game.
This reminded me of a an interesting problem from the world of maths – simulated annealing. Described considerably more succinctly, here on Wolfram, this is method used when trying to optimise a given function, and, because of the complexity of the function, there is a risk of getting caught in local optima, particularly when using standard “hill climber” functions. If a function is too complex, with too many variables to find the optimal solution directly, one has to start from what we know today then use “hill climber” functions to find “What’s the direction I should go in?”. For example, we’re currently charging $x for our product, what should I do next to optimise revenue? Increase slightly or decrease slightly? And if the function your trying to optimise is the “Best price for this particular product”, then this will reach an optimal solution – you’ve optimised when all experiments to change the price just make profits worse. I.e. you’ve used “the maths” to find the optimal solution for this particular problem.
But this is a local optimisation. If what you’re trying to optimise is the bigger, much more complex function of “What is the best set of prices for everything my company offers?” then the danger of this approach is that you optimise the individual cases whilst de-optimising the problem as a whole. Simulated Annealing as a process tries to solve this problem in two ways. Firstly and most relevant here, is that so-called “Bad” decisions are allowed to be made locally if it allows one to explore more of the landscape of possible outcomes. So, for example, if your product currently sells at $1000 – the hill-climber approach would say “What if I charge $1001, or $999 – which is better?”. Keep repeating this till you maximise profits – may be at $783 for example.
However, with simulated annealing, you would allow a test which would be “What if we charge $10? Or $10,000?”. Possibilities that seem ridiculous, but what if these experiments take your business in a whole new direction, reaching new customer types you never dreamed of? What if that, in the long term, led to optimising your business as a whole? You would never realise this success without trying these experiments that fly in the face of what the maths is telling you, and letting those experiments run for a while.
I know, of course, that I’m slightly over-extending an analogy here – in fact what Jeff Bezos is doing is not taking larger, experimental punts to try and jump out of local minima. Instead he has a long term strategy which overrides that – a belief that in the long term, the lower prices for everything is the optimum for his business. A strategy that’s hard to argue with given his success!
But the point really is that, sometimes, taking a strictly data-driven decision in the short term, though it may show nice charts of increased revenue (i.e. you’ve optimised locally), can have longer term implications that you might regret, if those decisions don’t sit well with the position in the market your company is trying to achieve. If you’re trying to achieve the position of “The cheapest”, don’t increase your prices (see Amazon). If you’re trying to achieve the position of “The best”, don’t lower them (see the Apple 5C). If you’re trying to achieve “Pay as you Earn” (i.e. people pay according to ability to pay), then set an intelligent pricing strategy that caters for all wallets and stick to it – see any Atlassian product, e.g. JIRA:
This last example is a good case in point. I’m sure it would be very simple to show how increasing the lowest price point here ($10 – that’s ridiculous, how can they make money on that!?) will make more money in the short term. But there’s a long term strategy here based on Life time Value – today’s startups are tomorrow’s big businesses. Get them hooked on the product now and in the future they’ll be buying at the higher price points.
In the phrase”data-driven” it’s not the term “data” that worries me. It’s the term “driven” – when we are “driven” to decisions, rather than sticking to a long term strategy, instead letting the numbers make the decisions for us. By doing this we’re in danger of optimising in the short term, and not for the long term.
Obviously I’m not saying “Ignore the maths” all the time. Just be very careful when it suggests doing something that doesn’t align with your long term strategy of where you want the company to be in 10, 20 or 50 years from now.