From pull to ‘stream’ economies

I was interested to read Ben Evan’s recent take on the “Facebook of eCommerce”. He concludes:

That kind of scalable automation, though, could also go in completely the opposite direction for some things – away from any kind of decision at all. You put an Amazon Dash on the machine, or perhaps it can measure what you’re used and re-order by itself, and so you in effect subscribe to the product, and once done you’ll probably never bother to change brand. Or, say to Siri or Alexa or Google Assistant ‘Hey, order some more soap powder’ and the same brand is added to your next delivery. (And in both cases your choice of channel is just as now locked in as your choice of soap powder, once you’ve set the default.) Either way, an impulse purchase in one of 2 or 3 retailers you might have stopped in at, based on real-estate portfolio on one hand and eye-level placement and brand equity on the other, shifts to auto-renewal or a natural language parser. Given that P&G and Unilever’s combined ad budget is larger than the global revenue of the recorded music industry, this means that subscription soap powder could be a much bigger deal than subscription music. What will you have to pay to be Google Assistant’s default choice of dishwasher tablets?

It’s a well made point. But I think it could be looked at from another angle.

One of the core philosophies we developed for building systems at Storyful was a switch away from search-based systems to stream-based systems. I always felt that one of Twitter’s core innovations was its Stream API. Unfortunately it remains one of the few publicly available stream APIs out there (and to get it at any scale you need GNIP too).

When you’re trying to detect signal in noise, streams of data that you can filter can work incredibly well. Too many APIs, like for example YouTube’s, were based on the idea of repeatedly polling it to ask the same or similar questions of their data. Asking “any new videos uploaded containing the word ‘x'”, millions of times a day is not very efficient. (There were some attempts to streamify YouTube’s data using PubSubHubbub in the V3 API but this isn’t quite the same)

Rather, just getting the raw ‘stream’ data to manage and act on ourselves was far better – hence we spent a good deal of time converting REST APIs into Stream ones for our own purposes (using lots of calls) – and then building secondary systems and algorithms on top of those streams to detect events, anomalies, patterns and so on – across multiple platforms.

The same could be said of what Ben hints at – a switch away from user intent, ie “search“, or “GET”, to deliver, stream, or ‘push’. Google and Amazon are search systems. A user has to go find stuff and order/click it, usually in discrete transactions. Based on your behaviour the system might suggest other products or results that might interest you. The infrastructure that Ben mentions is what I would describe as streaming products. I subscribe to a “stream” of washing-up powder and it just arrives when required (based on either censors or figuring out on average how frequently I use it up).

The obvious next step from these kind of rudimentary streaming products is smarter streaming products. That world is one where I divest most control over rudimentary purchases entirely to a digital assistant (and by mine, I mean one designed for me, by me, that’s independent of platform or service). One could imagine entire industries built on trying to convert me one from one product “stream” to another, and users arbitraging en masse to receive either greater discounts, or alter the behaviour of producers. I assume this is where things like Jet are going.

The system will figure out what I need, when I need it, and even what I don’t need, but probably want. Then it will stream it to me. And this goes for digital products as much as it goes for physical ones. (An odd logical extension of this will be machines ‘advertising’ and ‘negotiating’ with other machines to change streams on my behalf).

Push, not pull. Streams, not requests.