The Sentinel #24: Limits to Scale
Returns, Nimbyism and Cultural Constraints
We are nearing 25 posts!
Based on conversations and suggestions from some readers of this newsletter (thank you!) some fun topics have emerged. Working on some essays on those topics (Podcasts on Ops., Intel, Revisiting WFH predictions, Understanding Space tech, Apps and Antitrust, VisionPro, AR, and productivity).
After I wrote my NVIDIA as a platform couple of weeks back, I was gratified to see The Economist do a podcast dive on NVIDIA covering the same CUDA-related topics of my post. If audio is your thing, you may be able to listen to it here (Nvidia’s trillion-dollar bet).
—
Data and Model Collapse
Mostly, this week I have been thinking about problems of scale and execution. Last week, through a colleague/subscriber (thanks, ss), I came across a paper called “Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks” (arxiv link) — which looks at a case study, and estimates that (33-46%) of the human workers in a crowdsourced (AmTurk) project used LLMs to label things. So, LLMs lead to more LLM data. I don’t know if the cyclic synthetic data is the tip of the iceberg or just a red herring. However, such “model collapse” is a real threat that comes with scale.
During the pandemic, I attended a research talk that I found bizarre, as the author insisted that tensor completion using synthetic data is a theory — a theory that predicts what will happen in the same way as the laws of physics do. I believe the comparisons were Hooke’s Law and Newton’s Laws. If the synthetic data were from an identical (incorrect) process, it could end up “proving” wrong theories.
I was wondering if the likelihood of such a model collapse would happen due to problems of scale (to fill LARGE data one adopts shortcuts). Even before we address the scaling issues with information, it is important to recognize scaling issues we have known for a while in operations. Let’s delve into it.
Diseconomies of Scale
Operations folks are already familiar with the clunky problem of largeness. In addition to large problems (i.e., many unknown variables), we address specific issues of diseconomies of scale (for instance, inventory carrying costs can be more expensive as you carry more) and capacity constraints (e.g., there may not be enough water in the town to cool the power-hungry data centers). Hence, the knowledge of such diseconomies among researchers is not a new concept. In fact, it is ancient even among people of the world.
In Madrid, at the Museo del Prado, hangs a painting of Aesop by Diego Velázquez, which I had the pleasure of seeing. Aesop, known for his tales of folksy wisdom, was born a slave and mute without language for many years and later served in the court of Lydian King Croesus. Velázquez gave Aesop's face, a disfigured physiognomy of “ox-type head”. A vignette of ancient wisdom — popularized in a famous Aesop’s fable about a man and his son and their donkey — says “Please everyone and you will please no one”.
Limits of Growth
One company that has been challenging the idea that one could maintain quality with scale is Amazon. However, from recent developments, it is evident that they are also reaching their limits.
From Day 1, Amazon has shown a willingness to enter new markets, persevere, and expand into many markets (both in products and geographies). Even the name Amazon was clearly chosen to expand beyond books. The other contender for the name was Relentless. Now (a) Amazon wants everyone to be their customer and (b) is willing to compete with the smallest firms. If you are selling anything online, you will soon either compete with Amazon or depend on Amazon to sell it for you.
To be clear, this is not my observation. Bezos made this point in his interview with Wired magazine, when in 2011. “CEO of the Internet” is a great rare early interview. (Bezos was highly reticent and highly unavailable to media then, a far cry from these days on his new engagements and pursuits).
“There are two ways to build a successful company. One is to work very, very hard to convince customers to pay high margins.
The other is to work very, very hard to be able to afford to offer customers low margins. They both work. We're firmly in the second camp. It's difficult -- you have to eliminate defects and be very efficient. But it's also a point of view. We'd rather have a very large customer base and low margins than a smaller customer base and higher margins.”
-Jeff Bezos
A simple idea really. Sell to everyone. A dozen years from the interview, the all-pervading nature of Amazon has become common knowledge, after their astounding growth. In the last three years, I had exactly one student who declared in class that she wasn’t an Amazon Prime member.
With multiple Fulfillment Centers (FCs) in each state and aided by the physical retail presence of Whole Foods Inc., Amazon’s geographic and demographic penetration, especially among the upper middle class is considerable. CIRP estimates Amazon membership is finally flattening with 170 million Amazon Prime members in the United States as of March 2023. As a comparison, only 120 million people voted in the 2022 US elections (according to the Brookings Institute report), among the highest turnouts in midterm elections in the United States since 1966.
Diminishing Returns
Scale hurts many processes (and creates fun new informational problems): Fake products, Fake reviews, Disintermediation, knowledge gap, etc. Arguably, the best evidence for the massive growth in scale-hurting operations is the policy change with respect to returns. Amazon currently allows returns at 18,000 locations, including the option to drop off items without a box or label at Kohl’s, UPS, and some Whole Foods stores. The “Try Before You Buy” program is used to entice Prime members to make returns for clothes even easier, with return labels already included in the box. After a long history of encouraging free returns, Amazon finally is clamping it as it gets increasingly unwieldy. As of May 2023, if you choose to drop off an Amazon-purchased item at a UPS store, you could be charged a $1 fee.
Returns are costly for companies. According to a CNBC report, a record $761 billion of merchandise was returned in 2021. Of all online purchases, nearly 21% are returned. Every returned product must be mailed, sorted, and re-stocked—which adds to the cost. Complicating the process for Amazon is the product is also stowed back to one of many warehouses increasing process complexity if they need to be refurbished.
Political Limits of Scale
Amazon is not the first company to try and grow everywhere. Walmart tried selling to everyone. The Great A&P before that. Sears before that.
As Aesop’s fable shows, it is challenging to realize it in practice. What stops a firm from realizing this? Usually, there are institutional and cultural barriers.
In Walmart’s case, the leap from the rural and exurban markets to urban markets was challenging. Chicago passed an ordinance preventing Walmart from entering the South Side. Walmart then opened in Evergreen Park, just one block south of the boundary. As late as 2017, NYC stopped Walmart. Walmart entered NYC in 2018 (through its then “tech” acquisition, Jet). Walmart E-commerce is based in Hoboken, NJ, across the Hudson from Manhattan. Walmart in Washington DC has had a chequered history since its entry in 2012. Surely, some local NIMBYism and valid labor considerations were behind these difficulties. Despite all the hiccups around opening an HQ in New York, Amazon has had an easier time with its retail expansion than Walmart.
When Amazon threw its support behind the BLM protests in 2020 and made a stand, there was some consternation from some of its customers. As I mentioned, the 2020 Bezos was more vocal compared to the 2011 Bezos. There he was, batting for Amazon’s support for the cause. He wrangled with many users. To one particular customer, Bezos wrote:
“There have been a number of sickening but not surprising responses in my inbox since my last post. This sort of hate shouldn’t be allowed to hide in the shadows. It’s important to make it visible. This is just one example of the problem.
And, Dave, you’re the kind of customer I’m happy to lose.”
(See here for the customer’s note and Bezos’s response. Warning: Vile Language).
In my view, regardless of wherever you place yourself politically, it was the first time that Amazon had declared that it didn’t need a customer’s business. We were beginning to reach the limits of their customer obsession. Amazon (like Target, Budweiser, Blackrock, and others), whether they want it or not, is now a part of the vast conversation in the political landscape.
With size comes attention.
In 2021, Amazon halted law-enforcement use of its facial-recognition software, adding its voice to a growing chorus of companies, lawmakers, and civil rights advocates calling for greater regulation of the technology, issuing a temporary moratorium, that was later extended indefinitely.
From Amazon’s blog post:
We’re implementing a one-year moratorium on police use of Amazon’s facial recognition technology. We will continue to allow organizations like Thorn, the International Center for Missing and Exploited Children, and Marinus Analytics to use Amazon Rekognition to help rescue human trafficking victims and reunite missing children with their families.
We’ve advocated that governments should put in place stronger regulations to govern the ethical use of facial recognition technology, and in recent days, Congress appears ready to take on this challenge. We hope this one-year moratorium might give Congress enough time to implement appropriate rules, and we stand ready to help if requested.
There is some emerging research about the efficacy of identification systems and their implementation across various racial groups, including some research from my departmental colleagues. Perhaps, along with technological and operational constraints, scaling difficulties also emerge from cultural constraints. As Adam Smith wrote in the Theory of Moral Sentiments (1759), firms dealing with possessions and property, eventually head into personal rights, and see themselves as establishing justice (however defined), as part of their virtue.
The most sacred laws of justice, therefore, those whose violation seems to call loudest for vengeance and punishment, are the laws which guard the life and person of our neighbour; the next are those which guard his property and possessions; and last of all come those which guard what are called his personal rights, or what is due to him from the promises of others. (Theory of Moral Sentiments).
Until next Friday!