The Metamorphosis
The great Franz Kafka published “The Metamorphosis” in 1915. An allegorical tale about the transformation of a man from human to an insect, it is a story filled with the themes of change, maturity and alienation.
This (not so) short essay below is not an addendum to the original story. It is about AI and the Kafkaesque transformation we are currently undergoing. But are we? If we were to believe the foundation models, the transformation has already happened with the advent of LLMs and now all we need to wait for is that one awesome large language model that will achieve Artificial General Intelligence (AGI) and solve all our individual and enterprise use cases. Now whether it’s Open-weights (Llama, Mistral) or Closed-weights models (Open AI, Anthropic, Grok/X, Google), you pick. Because they are all unified in scaling-up pre and post training to achieve their underlying mission of a ‘one-size-fits-all’ model. In fact, these AGI aspirations are the reason why these frontier model labs are pushing the frontiers of funding itself. Even Ivanka Trump endorsed the race to the trillion dollar cluster recently.
It is worth comparing the current Gen AI revolution to that of the Computer industry in the late 1980s. Initially, the entire industry was built around vertical integration with the key players owning the entire stack from manufacturing chips and hardware all the way to selling them to the enterprises. However, with the invention of the microprocessor chips, the economics of mass production kicked in and suddenly manufacturing computers became very cost effective. This changed the basis of computing and the competition. Andy Grove, in his book ‘Only the Paranoid Survive’, wrote about this transformation of the Computer industry from a set of vertically integrated ‘silos’ like IBM, DEC, Sperry, and Wang to a large number of companies organized in horizontal ‘layers’ like the chip layer, the hardware layer, the operating system, the software layer etc. (See Figure below).
He called this invention of the chip a “strategic inflection point” because it was a 10x force from inception. Even though it was more of a ‘toy’ in the beginning – ‘unreliable and incapable of doing things pertinent to business processes’; a decade later that same force had revolutionized and reshaped the entire industry. To further quote Andy, he remarked that this decade of transition was like watching that computer generated image of a person on a screen ‘morphing’ from one face to another. One face imperceptibly dissolving and another simultaneously emerging to form shape. It wasn’t yet clear where to go from here, but the status quo had surely been upended.
It feels like we are going through a similar transition right now with Gen AI. OpenAI’s release of ChatGPT was an inflection point for this new technology and a 10x force of what is about to be. But it’s also still just a toy which likes to hallucinate, make up random stuff and miscount the ‘r’s in strawberry.
Ok, so going back to the Computer Industry… what happened next in that transition? Well, a lot. And that metamorphosis is well supported by the data. Going into the ‘80s, the Computer industry was vertically integrated and IBM was the largest player, accounting for more than half of the market share. Other key players included the Japanese firms, Hitachi and NEC, and the US based Hewlett Packard and DEC. (See Figure below. The NAICS Segments on the left refer to the different functional layers of the stack. However, as represented by the colored bars, the entire stack was controlled vertically by a few key players).
Pre microprocessor inflection, IBM had projected to become a $100 billion company by the end of the decade. But the opposite ensued as between 1985 and 1990, it lost a substantial market share as did DEC, National Cash Register, Sperry, Unisys and Wang Labs. Microsoft, which went public in 1986, broke into the top 5, followed by Intel. Packaged Software emerged as a new layer and now accounted for 10% of the industry’s total market value (Figure below).
Between 1990 and 1995, the industry map changed dramatically. Firstly, the total market value of the entire industry expanded substantially and the horizontal layers now accounted for three-quarters of the industry’s value. IBM continued its descent into the valley of death and Microsoft emerged as the largest company. New layers in the stack emerged and so did new players, including Cisco, Oracle, First Data Corp. and Micron.
By 2000, the verticals had entirely disappeared. IBM went through a major restructuring and was concentrated in systems, services and software. Cisco emerged as the top platform and new members entered the top tier arena, including EMC, Lucent Technologies, Dell, Taiwan Semiconductor, Juniper Networks and STMicroelectronics. The high flyers a decade ago like Hitachi, Toshiba, NEC, Compaq, and Sun Microsystems were all ranked outside the top 15 (by market cap).
In the words of Hemingway, it happened gradually then suddenly, but the metamorphosis was finally complete. Between 1980 and 2000, the vertical structure of the computer industry was replaced by a new horizontal industry structure built on a more modular technical architecture where different firms produced different functional components of the overall stack.
Similarly, we can use the Computer industry’s transition as a stencil and draw an outline of what our Gen AI transition might look like –
Overall industry size will expand substantially.
The vertical stack WILL become more horizontal.
As the AI stack gets built out, new layers will emerge that will glue the foundational layers with the application layer. For example – the inference layer is decoupling from the foundation model layer to emerge as its own stand alone layer.
New layers will make room for new players. Some of the new inference players include Groq and Cerebras.
Some p/layers will become obsolete.
As we await this outline of our Gen AI metamorphosis to take shape, the next few years are going to be like watching that computer generated image morphing from one face to another. One face imperceptibly dissolving and another simultaneously emerging to form shape.
