Here are my predictions and thoughts on the current LLM (and GenAI) tech cycle. I think it would be useful to register my predictions to review later. These are ‘strong convictions, weakly held’.

Predictions

  1. LLMs will not be a useful basis for AGI.
    • Yann Lecun’s statement: “LLMs can do none of those or they can only do them in a very primitive way, and they don’t really understand the physical world. As a pass towards human-level intelligence, they’re missing essential components.”
  2. The main family of use cases for LLM will be as a new interface for interacting with computing.
    • “You now have a computer that will do what you tell it to do”.
    • Some aspects of computing will become more democratized, as was the case with GUIs.
    • Power users will not benefit so much from this (similar to GUI). Neophytes will benefit immensely, but technical skills growth will look different, and possibly many more engineers will have technical skills gaps as their careers develop.
  3. Hyperscalers are over-investing in AI, and at some point they will scale back their investments to match the actual benefit to their business. Accelerators will be re-purposed for other workloads, such as recommender systems or developing models in other domains.

Observations/Thoughts

Some other observations/opinions:

  • How models will continue to scale in the future is uncertain. A lot of short-term bets on LLMs are predicated on the models becoming good enough to do X within the next k years (before they run out of funding).
  • There is no moat for training foundation models, once the hurdles are met for data and compute (though this narrows down the field to hyperscalers and very well-funded startups). Foundation models are on the way to becoming commoditized.
  • Regulators are over-reacting (or there is already some regulatory capture) in a largely symbolic attempt to appear pro-active and up to speed with recent tech.
  • The recent AI boom is due to archs enabling unsupervised learning, along with high availability of parallel computing resources. Other applications which can utilize these accelerators for compute will also have their moment once investment in LLMs falters.