Two Worlds of AI

At this point almost everyone in the world is LLM pilled, but not necessarily Superintelligence pilled. There are only a few sceptics left who don't think today's AI will be economically and politically transformative: at least on the level of the Internet or the Personal Computer. But there's much less agreement on whether LLMs can keep scaling up to reach a significantly higher level of capability again, beyond the capabilities of today's experts.

One way to label this is that in one world LLMs are imitative. In another, LLMs are complete. Which world you think we're in radically changes what short term predictions you make on the impact of AI on growth, employment, business strategy or safety. A lot of talking past each other I see on issues around AI is when one person assumes we're in one world, and another the other.

In the imitative world, we agree that today's AIs have proved amazingly successful at learning ever more complex patterns. By investing in RL training runs and written examples in ever more granular skills, we are continuously increasing the economic footprint of the types of human workflows that AI tools can take over.

But crucially, in this world we believe that most of the training pressure in AIs is still going towards learning to imitate its training corpus - building on the original goal of predicting the next token. While LLMs can do some in-context learning, this is relatively limited - which is why they often fail in such sharp ways when they encounter a task outside their training distribution, still see drastically lower data efficiency in learning than humans and have yet to produce the kind of new creative breakthroughs you would expect from a similarly talented human.

While the transformer architecture allows some kind of world modelling, and the KV cache lets LLMs keep some sense of the state of the world, this is clearly where they are the most brittle. If we want to get past this, we will need some kind of new architecture, whether that be neuralese or continual learning or whatever you want to call it, that allows for much more fluid thinking and learning in real time.

In the complete world, by contrast, this is just one more example of human chauvinism. LLMs' capability for in-context learning and grokking already goes far beyond what we initially expected. If we just keep scaling them up to something closer to a human level of parameters, continue building out the context window and feeding in ever more examples, then at some point they will achieve a take off level of intelligence - or at least get to the point where they can solve any remaining architectural barriers themselves.

If you are in the imitative world, then LLMs may prove highly complementary to human workers - but like every seeming transformative technology before them, they are highly unlikely to lead to mass unemployment. As long as some bottleneck tasks remain that only humans can do, we will just shift more time towards people doing these.

If you are in the complete world, by contrast, almost by definition tautologically an AI can do everything a human can - not just in knowledge work, but you would have to assume very soon in the physical world as all robotics tasks get solved too.

This divide applies not just to individuals, but to companies too. If you are in the imitative world, then industries like law, finance and consulting may shift their focus - just as media had to respond to the rise of the Internet - but they are unlikely to disappear. Company value and moats will instead reorganise around the tasks and expertise that only a human can do. If you are in the complete world, there may be none of those left.

And then finally on safety: if you are in the imitative world then yes, you can ask an LLM to role play as an evil mastermind and that probably could be really, really dangerous. This is probably a big part of the reasons why we see LLMs misbehave and scheme today - it is a significant part of their training data, whether from fiction or (ironically) people worrying about AI safety. But you are not dealing with a fundamentally new kind of threat - there have always been bad actors - and AI is not going to completely kick over the game board. If an AI is fundamentally imitative, then at least some of what it has learned to imitate is human values.

If you are in the complete world, then well, the sky's the limit. We have no idea what new technologies an AI could create, what goals it will end up with or how much chaos it could create just through social deception and persuasion. On the flip side, neither do we have a good way of figuring out the upper bound on how much good an AI could create if it was aligned. (This is where people start to vaguely wave at curing all disease.)

At the moment, we are almost certainly in the imitative world. Sometime in the next 1 to 100 years we are likely to flip into the complete one. But before that eliding the difference between the two causes all sorts of confusion.