The impossibility of intelligence explosion

[ClaytonB]

François Chollet
Nov 27

The impossibility of intelligence explosion

...

Conclusions

The expansion of intelligence can only come from a co-evolution of brains (biological or digital), sensorimotor affordances, environment, and culture — not from merely tuning the gears of some brain in a jar, in isolation. Such a co-evolution has already been happening for eons, and will continue as intelligence moves to an increasingly digital substrate. No “intelligence explosion” will occur, as this process advances at a roughly linear pace.

Remember:

- Intelligence is situational — there is no such thing as general intelligence. Your brain is one piece in a broader system which includes your body, your environment, other humans, and culture as a whole.
- No system exists in a vacuum; any individual intelligence will always be both defined and limited by the context of its existence, by its environment. Currently, our environment, not our brain, is acting as the bottleneck to our intelligence.
- Human intelligence is largely externalized, contained not in our brain but in our civilization. We are our tools — our brains are modules in a cognitive system much larger than ourselves. A system that is already self-improving, and has been for a long time.
- Recursively self-improving systems, because of contingent bottlenecks, diminishing returns, and counter-reactions arising from the broader context in which they exist, cannot achieve exponential progress in practice. Empirically, they tend to display linear or sigmoidal improvement. In particular, this is the case for scientific progress — science being possibly the closest system to a recursively self-improving AI that we can observe.
- Recursive intelligence expansion is already happening — at the level of our civilization. It will keep happening in the age of AI, and it progresses at a roughly linear pace.

https://medium.com/@francois.chollet...n-5be4a9eda6ec

I agree with much of what Chollet has said here. The only asterisk I would add is that we cannot rule out arbitrarily large "hockey-sticks" in the growth of AI at any given point. During such a period (say, right after a major breakthrough) it might seem that AI has transcended human intelligence and completely left us in the dust. But I think that, eventually, we will catch up with it because we will be able to track along with progress in AI by enhancing our own abilities. Think of it like consulting a chess engine to play against a chess engine. This has been tried and even though chess engines are unconditionally stronger than any human player, when a human player works with a chess engine against another chess engine, the human+computer team is stronger. So, the real dangers of AI lie in the distortions that can be made about what AI actually can and cannot do.

[dannno]

Well, first women have to start $#@!ing more intelligent men.

[Lamp]

Well, first women have to start $#@!ing more intelligent men.

MMmmm doggystyle with Sheldon Cooper. I'm sure every woman would want that.

[dannno]

MMmmm doggystyle with Sheldon Cooper. I'm sure every woman would want that.

The world needs more Pennies.

[Lamp]

The world needs more Pennies.

I agree

[DamianTV]

I agree with much of what Chollet has said here. The only asterisk I would add is that we cannot rule out arbitrarily large "hockey-sticks" in the growth of AI at any given point. During such a period (say, right after a major breakthrough) it might seem that AI has transcended human intelligence and completely left us in the dust. But I think that, eventually, we will catch up with it because we will be able to track along with progress in AI by enhancing our own abilities. Think of it like consulting a chess engine to play against a chess engine. This has been tried and even though chess engines are unconditionally stronger than any human player, when a human player works with a chess engine against another chess engine, the human+computer team is stronger. So, the real dangers of AI lie in the distortions that can be made about what AI actually can and cannot do.

Sadly, I have a feeling there IS a hockey stick, but it is going the wrong way.

[heavenlyboy34]

relevant: