Settings

Theme

AI might not recursively self improve (part 2)

secondsight.dev

4 points by ffwd · 3 comments

Reader

2 threads
techpineapple

So, nothing about my mental model of the world thinks that AI recursively self-improving (limitlessly) is possible but wouldn't a basic sort of math-y version of this be that most improvements AI can find in itself would be linear, but the marginal cost of improvements would be exponential?

I get that none of this stuff is like solid, but recursive self-improvement feels similar to a perpetual motion machine.

Base assumptions about the fundamental nature of the world have changed in the past (I like some of the examples in `sapiens`, even arguing that the idea of being able to improve our environment as a species was not believed for most of history) so I could be wrong, but fast takeoff feels like a fantasy.

I also think about cookie clicker math. A new technological paradigm may give us exponential improvement, but we eventually hit the next exponential. i.e. when you buy a big upgrade in cookie clicker, it can start making your current exponential (say if you're at millions of cookies) really fast, the wheel spins really fast until you hit about 1 billion in which case the wheel slows down to a crawl.

darad

technically its not possible to have AI limitlessly improve just because nothing can hold infinity for to hold infinity you yourself have to be infinite(storage and data)

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection