Complex systems emerge from simple rules
tecnica.substack.comOMG one must wonder about a system that produces this article and especially these comments. If you are interested in this area you might consider:
* Hidden Order by John Holland. Especially if you are technical, then read it from the perspective of "how would I build one of these"? (See RIP).
* Signs Of Life: How Complexity Pervades Biology. An overwhelming book. At the very least you should come out aware of how little we understand.
* What is money? Really. We all know the mythical person month perspective, but according to that New York City can not exist.
* The RIP routing protocol.
* A New Kind of Science.
Gestalts, created from complex adaptive systems, surround us. (Inside joke). I think this is an example of the old adage "I won't see it until I believe it" which is just a simplified version of Sapir Whorf.
[edit: add Wolfram's "A New Kind of Science" ]
Utterly banal. For one thing, to say that life supervenes upon chemistry which supervenes upon physics is actually taking the _opposite_ position than "life is emergent," at least to the extent that philosophers agree about what emergent means. I understand emergence to mean what Deacon means: the properties of the emergent system are independent from the properties of the system underneath it at least to the extent that you couldn't predict the behavior of the emergent system with just the physics, at least not without a lot of work and the specific initial conditions required. The idea being that many possible systems could serve as the basis for the emergent system and that the emergent system is some kind of set of self-reinforcing constraints imposed on the underlying system.
As it happens, I don't believe this argument if taken all the way. Sure, emergence has the property that, in some sense, one knows more about the emergent system if it is described at its own level rather than at the level of the underlying system (eg, if you tell me you are hungry I'm much more likely to correctly predict you will order pizza than if you give me a catalog of the state of all your neurons), but your behavior still supervenes upon the neurons inexorably. Even the constraints that maintain the emergent system ultimately are expressed in terms of the underlying system.
Your interpretation stumbled a bit.
Emergent is that produced/provided (as a potential) of a system which is greater than the sum of its parts.
Absolutely everything new you can do by arranging a few independent things for an outcome which could not have been achieved otherwise (literally creating potential in the universe) is "emergent."
So driving great distance and speed in metal chariots emerges from our technology for cars.
Flight emerges by airflow physics and the trimtab.
Life emerges by eating and crapping dense energy, propagating the continuity of existential being.
I guess dude, but that isn't a very useful definition and its not what philosophers are talking about.
I don't think this is a particular good definition. It would preclude emergent behavior in deterministic systems. You have to weaken the predictability somewhat. Maybe: if the system is more than "sum of it parts with perturbations on top". Example: An atomic system, the mass of the atom is close to the mass of the nucleus + the mass of the electrons. The fact that it's a bound system adds only a small perturbation on the total mass.
On the other hand: For quark systems, the total mass seems completely unrelated to the mass of the constituents. A pion is 2 quarks, 139 MeV. A proton has three, same type of quarks. Mass 938 MeV. The mass is emergent, from the interactions of the quarks and gluons. It's not predictable easily from the constituents.
Nevertheless, we can put QCD on the lattice and brute force it.
It is not clear to me if this is fundamental. I.e. if there are systems where you can't predict the behavior short of a full simulation. I suspect that there is a fundamental statement about nature here, similar to computability arguments. But I also think there are still a lot of techniques to develop to get closer.
Your confusion arises from the methodological mind (conventional theoretical physics) refusing to believe that state is potential bound, and potential unbound resolves to some{? Your need for true random} distribution, yet these normalize so not everything you swear by of potential and probability is universally true.
Emergence in some esotericly romantic specialized science means "the symbol derived." In existential Reality without confusing theoretical limits emergence is any new potential created by a system, whether planned that way or a surprise to you.
Granted, surprises are also "emergent."
> The mass is emergent, from the interactions of the quarks and gluons
I argue the mass emerges as a standing wave against the mass relationship with all other bound potential. A hyperdimensional shape [knot] causes constructive/destructive interference to whatever potential is bound (here miniscule yet accumulative.)
The spin characteristic of this hyperdimensional knot of essentially light. Will be such that it will have a discrete mass (as full spins are at the speed of light, half spins are dimensional vectors causing fundamental force interference.) this relationship between the speed of light and half spin vectors may account for mass as the particle comes to rest. So a light speed particle is slowed down to mass rest by spin disposition (which was formed in the particle collider of infinite space and time.)
A small bias in a standing wave has pseudo-deterministic influence in everything it interferes with.
All of existential reality emerges by constructive and destructive interference of otherwise uncommitted particles.
It is! I have studied this past ten thousand years of civilization, and have come to the conclusion that specialization is easy, and coharent generalizations with integrity are hard.
As soon as some perfection is found, it crumbles and reconstitutes in the minds of those who express and those who perceive.
Seeing "potential" as the underlying and the "things" (which we value) as meta symbolic liberates my definition of a dozen contingencies.
There is a great mystery, yet this is a very useful definition.
Emergent properties are those potential created by a system of which such potential does not exist in any part otherwise.
A product emerges from an assembly line.
All that other stuff you said must be scrapped off one meaning, and further distilled toward whichever mystery you actually want to solve.
... which may be consciousness or meaning or edge cognitive technology.
While the article is not particular in depth, it is mostly accurate and provides a mostly well intentioned introduction to the concept and how that could relate to emergent property of LLMs. I am curious why you think it was "utterly banal".
Additionally, abiogenesis is well accepted in academia as a prime example of emergence.
Abiogenesis is accepted, sure, but whether it constitutes an example of emergence isn't clear, at least by the definition I gave above, which is about supervenience. If you think that life is just chemistry which is just physics then life does not emerge from physics, it simply is physics and one needs only understand physics to understand life. If, on the other hand, you think that life has dynamical structures which are independent from and could not be predicted by someone who knew only the laws of physics, then it is a genuine case of emergence.
If you use "emergence" to mean "complicated stuff in a simple system" then I guess it is emergence, but that definition is philosophically useless or at the very least uncontroversial. It's trivially clear simple rules give rise to complex behaviors at this point. The question is the causal role of the roughly conserved elements of the complex system in question.
Look at it this way: life on Earth is composed primarily of C, H, O, N , and a number of trace atoms.
If life evolved here on Earth, then somehow, CHON+trace all self-organized into all of Earth life today, including us, and all we humans have done and will do in the future.
Now: say we could go back just before life evolved. Even with very very good data, and with whatever talent (AI, science, anything) and technology from today, would we be able to truly describe the emergence paths for those CHON+trace atoms?
Impossible. This would be far beyond our level of science and technology. We can't even do this for much simpler systems and shorter time horizons. You'd have to predict the emergence of life, the full properties and behavior of all forms of life in the last 4 billion years, and last but not least, humans and all that humans have done and thought since then.
Yet clearly nature emerged all this from perhaps small amounts of 11 elements or so. Complexity is one of the greatest unknowns in our present civilization.
One way of thinking is that the rules are designed by humans. The game is not a game until we establish the rules. At any point in time in this realm of life, if we try to establish some control, or try to create a frame of "simple" rules around it, it becomes complex.
So it doesn't matter if the ChatGPT is designed by simple rules, the minute we try to control the randomness, it becomes complex.
Most systems don’t have interesting stable states like Conway’s Game of Life, they either collapse or explode. You can verify this yourself by changing the rules of the Game of Life.
LLMs aren’t simple systems either, they are density estimators pretrained on terabytes of text, then fine tuned with human feedback. The simple fact that the author believes this suggests he doesn’t know much about LLMs.
Not a great analogy, since Game of Life in Game of Life didn't emerge, it was programmed by a human.
Arguably, the fact that the game of life was made is an emergent property of biology, and thus chemistry, and thus physics, etc.
Isn't that kinda the point? Conway designed the rules of the Game of Life, that was entirely his doing. But he didn't design any of the consequences of those rules—oscillators, gliders, glider guns, garden of edens, the fact that it's Turning complete. All of those exist as a consequence of the rules he did design, but nowhere are they specified in those rules.
Other people might be interested in Stephen Wolfram's approach to physics[0].