Chat-based Large Language Models replicate the mechanisms of a psychic’s con
softwarecrisis.devTo be honest, there have been occasions where I've been failing miserably for some time to come up with solutions to some sysadmin nature problems with my own systems and I've taken my problem straight to GPT-4 with a clear explanation of the issue alongside with any diagnostics I thought would make sense.
To my very surprise, GPT-4 did an astonishing job reasoning on the specifics of my system (aarch64 exotic setup, alpinelinux on asahi) and came back with a very specific on point list of suggestions which included the very solution as #1.
I've had it hold my hand many times while navigating relatively complex and niche systems like android smartphones with custom partitioning schemes booting linux and what have you and yes, it still was still, very reasonable, to say the least.
So to conclude, it has the ability to reason properly for systems and situations that it's not necessary trained in and displays the ability for coherent reasoning on specifics over things which at least for September 2021 were relatively unknown. I'm really wondering how far this thing needs to get in order for some people to admit its more than a model spitting a token next to another, or some type of mentalist doing an excellent job in hypnotising most of us into thinking it already displays incredible intelligence but its smoke and mirrors.
Papers on arXiv where people try prompt engineering systematically find that chatbots, by short-circuiting, gets the answer right from 30-90% of the time depending on if the problem is terribly difficult or relatively easy.
You’re very likely to get good results with the first example that comes to your head because it will be something conventional for which the short-circuiting gets the right answer. Once you sample the problem space fairly you find the glass is maybe half-full or half-empty but it’s clear to me that one of the competences the chatbot has is getting people to give it more credit than it deserves.
Can you share the log?
Oops, another blogger falling into the trap of not specifying how they define "intelligence" and then making a "no true scotsman" argument against their loose pre-existing beliefs.
If you're thinking about writing an article like this, please just define what you think intelligence is right at the top. That's the entirety of the discussion, the rest is fluff.
Also, as a society we need to minimize the amount of attention we give to debates over definitions. Once a discussion or political debate is reduced to a definitional issue, everyone starts talking past each other and forgets what the argument even is. (See discussions about the definitions of "life", "woman", "socialism", "capitalism", etc.) Words are lossy proxies to ideas, and they only matter insofar as they allow us to understand one another.
They need to go beyond defining intelligence. They need to operationalize intelligence.
The problems still are many. First, we've already operationalized intelligence, namely through IQ tests see Stanford-Binet Intelligence Scale, Universal Nonverbal Intelligence, Differential Ability Scales, Peabody Individual Achievement Test, Wechsler Individual Achievement Test, Wechsler Adult Intelligence Scale, &c. Even so, HN users will both point out the (in)validity of these but also the (in)validity with respect to applying them to AI.
The real problem is that intelligence is a socially defined phenomenon, as opposed to an essential metaphysical property. If we admit that, many of the definition foibles we have become irrelevant.
As someone who admittedly belongs more to the "AI believer" side, I find the vagueness of the training data increasingly frustrating.
The thing that impressed me most about LLMs so far is less the factual correctness or incorrectness of its output but the fact that it appears (!) to understand the instructions that are given. I.e., even if you give it an improbable and outlandish task ("write a poem about kernel debugging in the style of Edgar Allan Poe", "write a script for a Star Trek TNG episode in which Picard only talks in curse words "), it always gives a response which is a valid fulfillment of the task.
Of course it could be that the tasks weren't really as outlandish as they seemed and somewhere in the vast amounts of training data there was already a matching TNG fanfic which just needed some slight adjustments or something.
But those kinds of arguments essentially shift the black box from the model to the training data: Instead of claiming the model has magical powers of intelligence, now the training data magically already contains anything you could possibly ask for. I personally don't find that approach that much more rational that believing in some kind of AI consciousness (or fragments of it).
...but of course it could be. This is why I'd wish for foundation models with more controlled training data, so we can make more certain statements about which responses could be reasonably be pulled from the training data and which would be truly novel.
Intelligence need not be magical.
I suspect when it comes to training data, it may need to be general enough to allow the architecture a chance to learn the meta concept of "learning". Ie identify the latent gestalt within a text corpus that we most identify as "reasoning ability".
If the training data is not rich enough, then these more refined emergent abilities will not be discovered through our current algorithms/architecture. Maybe in the future when more efficient algorithms are found (we know the lower bound must be at least as efficient as our human brains for example) then we won't need as much/as rich data. Or use Multi modal data.
From what we're seeing I believe we can already discount the tainted training data as likely hypothesis, and trend to the suspicion there is something deeper at play.
For instance, what if LLMs through pattern recognition of text alone may have built a coherent enough world model that it yields answers indistinguishable from human intelligence?
Nothing about that seems improbable from current neuroscience theories https://en.m.wikipedia.org/wiki/Predictive_coding
It may also suggest there to be nothing special functionally about the human brain; the ability for a system to recursively identify, model, and remix concepts may be sufficient to give rise to the phenomenology we know as intelligence.
Qualia, goals, "feelings", that sounds more nebulous and complicated to define and assess though.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
In a parallel thread hardcore scientists struggle to understand a 100 neuron worm, while here less hardcore scientists proclaim they've understood nuances of a human brain.
Note, that there is a rapid rise of the "mechanical consciousness" dogma. Some very smart individuals are so impressed by it that rather than doubting the existence of intelligence in LLM AI, they've started thinking that they themselves might be machines! From there it's one step to giving machines rights on par with humans. The dogma is very powerful.
Scientism is a hell of a drug.
Since it is objectively true that LLMs are predictive text engines first and foremost, it leads me to the hypothesis that the intelligence displayed by them, and by association, perhaps humans as well, is in fact imbedded into memetic structures themselves in some kind of n-dimensional probability matrix.
In the same way that an arbitrarily detailed simulation could in theory be made into a “make your own adventure”lookup table, where the next “page” (screen bitmap) was determined by the “control” inputs, the underpinnings of reason could easily be contained in a mundane and deceptively static medium such as a kind of multidimensionally linked list structure.
It could be that neural networks inherently gravitate towards the processing of symbolic grammar (sequences of “symbols”) and that the ordered complexity inherent in arbitrarily high dimensional interrelations of these symbols in human memetic structures is sufficient to create the process that we think of as reasoning or even sentience.
While I definitely struggle to intuit this interpretation from an emotional standpoint, the sheer multitude of states possible inside such a system are sufficient to appear infinite and therefore intrinsically dynamic, and I fail to find evidence that they could not be instead developed from a static data structure .
If there is a grain of truth to this hypothesis it would fundamentally change the philosophical landscape not only around LLMs but also regarding intelligence itself, the implication being not that LLMs might be intelligent, but rather that biological intelligence might in fact derive its behavior from iterating over multidimensional matrixes of learned data, and that human intelligence owes much more to culture (a vastly expanded data set) than we may have previously imagined.
I'd love to get an AI philosopher to address the interesting thought experiment here. I think this really needs to get at the truth - paraphrasing Stephen Wolfram - of whether language is actually much simpler than we've all assumed all along.
> There are two possible explanations for this effect:
> 1. The tech industry has accidentally invented the initial stages a completely new kind of mind, based on completely unknown principles, using completely unknown processes that have no parallel in the biological world.
> 2. The intelligence illusion is in the mind of the user and not in the LLM itself.
Great, now write 10k words more, but this time about the psychology of your unwillingness to change from (2) to (1) when the facts changed.